System Admin

Tutoriales para IT Admins.

Scripts Backup Automatico postgress con pg_dump
http://system-admin.es

Author : Carlos Cortes Zabala

Si queremos tener una script que automatice el backup de todas nuestras bases de datos postgress
utilizando pg_dump en vez de pg_dumpall, para poder así recuperar cada base de datos por separado en
un entorno MAC, y tener un backup Full y 1 por cada base de datos.

Lo primero que debemos hacer es añadir al $PATH de macos la ruta al /bin de postgress, para ello
podemos ejecutar la siguiente orden.

sudo mkdir -p /etc/paths.d && echo /Library/PostgreSQL/9.6/bin | tee /etc/path

s.d/postgresapp

modificando la ruta /library/postgress a donde tenemos nuestra instalación(versión) de postgress.

Ahora, para que no pida el password todo el rato las scripts, deberemos crear el fichero .pgpass en el
home del usuario que va a ejecutar la script, este fichero es consultado al ejecutar scripts cuando se
utiliza postgress para que no solicite la clave, debera tener unos permisos EXACTOS 0600 o no se podrá
ejecutar y contener la información en el siguiente formato.

hostname:port:database:username:password

mas información sobre esto aquí : informacion pgpass

Así si queremos que el password sirva para cualquier puerto, cualquier base de datos y cualquier usuario,
pero solo de forma local, deberemos editarlo de la siguiente forma.

localhost:*:*:*:PASSWORD

1/6
 System Admin
Tutoriales para IT Admins.
modificando password por nuestro pass de postgres.
http://system-admin.es

para ello

vi .pgpass

añadimos la configuración localhost:*:*:*:PASSWORD

chmod 0600 .pgpass

Ahora que ya tenemos solucionado las llamadas a los binarios, y el password, vamos con las scripts.

La fuente de las scripts es la siguiente : FUENTE

Si bien la tenéis abajo del todo, listo para copiar y pegar en vuestro sistema.

Esta scripts se compone de 1 archivo de configuración

pg_backup.config

y dos scripts, dependiendo de si queremos que rote los backups borrando los anteriores o no, basado
siempre en el fichero de configuración.

una vez editado el archivo de configuración pg_backup.config a gusto.

vi pg_backup.config

ahora deberemos añadir la siguiente linea en las dos scripts .sh para que cuando lo ejecutemos desde
crontab, encuentre los binarios necesarios.
PATH=/Library/PostgreSQL/9.6/bin/:/usr/local/bin:/usr/local/sbin:~/bin:/usr/bin:/bin:/usr/sbin:/sbin

para ello

vi pg_backup_rotated.sh && vi pg_backup.sh

y añadir la linea PATH debajo justo de #!/bin/bash

como ejemplo quedaría de la siguiente forma

#!/bin/bash PATH=/Library/PostgreSQL/9.6/bin/:/usr/local/bin:/usr/local/sbin:
~/bin:/usr/bin:/bin:/usr/sbin:/sbin ########################### ####### LOAD
CONFIG ####### ###########################

2/6
 System Admin
Tutoriales para IT Admins.
Por ultimo lo añadimos al crontab de la maquina en el horario que queramos, 1 vez al día y realizara el
http://system-admin.es
backup según tengamos configurado el archivo de configuración.

recordar que podemos crear un archivo de log, simplemente haciendo tubería desde el crontab,quedando
el crontab por ejemplo de la siguiente forma.

00 06 * * * /var/root/pg_backup_rotated.sh > /var/root/error.log

Esto ejecutara la script a las 6 de la mañana todos los días.

También podemos hacer que nos lo mande por correo añadiendo lo siguiente en el crontab

30 06 * * * cat /var/root/error.log | mailx -s "BACKUP_postgress_DB" "admin@do

minio.com"

Esta orden hace que mande un mail a admin@dominio.com a las 6:30 de la mañana con el fichero de log.

scripts necesarias

pg_backup.config

############################## ## POSTGRESQL BACKUP CONFIG ## ##############

################ # Optional system user to run backups as. If the user th
e script is running as doesn't match this # the script terminates. Leave bla
nk to skip check. BACKUP_USER= # Optional hostname to adhere to pg_hba po
licies. Will default to "localhost" if none specified. HOSTNAME= # Optio
nal username to connect to database as. Will default to "postgres" if none sp
ecified. USERNAME= # This dir will be created if it doesn't exist. This
must be writable by the user the script is # running as. BACKUP_DIR=/home/ba
ckups/database/postgresql/ # List of strings to match against in database
name, separated by space or comma, for which we only # wish to keep a backup
of the schema, not the data. Any database names which contain any of these #
values will be considered candidates. (e.g. "system_log" will match "dev_syste
m_log_2010-01") SCHEMA_ONLY_LIST="" # Will produce a custom-format backup
if set to "yes" ENABLE_CUSTOM_BACKUPS=yes # Will produce a gzipped plain
-format backup if set to "yes" ENABLE_PLAIN_BACKUPS=yes # Will produce gz
ipped sql file containing the cluster globals, like users and passwords, if se
t to "yes" ENABLE_GLOBALS_BACKUPS=yes #### SETTINGS FOR ROTATED BACKUP
S #### # Which day to take the weekly backup from (1-7 = Monday-Sunday) D
AY_OF_WEEK_TO_KEEP=5 # Number of days to keep daily backups DAYS_TO_KEEP=
7 # How many weeks to keep weekly backups WEEKS_TO_KEEP=5 ###########
###########################

3/6
 System Admin
Tutoriales para IT Admins.
http://system-admin.es

pg_backup.sh

#!/bin/bash ########################### ####### LOAD CONFIG ####### ####

####################### while [ $# -gt 0 ]; do case $1 in
-c) if [ -r "$2" ]; then
source "$2" shift 2
else ${ECHO} "Unreadable confi
g file \"$2\"" 1>&2 exit 1
fi ;; *)
${ECHO} "Unknown Option \"$1\"" 1>&2 exit 2
;; esac done if [ $# = 0 ]; then S
CRIPTPATH=$(cd ${0%/*} && pwd -P) source $SCRIPTPATH/pg_backup.config
fi; ########################### #### PRE-BACKUP CHECKS #### ##########
################# # Make sure we're running as the required backup user i
f [ "$BACKUP_USER" != "" -a "$(id -un)" != "$BACKUP_USER" ]; then echo "This
script must be run as $BACKUP_USER. Exiting." 1>&2 exit 1; fi; ####
####################### ### INITIALISE DEFAULTS ### ########################
### if [ ! $HOSTNAME ]; then HOSTNAME="localhost" fi; if [ ! $USERN
AME ]; then USERNAME="postgres" fi; ########################### ###
# START THE BACKUPS #### ########################### FINAL_BACKUP_DIR=
$BACKUP_DIR"`date +\%Y-\%m-\%d`/" echo "Making backup directory in $FINAL_
BACKUP_DIR" if ! mkdir -p $FINAL_BACKUP_DIR; then echo "Cannot create ba
ckup directory in $FINAL_BACKUP_DIR. Go and fix it!" 1>&2 exit 1; fi;
####################### ### GLOBALS BACKUPS ### #######################
echo -e "\n\nPerforming globals backup" echo -e "--------------------------
------------------\n" if [ $ENABLE_GLOBALS_BACKUPS = "yes" ] then
echo "Globals backup" if ! pg_dumpall -g -h "$HOSTNAME" -U "$USE
RNAME" | gzip > $FINAL_BACKUP_DIR"globals".sql.gz.in_progress; then
echo "[!!ERROR!!] Failed to produce globals backup" 1>&2 else
mv $FINAL_BACKUP_DIR"globals".sql.gz.in_progress $FINAL_BACKU
P_DIR"globals".sql.gz fi else echo "None" fi #############
############## ### SCHEMA-ONLY BACKUPS ### ########################### f
or SCHEMA_ONLY_DB in ${SCHEMA_ONLY_LIST//,/ } do SCHEMA_ONLY_CLAUSE="$SCHEM
A_ONLY_CLAUSE or datname ~ '$SCHEMA_ONLY_DB'" done SCHEMA_ONLY_QUERY="sel
ect datname from pg_database where false $SCHEMA_ONLY_CLAUSE order by datname;
" echo -e "\n\nPerforming schema-only backups" echo -e "-----------------
---------------------------\n" SCHEMA_ONLY_DB_LIST=`psql -h "$HOSTNAME" -U
"$USERNAME" -At -c "$SCHEMA_ONLY_QUERY" postgres` echo -e "The following
databases were matched for schema-only backup:\n${SCHEMA_ONLY_DB_LIST}\n"
for DATABASE in $SCHEMA_ONLY_DB_LIST do echo "Schema-only backup of $DATABA
SE" if ! pg_dump -Fp -s -h "$HOSTNAME" -U "$USERNAME" "$DATABASE" | gzip
> $FINAL_BACKUP_DIR"$DATABASE"_SCHEMA.sql.gz.in_progress; then echo "[!!ERR
OR!!] Failed to backup database schema of $DATABASE" 1>&2 else mv $FINAL_
BACKUP_DIR"$DATABASE"_SCHEMA.sql.gz.in_progress $FINAL_BACKUP_DIR"$DATABASE"_S
CHEMA.sql.gz fi done ########################### ###### FULL BACKUP
S ####### ########################### for SCHEMA_ONLY_DB in ${SCHEMA_ONLY
_LIST//,/ } do EXCLUDE_SCHEMA_ONLY_CLAUSE="$EXCLUDE_SCHEMA_ONLY_CLAUSE and
datname !~ '$SCHEMA_ONLY_DB'" done FULL_BACKUP_QUERY="select datname from

4/6
 System Admin
Tutoriales para IT Admins.
pg_database where not datistemplate and datallowconn $EXCLUDE_SCHEMA_ONLY_CLA
http://system-admin.es
USE order by datname;" echo -e "\n\nPerforming full backups" echo -e "---
-----------------------------------------\n" for DATABASE in `psql -h "$HO
STNAME" -U "$USERNAME" -At -c "$FULL_BACKUP_QUERY" postgres` do if [ $ENABL
E_PLAIN_BACKUPS = "yes" ] then echo "Plain backup of $DATABASE" if
! pg_dump -Fp -h "$HOSTNAME" -U "$USERNAME" "$DATABASE" | gzip > $FINAL_BACKUP
_DIR"$DATABASE".sql.gz.in_progress; then echo "[!!ERROR!!] Failed to produ
ce plain backup database $DATABASE" 1>&2 else mv $FINAL_BACKUP_DIR"$DAT
ABASE".sql.gz.in_progress $FINAL_BACKUP_DIR"$DATABASE".sql.gz fi fi
if [ $ENABLE_CUSTOM_BACKUPS = "yes" ] then echo "Custom backup of $DATABA
SE" if ! pg_dump -Fc -h "$HOSTNAME" -U "$USERNAME" "$DATABASE" -f $FINAL
_BACKUP_DIR"$DATABASE".custom.in_progress; then echo "[!!ERROR!!] Failed t
o produce custom backup database $DATABASE" 1>&2 else mv $FINAL_BACKUP_
DIR"$DATABASE".custom.in_progress $FINAL_BACKUP_DIR"$DATABASE".custom fi
fi done echo -e "\nAll database backups complete!"

pg_backup_rotated.sh

#!/bin/bash ########################### ####### LOAD CONFIG ####### ####

####################### while [ $# -gt 0 ]; do case $1 in
-c) CONFIG_FILE_PATH="$2"
shift 2 ;; *)
${ECHO} "Unknown Option \"$1\"" 1>&2 exit
2 ;; esac done if [ -z $CONFIG_FILE_PAT
H ] ; then SCRIPTPATH=$(cd ${0%/*} && pwd -P) CONFIG_FILE_PA
TH="${SCRIPTPATH}/pg_backup.config" fi if [ ! -r ${CONFIG_FILE_PATH} ] ;
then echo "Could not load config file from ${CONFIG_FILE_PATH}" 1>&2
exit 1 fi source "${CONFIG_FILE_PATH}" #####################
###### #### PRE-BACKUP CHECKS #### ########################### # Make su
re we're running as the required backup user if [ "$BACKUP_USER" != "" -a "$(
id -un)" != "$BACKUP_USER" ] ; then echo "This script must be run as $BACKUP
_USER. Exiting." 1>&2 exit 1 fi ########################### ### INI
TIALISE DEFAULTS ### ########################### if [ ! $HOSTNAME ]; then
HOSTNAME="localhost" fi; if [ ! $USERNAME ]; then USERNAME="postgres
" fi; ########################### #### START THE BACKUPS #### ######
##################### function perform_backups() { SUFFIX=$1 FINAL_BA
CKUP_DIR=$BACKUP_DIR"`date +\%Y-\%m-\%d`$SUFFIX/" echo "Making backup dir
ectory in $FINAL_BACKUP_DIR" if ! mkdir -p $FINAL_BACKUP_DIR; then ech
o "Cannot create backup directory in $FINAL_BACKUP_DIR. Go and fix it!" 1>&2
exit 1; fi; ####################### ### GLOBALS BACKUPS ### #####
################## echo -e "\n\nPerforming globals backup" echo -e "---
-----------------------------------------\n" if [ $ENABLE_GLOBALS_BACKUPS
= "yes" ] then echo "Globals backup" if ! pg_dumpall -g -h
"$HOSTNAME" -U "$USERNAME" | gzip > $FINAL_BACKUP_DIR"globals".sql.gz.in_prog
ress; then echo "[!!ERROR!!] Failed to produce globals backup"
1>&2 else mv $FINAL_BACKUP_DIR"globals".sql.gz.in_progre
ss $FINAL_BACKUP_DIR"globals".sql.gz fi else echo "None" fi
########################### ### SCHEMA-ONLY BACKUPS ### ##############

5/6
 System Admin
Tutoriales para IT Admins.
#############
http://system-admin.es for SCHEMA_ONLY_DB in ${SCHEMA_ONLY_LIST//,/ } do
SCHEMA_ONLY_CLAUSE="$SCHEMA_ONLY_CLAUSE or datname ~ '$SCHEMA_ONLY_DB'" d
one SCHEMA_ONLY_QUERY="select datname from pg_database where false $SCHEM
A_ONLY_CLAUSE order by datname;" echo -e "\n\nPerforming schema-only back
ups" echo -e "--------------------------------------------\n" SCHEMA_ON
LY_DB_LIST=`psql -h "$HOSTNAME" -U "$USERNAME" -At -c "$SCHEMA_ONLY_QUERY" pos
tgres` echo -e "The following databases were matched for schema-only back
up:\n${SCHEMA_ONLY_DB_LIST}\n" for DATABASE in $SCHEMA_ONLY_DB_LIST do
echo "Schema-only backup of $DATABASE" if ! pg_dump -Fp
-s -h "$HOSTNAME" -U "$USERNAME" "$DATABASE" | gzip > $FINAL_BACKUP_DIR"$DATA
BASE"_SCHEMA.sql.gz.in_progress; then echo "[!!ERROR!!] Fail
ed to backup database schema of $DATABASE" 1>&2 else
mv $FINAL_BACKUP_DIR"$DATABASE"_SCHEMA.sql.gz.in_progress $FINAL_BACKUP_DIR
"$DATABASE"_SCHEMA.sql.gz fi done ########################
### ###### FULL BACKUPS ####### ########################### for SCHEM
A_ONLY_DB in ${SCHEMA_ONLY_LIST//,/ } do EXCLUDE_SCHEMA_ONLY_CLAUSE="$EXC
LUDE_SCHEMA_ONLY_CLAUSE and datname !~ '$SCHEMA_ONLY_DB'" done FULL_BAC
KUP_QUERY="select datname from pg_database where not datistemplate and datallo
wconn $EXCLUDE_SCHEMA_ONLY_CLAUSE order by datname;" echo -e "\n\nPerform
ing full backups" echo -e "--------------------------------------------\n"
for DATABASE in `psql -h "$HOSTNAME" -U "$USERNAME" -At -c "$FULL_BACKUP_Q
UERY" postgres` do if [ $ENABLE_PLAIN_BACKUPS = "yes" ] then echo
"Plain backup of $DATABASE" if ! pg_dump -Fp -h "$HOSTNAME" -U "$USERNA
ME" "$DATABASE" | gzip > $FINAL_BACKUP_DIR"$DATABASE".sql.gz.in_progress; then
echo "[!!ERROR!!] Failed to produce plain backup database $DATABASE" 1>&
2 else mv $FINAL_BACKUP_DIR"$DATABASE".sql.gz.in_progress $FINAL_BACK
UP_DIR"$DATABASE".sql.gz fi fi if [ $ENABLE_CUSTOM_BACKUPS = "yes
" ] then echo "Custom backup of $DATABASE" if ! pg_dump -Fc -h "
$HOSTNAME" -U "$USERNAME" "$DATABASE" -f $FINAL_BACKUP_DIR"$DATABASE".custom.i
n_progress; then echo "[!!ERROR!!] Failed to produce custom backup databa
se $DATABASE" else mv $FINAL_BACKUP_DIR"$DATABASE".custom.in_progress
$FINAL_BACKUP_DIR"$DATABASE".custom fi fi done echo -e "\nAl
l database backups complete!" } # MONTHLY BACKUPS DAY_OF_MONTH=`date
+%d` if [ $DAY_OF_MONTH -eq 1 ]; then # Delete all expired monthly dire
ctories find $BACKUP_DIR -maxdepth 1 -name "*-monthly" -exec rm -rf '{}' ';'
perform_backups "-monthly" exit 0; fi # WEEKLY BACKUPS DAY
_OF_WEEK=`date +%u` #1-7 (Monday-Sunday) EXPIRED_DAYS=`expr $((($WEEKS_TO_KEE
P * 7) + 1))` if [ $DAY_OF_WEEK = $DAY_OF_WEEK_TO_KEEP ]; then # Delete
all expired weekly directories find $BACKUP_DIR -maxdepth 1 -mtime +$EXPIRE
D_DAYS -name "*-weekly" -exec rm -rf '{}' ';' perform_backups "-weekly"
exit 0; fi # DAILY BACKUPS # Delete daily backups 7 days old or m
ore find $BACKUP_DIR -maxdepth 1 -mtime +$DAYS_TO_KEEP -name "*-daily" -exec
rm -rf '{}' ';' perform_backups "-daily"

Y LISTO!

Agur!

Powered by TCPDF (www.tcpdf.org)

6/6