Encrypted incremental backups for BOA with Duplicity

2 minutters læsning

After reading Encrypted Incremental Backups With Duplicity on Amazon S3 I decided that I wanted to try Duplicity to backup the data of my BOA-server.

Getting started

First off course, you need to install duplicity on the server:


apt-get install duplicity

Duplictiy makes use of gpg and handles both private/public key pairs and a paraphrase. I just chose to use a passphrase.

I did not want to use Amazon but just scp to another server, so I modified the script a bit and changed the name to dupthat.sh:

<pre type="shell">#!/bin/bash

# user info
USER=USERNAME
PASSWORD=SECRET
export PASSPHRASE=GPG-PARAPHRASE

# server info
PROTOCOL="scp"
SERVER=SERVERNAME
DIRECTORY=/home/${USER}

# directories, space separated
SOURCE="/data/disk/o1/backups /data/disk/o2/backups /data/disk/o3/backups"
BUCKET=${PROTOCOL}://${USER}:${PASSWORD}@${SERVER}${DIRECTORY}
LOGFILE=/var/log/duplicity.log
# set email to receive a backup report
EMAIL=""

backup() {
  INCLUDE=""
  for CDIR in $SOURCE
  do
    TMP=" --include  ${CDIR}"
    INCLUDE=${INCLUDE}${TMP}
  done
  # perform an incremental backup to root, include directories, exclude everything else, / as reference.
  duplicity --full-if-older-than 30D $INCLUDE --exclude '**' / $BUCKET > $LOGFILE
  if [ -n "$EMAIL" ]; then
    mail -s "backup report" $EMAIL < $LOGFILE
  fi
}

list() {
  duplicity list-current-files $BUCKET
}

restore() {
  if [ $# = 2 ]; then
    duplicity restore --file-to-restore $1 $BUCKET $2
  else
    duplicity restore --file-to-restore $1 --time $2 $BUCKET $3
  fi
}

status() {
  duplicity collection-status $BUCKET
}

if [ "$1" = "backup" ]; then
  backup
elif [ "$1" = "list" ]; then
  list
elif [ "$1" = "restore" ]; then
  if [ $# = 3 ]; then
    restore $2 $3
  else
    restore $2 $3 $4
  fi
elif [ "$1" = "status" ]; then
  status
else
  echo "
  dupthat - manage duplicity backup
  
  USAGE:
  
  ./dupthat.sh backup 
  ./dupthat.sh list
  ./dupthat.sh status
  ./dupthat.sh restore file [time] dest
  "
fi

EXPORT PASSPHRASE=

Using the script

Put the script on the server, and modify the settings in the top of the file.

Backup


sh dupthat.sh backup

List/status


sh dupthat.sh list
sh dupthat.sh status

Restore

Do not prepend a slash when restoring files.

Restoring a single file to /tmp


sh dupthat.sh restore home/username/somefile tmp/somefile

Restoring an older version of a directory to tmp (interval or full date)


sh dupthat.sh restore home/username/directory 1D3h5s tmp/directory
sh dupthat.sh restore home/username/directory 2012/7/5 tmp/directory

All credits off course goes to the original article Encrypted Incremental Backups With Duplicity on Amazon S3. —————————————————————————————————————————————————————————————————————————————

Using it to backup BOA

I have enabled automatic backups and backup garbage collection under experimental on admin/hosting/features.

Now I want to backup user folders with the backup in, so I put the following into cron


30 03 * * * sh /root/dupthat.sh backup

Improvements? ————-

How do you back up your BOA-server? Do you back up other stuff? Which tools do you use?

Kommentarer

Daniele

Just to say that something has changed in how simplicity is managing the s3 protocol and also in case you’re using an s3 european there is one more parameter you have to add, so to accomplish the whole backup thing you have to add these two parameters to simplicity line: <pre style="margin-top: 0px; margin-bottom: 10px; padding: 5px; border: 0px; font-size: 14px; vertical-align: baseline; background-color: rgb(238, 238, 238); font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; overflow: auto; width: auto; max-height: 600px; color: rgb(0, 0, 0); line-height: 18px;"> `duplicity --s3-use-new-style` --s3-european-buckets

Paulius Pazdrazdys

Hey Lars, Although this is Danish site, I’ll write in English before I learn Danish language. Thanks for the great post. I think BOA is lacking some manual (though it’s out of project scope) on doing manual backups with duplicity (or any other tool) - this post seems to fill that gap nicely. Do you have experience when using S3 as remote location? Do you adjust volume sizes, instead of default 25MB to maby 250MB? What about backing up large files folder, which is only symlinked inside drupal site (these files are not showing up in backup created in aegir) ? Thanks again for sharing this one, this helped me a lot.

Skriv en kommentar

Din e-mail bliver ikke offentliggjort. Obligatoriske felter er markeret *

Indlæser...