Docker Volume Backup

Backup Docker container volumes to Filebase using Docker Volume Backup.

What is Docker Volume Backup?

Docker Volume Backup is a Docker image that can be used as a sidecar container to any existing Docker setup. This sidecar container can be configured for recurring or one time backups of Docker container volumes to any S3-compatible storage. It can also be configured to periodically delete old backups and send email notifications of backup failures.

Prerequisites:

• Download and Install Docker and Docker-Compose.

• Sign up for a free Filebase account.

• Have your Filebase Access and Secret Keys. Learn how to view your access keys here.

• Create a Filebase Bucket. Learn how to create a bucket here.

Configure Recurring Backups In a docker-compose File

The following is a sample configuration for a docker-compose.yml file to add a backup service to your compose setup and mount the volumes you would like to see backed up:

version: '3'

services:
  volume-consumer:
    build:
      context: ./my-container
    volumes:
      - data:/var/my-container
    labels:
      # This means the container will be stopped during backup to ensure backup integrity.
      - docker-volume-backup.stop-during-backup=true

  backup:
    image: offen/docker-volume-backup:latest
    restart: always
		environment:
				AWS_S3_BUCKET_NAME="filebase-bucket-name"
				AWS_ACCESS_KEY_ID="Filebase-Access-Key"
				AWS_SECRET_ACCESS_KEY="Filebase-Secret-Key"
				AWS_ENDPOINT="s3.filebase.com"
				AWS_ENDPOINT_PROTO="https"
   
    volumes:
      - data:/backup/my-app-backup:ro
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`.
      # You can omit this if you do not want to keep local backups.
      - /path/to/local_backups:/archive
volumes:
  data:

env_file template:

Fill out the following environment variable template and save it as backup.env in the directory of your docker-compose file. If you use this method, you can replace the environment section in your docker-compose file with env-file: ./backup.env.

########### BACKUP SCHEDULE

# Backups run on the given cron schedule in `busybox` flavor. If no
# value is set, `@daily` will be used. If you do not want the cron
# to ever run, use `0 0 5 31 2 ?`.

# BACKUP_CRON_EXPRESSION="0 2 * * *"

# The name of the backup file including the `.tar.gz` extension. The default results
# in filenames like `backup-2021-08-29T04-00-00.tar.gz`.

# BACKUP_FILENAME="backup-%Y-%m-%dT%H-%M-%S.tar.gz"

# Whether to copy the content of the backup folder before creating the tar archive.
# In the rare scenario where the content of the source backup volume is continuously
# updating, but we do not wish to stop the container while performing the backup,
# This setting can be used to ensure the integrity of the tar.gz file.

# BACKUP_FROM_SNAPSHOT="false"

########### BACKUP STORAGE

# Filebase bucket name
# AWS_S3_BUCKET_NAME="filebase-bucket-name"

# Filebase credentials
# AWS_ACCESS_KEY_ID="Filebase-Access-Key"
# AWS_SECRET_ACCESS_KEY="Filebase-Secret-Key"

# Filebase endpoint 
# AWS_ENDPOINT="s3.filebase.com"

# Filebase only supports the HTTPS protocol.
# AWS_ENDPOINT_PROTO="https"

# Setting this variable to `true` will disable verification of
# SSL certificates. You shouldn't use this unless you use self-signed
# certificates for your remote storage backend. This can only be used
# when AWS_ENDPOINT_PROTO is set to `https`.

# AWS_ENDPOINT_INSECURE="true"

# In addition to storing backups remotely, you can also keep local copies.
# Pass a container-local path to store your backups if needed. You also need to
# mount a local folder or Docker volume into that location (`/archive`
# by default) when running the container. In case the specified directory does
# not exist (nothing is mounted) in the container when the backup is running,
# local backups will be skipped. Local paths are also subject to pruning of
# old backups as defined below.

# BACKUP_ARCHIVE="/archive"

########### BACKUP PRUNING

# **IMPORTANT, PLEASE READ THIS BEFORE USING THIS FEATURE**:
# The mechanism used for pruning old backups is not very sophisticated
# and applies its rules to **all files in the target directory** by default,
# which means that if you are storing your backups next to other files,
# these might become subject to deletion too. When using this option
# make sure the backup files are stored in a directory used exclusively
# for such files, or to configure BACKUP_PRUNING_PREFIX to limit
# removal to certain files.

# Define this value to enable automatic rotation of old backups. The value
# declares the number of days for which a backup is kept.

# BACKUP_RETENTION_DAYS="7"

# In case the duration a backup takes fluctuates noticeably in your setup
# you can adjust this setting to make sure there are no race conditions
# between the backup finishing and the rotation not deleting backups that
# sit on the edge of the time window. Set this value to a duration
# that is expected to be bigger than the maximum difference of backups.
# Valid values have a suffix of (s)econds, (m)inutes or (h)ours. By default,
# one minute is used.

# BACKUP_PRUNING_LEEWAY="1m"

# In case your target bucket or directory contains other files than the ones
# managed by this container, you can limit the scope of rotation by setting
# a prefix value. This would usually be the non-parameterized part of your
# BACKUP_FILENAME. E.g. if BACKUP_FILENAME is `db-backup-%Y-%m-%dT%H-%M-%S.tar.gz`,
# you can set BACKUP_PRUNING_PREFIX to `db-backup-` and make sure
# unrelated files are not affected by the rotation mechanism.

# BACKUP_PRUNING_PREFIX="backup-"

########### BACKUP ENCRYPTION

# Backups can be encrypted using gpg in case a passphrase is given.

# GPG_PASSPHRASE="Passphrase"

########### STOPPING CONTAINERS DURING BACKUP

# Containers can be stopped by applying a
# `docker-volume-backup.stop-during-backup` label. By default, all containers
# that are labeled with `true` will be stopped. If you need more fine grained
# control (e.g. when running multiple containers based on this image), you can
# override this default by specifying a different value here.

# BACKUP_STOP_CONTAINER_LABEL="service1"

########### EMAIL NOTIFICATIONS ON FAILED BACKUP RUNS

# In case SMTP credentials are provided, notification emails can be sent out on
# failed backup runs. These emails will contain the start time, the error
# message and all log output prior to the failure.

# The recipient(s) of the notification. Supply a comma separated list
# of addresses if you want to notify multiple recipients. If this is
# not set, no emails will be sent.

# EMAIL_NOTIFICATION_RECIPIENT="[email protected]"

# The "From" header of the sent email. Defaults to `noreply@nohost`.

# EMAIL_NOTIFICATION_SENDER="[email protected]"

# Configuration and credentials for the SMTP server to be used.
# EMAIL_SMTP_PORT defaults to 587.

# EMAIL_SMTP_HOST="posteo.de"
# EMAIL_SMTP_PASSWORD="SMTP-password"
# EMAIL_SMTP_USERNAME="[email protected]"
# EMAIL_SMTP_PORT="SMTP-port"

One Time Backups Using Docker CLI

To run a one time backup, mount the volume to be backed up into a container and run the backup command:

docker run --rm \\
-v data:/backup/data \\
--env AWS_ACCESS_KEY_ID="Filebase-Access-Key" \\
--env AWS_SECRET_ACCESS_KEY="Filebase-Secret-Key" \\
--env AWS_S3_BUCKET_NAME="Filebase-Bucket" \\
--env AWS_ENDPOINT="s3.filebase.com" \\
--env AWS_ENDPOINT_PROTO="https" \\
--entrypoint backup \\
offen/docker-volume-backup:latest

Or, you can pass the backup.env file as described above:

docker run --rm \\
-v data:/backup/data \\
--env-file ./backup.env

Restoring a volume from a backup

To restore a volume from a backup, use the following workflow:

1. Stop the container(s) that are using the volume.

2. Untar the backup you want to restore:

   tar -C /tmp -xvf backup.tar.gz

3. Using a temporary container to mount the volume (called ‘data’ in this example) and copy over the backup.

   docker run -d --name backup_restore -v data:/backup_restore alpine

   docker cp /tmp/backup/data-backup backup_restore:/backup_restore

   docker stop backup_restore

   docker rm backup_restore

4. Restart the container(s) that are using the volume.

Depending on your setup, there may be additional steps needed.