Docker Volume Backup

Backup Docker container volumes to Filebase using Docker Volume Backup.

What is Docker Volume Backup?

Docker Volume Backup is a Docker image that can be used as a sidecar container to any existing Docker setup. This sidecar container can be configured for recurring or one time backups of Docker container volumes to any S3-compatible storage. It can also be configured to periodically delete old backups and send email notifications of backup failures.

Prerequisites:

Download and Install Docker and Docker-Compose.
Sign up for a free Filebase account.
Have your Filebase Access and Secret Keys. Learn how to view your access keys here.
Create a Filebase Bucket. Learn how to create a bucket here.

Configure Recurring Backups In a docker-compose File

The following is a sample configuration for a docker-compose.yml file to add a backup service to your compose setup and mount the volumes you would like to see backed up:

version: '3'

services:
  volume-consumer:
    build:
      context: ./my-container
    volumes:
      - data:/var/my-container
    labels:
      # This means the container will be stopped during backup to ensure backup integrity.
      - docker-volume-backup.stop-during-backup=true

  backup:
    image: offen/docker-volume-backup:latest
    restart: always
		environment:
				AWS_S3_BUCKET_NAME="filebase-bucket-name"
				AWS_ACCESS_KEY_ID="Filebase-Access-Key"
				AWS_SECRET_ACCESS_KEY="Filebase-Secret-Key"
				AWS_ENDPOINT="s3.filebase.com"
				AWS_ENDPOINT_PROTO="https"
   
    volumes:
      - data:/backup/my-app-backup:ro
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`.
      # You can omit this if you do not want to keep local backups.
      - /path/to/local_backups:/archive
volumes:
  data:

env_file template:

Fill out the following environment variable template and save it as backup.env in the directory of your docker-compose file. If you use this method, you can replace the environment section in your docker-compose file with env-file: ./backup.env.

########### BACKUP SCHEDULE

# Backups run on the given cron schedule in `busybox` flavor. If no
# value is set, `@daily` will be used. If you do not want the cron
# to ever run, use `0 0 5 31 2 ?`.

# BACKUP_CRON_EXPRESSION="0 2 * * *"

# The name of the backup file including the `.tar.gz` extension. The default results
# in filenames like `backup-2021-08-29T04-00-00.tar.gz`.

# BACKUP_FILENAME="backup-%Y-%m-%dT%H-%M-%S.tar.gz"

# Whether to copy the content of the backup folder before creating the tar archive.
# In the rare scenario where the content of the source backup volume is continuously
# updating, but we do not wish to stop the container while performing the backup,
# This setting can be used to ensure the integrity of the tar.gz file.

# BACKUP_FROM_SNAPSHOT="false"

########### BACKUP STORAGE

# Filebase bucket name
# AWS_S3_BUCKET_NAME="filebase-bucket-name"

# Filebase credentials
# AWS_ACCESS_KEY_ID="Filebase-Access-Key"
# AWS_SECRET_ACCESS_KEY="Filebase-Secret-Key"

# Filebase endpoint 
# AWS_ENDPOINT="s3.filebase.com"

# Filebase only supports the HTTPS protocol.
# AWS_ENDPOINT_PROTO="https"

# Setting this variable to `true` will disable verification of
# SSL certificates. You shouldn't use this unless you use self-signed
# certificates for your remote storage backend. This can only be used
# when AWS_ENDPOINT_PROTO is set to `https`.

# AWS_ENDPOINT_INSECURE="true"

# In addition to storing backups remotely, you can also keep local copies.
# Pass a container-local path to store your backups if needed. You also need to
# mount a local folder or Docker volume into that location (`/archive`
# by default) when running the container. In case the specified directory does
# not exist (nothing is mounted) in the container when the backup is running,
# local backups will be skipped. Local paths are also subject to pruning of
# old backups as defined below.

# BACKUP_ARCHIVE="/archive"

########### BACKUP PRUNING

# **IMPORTANT, PLEASE READ THIS BEFORE USING THIS FEATURE**:
# The mechanism used for pruning old backups is not very sophisticated
# and applies its rules to **all files in the target directory** by default,
# which means that if you are storing your backups next to other files,
# these might become subject to deletion too. When using this option
# make sure the backup files are stored in a directory used exclusively
# for such files, or to configure BACKUP_PRUNING_PREFIX to limit
# removal to certain files.

# Define this value to enable automatic rotation of old backups. The value
# declares the number of days for which a backup is kept.

# BACKUP_RETENTION_DAYS="7"

# In case the duration a backup takes fluctuates noticeably in your setup
# you can adjust this setting to make sure there are no race conditions
# between the backup finishing and the rotation not deleting backups that
# sit on the edge of the time window. Set this value to a duration
# that is expected to be bigger than the maximum difference of backups.
# Valid values have a suffix of (s)econds, (m)inutes or (h)ours. By default,
# one minute is used.

# BACKUP_PRUNING_LEEWAY="1m"

# In case your target bucket or directory contains other files than the ones
# managed by this container, you can limit the scope of rotation by setting
# a prefix value. This would usually be the non-parameterized part of your
# BACKUP_FILENAME. E.g. if BACKUP_FILENAME is `db-backup-%Y-%m-%dT%H-%M-%S.tar.gz`,
# you can set BACKUP_PRUNING_PREFIX to `db-backup-` and make sure
# unrelated files are not affected by the rotation mechanism.

# BACKUP_PRUNING_PREFIX="backup-"

########### BACKUP ENCRYPTION

# Backups can be encrypted using gpg in case a passphrase is given.

# GPG_PASSPHRASE="Passphrase"

########### STOPPING CONTAINERS DURING BACKUP

# Containers can be stopped by applying a
# `docker-volume-backup.stop-during-backup` label. By default, all containers
# that are labeled with `true` will be stopped. If you need more fine grained
# control (e.g. when running multiple containers based on this image), you can
# override this default by specifying a different value here.

# BACKUP_STOP_CONTAINER_LABEL="service1"

########### EMAIL NOTIFICATIONS ON FAILED BACKUP RUNS

# In case SMTP credentials are provided, notification emails can be sent out on
# failed backup runs. These emails will contain the start time, the error
# message and all log output prior to the failure.

# The recipient(s) of the notification. Supply a comma separated list
# of addresses if you want to notify multiple recipients. If this is
# not set, no emails will be sent.

# EMAIL_NOTIFICATION_RECIPIENT="you@example.com"

# The "From" header of the sent email. Defaults to `noreply@nohost`.

# EMAIL_NOTIFICATION_SENDER="no-reply@example.com"

# Configuration and credentials for the SMTP server to be used.
# EMAIL_SMTP_PORT defaults to 587.

# EMAIL_SMTP_HOST="posteo.de"
# EMAIL_SMTP_PASSWORD="SMTP-password"
# EMAIL_SMTP_USERNAME="no-reply@example.com"
# EMAIL_SMTP_PORT="SMTP-port"

One Time Backups Using Docker CLI

To run a one time backup, mount the volume to be backed up into a container and run the backup command:

docker run --rm \\
-v data:/backup/data \\
--env AWS_ACCESS_KEY_ID="Filebase-Access-Key" \\
--env AWS_SECRET_ACCESS_KEY="Filebase-Secret-Key" \\
--env AWS_S3_BUCKET_NAME="Filebase-Bucket" \\
--env AWS_ENDPOINT="s3.filebase.com" \\
--env AWS_ENDPOINT_PROTO="https" \\
--entrypoint backup \\
offen/docker-volume-backup:latest

Or, you can pass the backup.env file as described above:

docker run --rm \\
-v data:/backup/data \\
--env-file ./backup.env

Restoring a volume from a backup

To restore a volume from a backup, use the following workflow:

Stop the container(s) that are using the volume.
Untar the backup you want to restore:
tar -C /tmp -xvf backup.tar.gz
Using a temporary container to mount the volume (called ‘data’ in this example) and copy over the backup.
docker run -d --name backup_restore -v data:/backup_restore alpine
docker cp /tmp/backup/data-backup backup_restore:/backup_restore
docker stop backup_restore
docker rm backup_restore
Restart the container(s) that are using the volume.

Depending on your setup, there may be additional steps needed.

If you have any questions, please join our Discord server, or send us an email at hello@filebase.com

PreviousDocker NextElixir Phoenix

Last updated 1 year ago