My Backup Solution for Linux

Many people do not realize the value of a good backup. I do and it therefore have hurt me over the years that I couldn't afford good backups. So far affordable medias have been too small for a price a private person could afford.

I've tried it all, I've tried backing up to cheap tape drives, I've tried backing up to DAT drives, I've tried backing up to writable cd's, and now finally I back up to a hard disk.

Requirements

My requirement to a good back up are basically as follows:

It must be easy to use both for back up and restore - lots of commercial applications exists, which requires a Master degree in computer science to figure out.
It must be fast to restore files - one solution I tried was backing up to a hard disk using tar. But given the files would be up to 4Gb large, and given that tar can only read files sequentially, restore was limited to the speed of the hard disk, and given I used an external USB hard disk, we were talking many hours at worst to restore a single file.
It must be easy to verify that a backup really has been executed - to sleep well at nights, you should have confidence in that your backup actually can be restored.
It must be possible to have several revisions of your files, that is not just a backup of what your files looked like last time the back up executed, but any day over the last month or so.
Errors on the media should be kept at a minimum, and when they occur the restore should be able to recover without manual intervention - This has indeed been the biggest problem to date when using tar on inexpensive tapes.

I believe I've found a solution which fits all these requirements.

The Backup system

The backup is done using a few shell scripts, which wraps the unix command rsync. The backup is done onto an IDE hard disk which is put into a simple USB wrapper box. The USB wrapper box cosy approximately 500DKK ( less thenan $100) and my latest IDE disk was less than $1/Gb. The hard disk thus looks like a USB storage device to Linux.

Transfer rates is in the order of magnitude 1-2Mb/sec, which is not amazing, on the other hand, its only for back up purposes, so it doesn't need to be utterly fast. If you have free IDE slots in the machine, and only need one disk for backup, then you can of course build it into the machine rather than wrap it in relatively slow USB.

The advantage of using the USB interface is that you can store backup for as long your money permits you, just buy extra hard disks.

The result of the backup is a snapshot of the current version. Parallel to this directory is a number of directory trees containing older versions of the files.

Here is an example:

/backup
  laptop
    current
      /var
        ...
      /etc
         passwd
         ...
    archives
      2004-02-02-19:30
        etc
          passwd

During the backup run on 2/2-2004 at 19:30, the passwd file was marked for backup. Before it was copied over from the backup source the - at that time - current version, was copied to the archives directory.

If I one day need to restore the /etc directory as it was most recent, then I simply must copy the version in the directory current over.

If I need to restore an older version, I must simply copy the files from current over, plus files from all the dates after the date I want a restoration of, starting with the newest version. This has one tiny disadvantage, namely files added after the latest date will appear in the restore, even though they should not. I consider this a tiny disadvantage.

The backup consist of the following scripts, which you indeed must modify to your own needs, on the other hand they are utterly simple.

backup-catatonia this is backup for my desktop machine, on which the USB device is attached. The real work is done in the .backup-base file.
.skip-catatonia list of files the backup should not include
.backup-base this is the file doing the real work - not that there is much for it to do, it simply calls rsync to do the real work.
backup-mac, .skip-mac - files for the Macintosh.

Alternative systems

Your requirements might not be the exact same as mine, and thus I'd like to point you to alternative systems that might better fit your needs:

RSyncBackup - this is the system I got inspired from to implement the above scripts. Its a bit more polished, but does not include support for ssh, which I guess should be fairly easy to add. In addition it also includes support for logging.
hdup - another tool for backing up to hard disk. Its strong side is backing up from several different machines.
cddump - backup to CD-R or CD-RW's

Jesper Kjær Pedersen <blackie@blackie.dk>

Last modified: Tue Feb 3 09:35:06 2004