I was looking for ways to do incremental backups with rsync, and all I could find were scripts that pretty much push down a stack e.g. backup.0 backup.1 backup.2 etc but I didn’t like how that works for daily backups. Instead, I modified the idea to go the other way and do backups into Day folders e.g. ‘monday’, ‘tuesday’, etc. Being organized this way, it’s easy to find a change that happened last Thursday, for example. Each day, before it runs the update, it also pushes what’s in that day’s folder (from the previous week) into a ‘lastweek’ folder, and on the first Monday of the month, it’ll push the preious Monday’s data into a ‘lastmonth’ folder. Deleted files on the server are also kept in a separate directory, inside each day’s folder. The entire folder tree is served up on a samba server, read only, so users can find old versions of files if they need to. Every time it runs, it creates hard links of existing files, so the only actual copies of files are whatever files get modified, or added between executions.
#!/bin/bash
TODAY=$(date +%A | tr '[:upper:]' '[:lower:]')
YESTERDAY=$(date --date="yesterday" +%A | tr '[:upper:]' '[:lower:]')
DAYOFMONTH=$(date +%d)
REMOTEDIR="root@x.x.x.x:/srv/external/workfiles"
BACKUPDIR="/srv/external/rollingBackup/"
LASTRUN="no-date"
TODAYDATE=$(date +%D)
cd $BACKUPDIR
#check if there was a previous run date present
if [ -a $BACKUPDIR/$TODAY/backup-date.txt ]
then
LASTRUN=$(cat $BACKUPDIR/$TODAY/backup-date.txt)
fi
# create folders if they dont exist yet
if [ ! -d sunday ]
then
mkdir sunday
fi
if [ ! -d monday ]
then
mkdir monday
fi
if [ ! -d tuesday ]
then
mkdir tuesday
fi
if [ ! -d wednesday ]
then
mkdir wednesday
fi
if [ ! -d thursday ]
then
mkdir thursday
fi
if [ ! -d friday ]
then
mkdir friday
fi
if [ ! -d saturday ]
then
mkdir saturday
fi
if [ ! -d lastweek ]
then
mkdir lastweek
fi
if [ ! -d lastmonth ]
then
mkdir lastmonth
fi
if [ $TODAY == monday ]
then
if [ $LASTRUN != $TODAYDATE ]
then
if [ $DAYOFMONTH -lt 8 ]
then
if [ -z "$(ls -A ./lastweek)" ]
then
echo "Last week is empty, so don't remove last month"
else
echo "First week of the month, move 'lastweek' to 'lastmonth'"
rm -Rf ./lastmonth/*
cp -al ./lastweek/* ./lastmonth/
fi
fi
if [ -z "$(ls -A ./$TODAY)" ]
then
echo "Today's folder is empty, so don't empty last week's folder"
else
echo "Moving 'monday' to 'lastweek' before update"
rm -Rf ./lastweek/*
cp -al ./$TODAY/* ./lastweek/
fi
echo "Copy 'yesterday' to 'today', but remove yesterday's deleted files from today"
rm -Rf ./$TODAY/*
cp -al ./$YESTERDAY/* ./$TODAY/
rm -Rf ./$TODAY/deleted_files
fi
echo "Starting rsync..."
rsync -avp --delete --backup --backup-dir=$BACKUPDIR/$TODAY/deleted_files -e 'ssh' $REMOTEDIR $BACKUPDIR/$TODAY
rm ./$TODAY/backup-date.txt
rm ./$TODAY/backup-time.txt
date +%D > ./$TODAY/backup-date.txt
date +%T > ./$TODAY/backup-time.txt
else
if [ $LASTRUN != $TODAYDATE ]
then
if [ -z "$(ls -A ./$TODAY)" ]
then
echo "Today's folder is empty, so don't empty last week's folder"
else
echo "Moving '$TODAY' to 'lastweek'"
rm -Rf ./lastweek/*
cp -al ./$TODAY/* ./lastweek/
fi
echo "Copy 'yesterday' to 'today', but remove yesterday's deleted files from today"
rm -Rf ./$TODAY/*
cp -al ./$YESTERDAY/* ./$TODAY/
rm -Rf ./$TODAY/deleted_files
fi
echo "Starting rsync..."
rsync -avp --delete --backup --backup-dir=$BACKUPDIR/$TODAY/deleted_files -e 'ssh' $REMOTEDIR $BACKUPDIR/$TODAY
rm ./$TODAY/backup-date.txt
rm ./$TODAY/backup-time.txt
date +%D > ./$TODAY/backup-date.txt
date +%T > ./$TODAY/backup-time.txt
fi
So far, it appears to work as expected, but I won’t know for sure until it has ran a while without any problems.
In the Mind of Chris 