Posted By

_reydin_ on 07/04/12


Tagged

backup rsync linux


Versions (?)

Who likes this?

1 person have marked this snippet as a favorite

_reydin_


Linux backup script using rsync and hard-links


 / Published in: Bash
 

This script will do a daily backup of directories. If the files has not changed then it creates a hard-link to the file. If the file has changed then it will copy the new file. The backup is not much larger than the directories being backed up.

  1. #!/bin/bash
  2. #This scrip is used for backing up SPCSERV. It will make a full backup and six incremental
  3. #backups. This adds the date to the begining of the file's name. To recover data from the backup
  4. #just go directly to the date you want to recover from and copy that file to another location.
  5.  
  6. ## Make the following entry into the crontab -e. The only part the will need to be changed is where the
  7. # rsync.sh file resides. For this example the rsync.sh file is in the /srv/ directory.
  8. # 0 1 * * * /srv/rsync.sh 1>>/var/log/backup.log 2>>/var/log/backup.log
  9.  
  10.  
  11. ####### Backup Directory Variable ##########
  12. bkdir=/mnt/intbackup/spcserv #to save the backup somewhere else.
  13.  
  14. ## DO NOT CHANGE THESE ENTRIES. Change the above bkdir entry to change the location of where the backup is saved.
  15. tstamp=$(date +%Y-%m-%d_%H:%M:%S) #Timestamp used for naming files.
  16. bkdir0=${bkdir}/${tstamp}_backup.0/ #This starts the naming convention. It sets the name and directory for the current backup.
  17. beforetstmp="$(date +%s)" #Used to calculate the amount of time the backup process took.
  18.  
  19.  
  20. #### DO NOT CHANGE THESE ENTRIES. ####
  21. ###### This section is used to move the backup files #######
  22. # This gets the name for each backup so that it can be moved. Since the backups' name is constantly changing
  23. # we need a way of getting the current name for each file.
  24. bkup0=`ls $bkdir | grep backup.0` #This gets the name of backup.0 so it can be moved to backup.1
  25. bkup1=`ls $bkdir | grep backup.1` #This gets the name of backup.1 so it can be moved to backup.2
  26. bkup2=`ls $bkdir | grep backup.2` #This gets the name of backup.2 so it can be moved to backup.3
  27. bkup3=`ls $bkdir | grep backup.3` #This gets the name of backup.3 so it can be moved to backup.4
  28. bkup4=`ls $bkdir | grep backup.4` #This gets the name of backup.4 so it can be moved to backup.5
  29. bkup5=`ls $bkdir | grep backup.5` #This gets the name of backup.5 so it can be moved to backup.6
  30. bkup6=`ls $bkdir | grep backup.6` #This gets the name of backup.6 so it can be moved to backup.tmp; which will
  31. #be moved to backup.0. We recycle this directory to reduce the amount of time
  32. #rsync has to run.
  33.  
  34. #### DO NOT CHANGE THESE ENTRIES. ####
  35. ## This sripts the ending digit so that file can be incremented to the next digit.
  36. bkupStrp0=`echo ${bkup0} | awk -F'.' '{print $1}'` #This removes the ending digit 0 from backup.0
  37. bkupStrp1=`echo ${bkup1} | awk -F'.' '{print $1}'` #This removes the ending digit 1 from backup.1
  38. bkupStrp2=`echo ${bkup2} | awk -F'.' '{print $1}'` #This removes the ending digit 2 from backup.2
  39. bkupStrp3=`echo ${bkup3} | awk -F'.' '{print $1}'` #This removes the ending digit 3 from backup.3
  40. bkupStrp4=`echo ${bkup4} | awk -F'.' '{print $1}'` #This removes the ending digit 4 from backup.4
  41. bkupStrp5=`echo ${bkup5} | awk -F'.' '{print $1}'` #This removes the ending digit 5 from backup.5
  42. bkupStrp6=`echo ${bkup6} | awk -F'.' '{print $1}'` #This removes the ending digit 6 from backup.6
  43.  
  44.  
  45. #### DO NOT CHANGE THESE ENTRIES. ####
  46. ###### This section is where we start moving the file and renaming them. ######
  47. mv ${bkdir}/${bkup6} ${bkdir}/backup.tmp #This moves backup.6 to backup.tmp
  48. mv ${bkdir}/${bkup5} ${bkdir}/${bkupStrp5}.6 #This moves backup.5 to backup.6
  49. mv ${bkdir}/${bkup4} ${bkdir}/${bkupStrp4}.5 #This moves backup.4 to backup.5
  50. mv ${bkdir}/${bkup3} ${bkdir}/${bkupStrp3}.4 #This moves backup.3 to backup.4
  51. mv ${bkdir}/${bkup2} ${bkdir}/${bkupStrp2}.3 #This moves backup.2 to backup.3
  52. mv ${bkdir}/${bkup1} ${bkdir}/${bkupStrp1}.2 #This moves backup.1 to backup.2
  53. mv ${bkdir}/${bkup0} ${bkdir}/${bkupStrp0}.1 #This moves backup.0 to backup.1
  54. mv ${bkdir}/backup.tmp ${bkdir0} #This moves backup.tmp to backup.0
  55.  
  56.  
  57. ###### This section is responsible for syncing all of the directories you want backed up to the backup directory. ########
  58. ## Edit this section if you want to add other directories to the backup. Here I am backing up 3 directories.
  59. ## When adding directories to this section do not use a trailing / after the directory's name.
  60. ## Example: rsync -a --delete --link-dest=/srv/bkup/${bkupStrp0}.1 /srv/Share /etc /home ${bkdir0}
  61. #rsync -a --delete --link-dest=${bkdir}/${bkupStrp0}.1 /etc /srv /home /var /ldaphome ${bkdir0}
  62. rsync -a --delete --link-dest=${bkdir}/${bkupStrp0}.1 /etc /var /home /srv /ldaphome /ldap_data ${bkdir0}
  63.  
  64. # Calculate the time the whole backup process took
  65. aftertstmp="$(date +%s)"
  66. elapsed="$(expr $aftertstmp - $beforetstmp)"
  67. hours=$(($elapsed / 3600))
  68. elapsed=$(($elapsed - $hours * 3600))
  69. minutes=$(($elapsed / 60))
  70. seconds=$(($elapsed - $minutes * 60))
  71.  
  72. # echo "\n"
  73. endtstamp=$(date +%Y-%m-%d_%H:%M:%S) #Timestamp used for naming files.
  74. ##### Make an entry in the log file that the backup ran successfully #####
  75. echo ${tstamp} Backup ran successfully. The backup process took: ${hours} hours ${minutes} minutes ${seconds} seconds to complete.
  76. echo The backup ended at ${endtstamp}

Report this snippet  

Comments

RSS Icon Subscribe to comments
Posted By: FrancoisScheurer on September 13, 2012

Hello

you can find a script to backup the whole disk with rsync here: http://blog.pointsoftware.ch/?p=527&preview=true It uses file deduplication thanks to hard-links, uses also MD5 integrity signature, 'chattr' protection, filter rules, disk quota, retention policy with exponential distribution (backups rotation while saving more recent backups than older). It was already used in Disaster Recovery Plans for banking companies, in order to replicate datacenters, using only little network bandwidth and transport encryption tunnel. Can be used locally on each servers or via network on a central remote backup server.

it is free and i hope it can suit your needs ^^ Francois Scheurer

Posted By: FrancoisScheurer on September 13, 2012

sorry the direct url is : http://blog.pointsoftware.ch/index.php/howto-local-and-remote-snapshot-backup-using-rsync-with-hard-links/

enjoy!

You need to login to post a comment.