How To: Using The Terminal To Backup Your System:
This post is part of the series:
Dump is a low level backup program (an ancient as the Mayans), created in the age of the Dinosaurs…wait no…by Unix admins, in an era were tape drives were used and random read/writes were not common. Dump will backup a filesystem as an archive and is highly useful for restoring an “image” back to the system. I will get into the frustrations of Dump later on. It is highly recommended you only run dump on unmounted filesystems/mount points, or when the system is being used very little, as it may encouter issues backing up something that is currently being modified. In and of itself, dump is very archaic and simplistic. It does what it needs to, but is very limited. It does have the ability to only detect new files and efficiently back them up. Restoring is done with the restore command, and is very easy, just as dump is. The fact that dump is low-level is very desirable for most.
Rsync is another command based backup tool for Unix-like file systems that provides similar features to dump, but not at the low-level like dump. This is not necessarily bad at all, and it effectively does it’s job very well, if not better. It’s algorithm is very fast, and contains recursion and compression options. For personal backups, I highly suggest rsync over dump. There will be much less of an issue on a live-running system, which I will get into below. Originally, before rsync, we had rcp and scp, and that is where rsync’s network abilities come into play. Rsync is highly flexible, scriptable, and fast.
Dump vs. Rsync: The Battle For Middle Earth! / Planet Earth! / you get the drill…
It is generally accepted nowadays, at least I would think so, that dump is great for periodic or one time backups of your system, but it’s impractical to use for daily personal backups, more so on the incremental scale. Why? Dumping a live filesystem can have some serious quirks that prove more serious than agitating. If dump is ran on a live filesystem, the filesystem could be changing as you run dump, especially the first non-incremental time. If you for instance, use tar, a file could be changed at the time it is read by tag, corrupting that resulting tar file. This involves complicated Linux kernel cache writes, and how dump reads the block device rather than what you see through a file listing with “ln -la /path.” Dump has always had this problem, and Unix Sysadmins know this and safely use the file on unmounted and read-only filesystems. Doing so on an “idle” system is OK, but not perfect. Doing so on a non-idle system is not advisable but doable when the system is not heavily loaded. The advantage does go to dump for being able to backup an unmounted filesystem, which rsync cannot do. With rsync, backups are done incrementally after the first backup, and synced/mirrored. across the two devices. I prefer using this utility for day to day backups, along with a cron job (scheduled task in Windows) to automate the whole affair. I have read dump is faster because it bypasses the kernel’s filesystem interface. But dump also must know at least some of the filesystem internals, and therefore does not work on every file system, but it will for common ones such as Ext2/3/4. Rsync, to me is far more versatile in it’s modern day usage.
Getting down to brass tax: implementing rsync:
First, you want to make sure you have rysnc installed:
sudo apt-get install rsync
Once installed, you can proceed. The general syntax is as follows (or you can do a “man rsync” for more):
sudo rsync -av --delete /Directory1/ /Directory2/
The options above are just one case scenario and do not reflect everyone’s need/case. The above command will backup the files from Directory1 to Directory2, both residing on the same hard drive (but would be the same if not). The options specified will leave no differences between the two. If Directory2 has a file that Directory1 does not, it will be deleted, reflecting the changes to Directory2.
- The -a option means it is recursive, links (symlinks are preserved), perms (preserves permissions), times (preserves modification times), groups (preserve group assignments), owner (preserves the owner), and also preserves special files.
- The -v option means it is verbose, and will show you what is happening real time I find this to be very important, as if you encounter an error, you will immediately see it.
- The –delete option tells rsync to delete any files that exist on Directory2, but are non on Directory1, effectively mirroring Directory1 to* Directory2.
- For more options, type “man rysnc” into the Terminal.
Some other thoughts on rysnc: Rsync can also be configured for external backups by tunneling through SSH. This is useful if you are backing up one machine on your network from another physical machine (see told you Rsync was awesome!). First install the necessary software:
sudo apt-get install ssh rsync
After installed, all you need to do is setup where you where you want the backup to go, and properly configure SSH for security. Make sure you use the correct port when running your backup as well. The following command is a typical use-case example:
rsync -av –delete -e 'ssh -p 12345' /Directory1/ firstname.lastname@example.org:/Directory2/
- My personal backup.sh bash script can be found here.
Setting up automatic Backups with rysnc/dump
: To setup the automatic backup with rysnc, or even dump, you need to create a Cron job. Cron handles the automatic execution of commands, such as the two referenced above. Once you get this portion setup, it is smooth sailing (provided you specify the right options on backup!). To get into your Cron settings, type this into your Terminal:
sudo crontab -e
Cron tab will use Vi as its text editor, commands can be found here. HOLY COW! What is all that !? Don’t be intimidated, it is not as scary as it looks! Let’s start with one example:
0 22 * * * rsync -av --delete /Directory1/ /Directory2/
In the above example, 0 specifies the minute of the hour, and 22 specifies 10 p.m. (24 hour format). If you want this command to run daily, leave the rest of the fields with asterisk and then pate the rsync command. See? Not so hard! Type “man cron” or “man crontab” to get more detailed informaion. If you wish to use dump, in the above example input dump, follow by your desired options after the command, instead of rsync. That’s it! You can also zip your files during rsync’s backup or after. It is up to you.
What about snap?
Snap is a utility to take snapshots on Linux, but because it is a whole snapshot it takes some time and is not particularly useful for the day to day backups. You can make a snapshot with snap and use dump to then use that as it’s backup point. This is a far safer way to go about things, and is not a bad idea. Rsync is independent, and has its own snapshot/purge schedule on PULL, which mimics being “offline” in reference to dump. The main difference here is doing a whole “image” snapshot vs. an incremental backup. It is your choice. Maybe time is on your side, but I don’t bother with my day to day backups in such things. You can use dump in conjunction with snap for a “dyanmic duo,” ala BATMAN STYLE.
snap -create /dev/hda1 /dev/snap1 # create a snapshot of /dev/hda1 dump 0f /dev/nst0 /dev/snap1 # backup the snapshot instead of the real +fs snap -delete /dev/snap1
Everyone loves the dd utility! Well…at least I* do heh. dd has been around for ages and it is a tried and true snapshot tool, before there was a snapshot tool. his tool can copy a single file or an entire drive. When you need to clone a drive, and the command line is all you have available, dd is what you will need. Think of rsync but for the entire drive, as a snapshot. Basic use is as follows:
dd if=<source> of=<target> bs=<byte size>
dd if=/dev/sda2 /media/mikeyd/my_backup.img
Note: usually some power of 2, and usually not less than 512 bytes (ie, 512, 1024, 2048, 4096, 8192, 16384, but can be any reasonable whole integer value.) skip= seek= conv=<conversion>
Using Tar is sort of akin to making a giant zip file of your system in Windows, not that you would ever do that, right? 🙂 Basic syntax is as follows:
tar -cvpzf backup.tar.gz --exclude=/backup.tar.gz --one-file-system /
The above example assumes you want to Tar, using only one filesystem (the source’s), using / (root) as the starting point (which is everything), into a file called /backup.tar.gz (the / denotes the save will be a the “root” layer. More details can be found here.
Not so tough, right?
More on the subject: