Automated remote backup on OS X

From Livid Wiki
Jump to: navigation, search

How to backup automatically to a remote server from OS X.

This worked great using my dreamhost (http://www.dreamhost.com) server. This will work between any OS X computer and a remote server that has ssh and shell access. Once you have completed this, you'll never (again?) experience the trauma and vacant feeling and say "I lost all my important work!"

Generate a key

This set of instructions sets up a secure, password-free "handshake" between your computer and the server, which allows totally secure automated backup. If we didn't do this, the process would require a password at some point, which kills the whole idea that this whole thing happens without human intervention or effort!

  1. Open Terminal and generate an ssh keypair:
    ssh-keygen -t rsa
    (hit return for all questions, do not enter a password when asked)
  2. Copy this key to the backup server's root directory. You must have shell access and ssh enabled on your remote server:
    scp ~/.ssh/id_rsa.pub <username>@<your server.com>:~/
  3. Login to the server shell:
    ssh <username>@<your server.com>
    and enter your password to login.
  4. If the server hasn't yet been been used for for ssh keys, we need to type these lines (enter after each one):
    mkdir .ssh
    cat id_rsa.pub >> .ssh/authorized_keys
    rm id_rsa.pub

    These commands create a .ssh directory on the server, then adds the key (that we copied over in step 2 using "scp") to the server's authorized_keys file, then removes the key from the server's root. You may get an error after the "mkdir" step that the folder exists. This is fine.
  5. Now we have to set up our permissions on the server:
    chmod go-w ~
    chmod 700 ~/.ssh
    chmod 600 ~/.ssh/authorized_keys

rsync: copy only the files that have changed

The rsync command lets you copy files from one directory to another, and even one computer to another. The beauty of this is after the first time you run your rsync command, all other backups only need to copy the changed files, not ALL the files. We are now setup to sync folders with rsync. Type logout to leave your server's shell, then, still in the OS X Terminal window, you can use the command:

rsync -avzun  /Volumes/<hard drive name>/<path to folder you want to back up> <username>@<media server.com>:<media folder>

which, because of the option "n", will do a test run of the sync. It will print out a bunch of stuff. As long as it doesn't have any errors, you are in good shape. Now, you can do it for real, if you want, by removing the "n". You might want to do this with a small, test folder on your computer, with only a few small files:

rsync -avzu  /Volumes/<hard drive name>/<path to folder you want to back up> <username>@<media server.com>:<media folder>

Make a shell script and command

Make the shell script file

That line above (with all the proper paths, of course), can be put into a text file (you can use TextEdit to make a plain text file to do this) with the extension ".sh", thereby creating a shell script. Even better, you can add many lines of different rsync commands to this text file, backing up many folders (even to different servers)! You can also add the line

osascript -e 'tell application "Terminal" to quit saving no'

to the end, so Terminal will quit when the script is completed. Make sure it is a plain text file, and not an .rtf file. There's an option in TextEdit's "Format" menu to change a file to Plain Text if needed. Save with a relevant name, such as "dailybackup.sh" We'll need to let OS X know that this file is something that needs to be executed, so let's type:

sudo chmod +x dailybackup.sh

Make it a command file

Here's where OS X makes this a bit easier. All we have to do to make this shell file something that can be double clicked like an application is change the extension from ".sh" to ".command" - really, that's all!

Schedule this task

On linux and unix machines, this is usually done with cron and crontab, and OS X has something called launchd. Fortunately, we don't have to go there, because iCal can take care of this. In the same way you might use iCal to remind you that you have a dentist's appointment, you can use iCal to regularly open this command file and backup your important files.

  1. Open the iCal application.
  2. Create a new event on some day.
  3. Click on the new event and click on the "Edit" button.
  4. Create a start time for the backup.
  5. Click on the "repeat" menu to select how often you want your backup script to run. Every day is not a bad idea!
  6. Click on the "alarm" menu and select "Open file." This adds a few more menus.
  7. In the next menu, it says "iCal." Click there, and select "Other..." and find your .command file.
  8. In the menu after that, it will say "15 minutes before." This means the file will be opened 15 minutes before the scheduled time of the event. That is, if you created the event at 10 am, the .command would actually run at 9:45. You can change this to "1 minute before" or "1 minute after."

SAFE!

You are now very well protected against losing important files in the event of computer failure, or a local disaster such as theft, fire, flood, etc. Your computer should now run this shell script daily at the time you specified, securely backing up your important files offsite.


...but if you REALLY want to use launchd...

Perhaps you really want to get in deep. Perhaps you don't want the Terminal window to open every time this script runs. Perhaps you are just curious and learn something new. Well, here's how. You'll need to stop at "Make the shell script" in the above instructions, then continue from below....

Make the .sh executable

Now we need to "install" this shell script. Let's say you called it "importantbackup.sh" and saved it to your username directory. In the Terminal, type the following, hitting enter after each line:

cd ~
sudo cp importantbackup.sh /usr/bin
cd /usr/bin
chmod +x importantbackup.sh

This set of commands moves you to your home directory (where the .sh file lives), then copies the file to /usr/bin using "sudo" which basically just gives you permission to do it. You'll be asked for your password after that line. Then we'll change to the /usr/bin directory where we copied the .sh file. Finally, you change the permissions on that file so it can be "executed" from the command line. Now, if you type

importantbackup.sh

in Terminal, this is exactly the same as typing that complete "rsync" command you made above (or if you have many lines in that file, typing all the lines).

Schedule this task

On linux and unix machines, this is usually done with cron and crontab. However, OS X has a thing called launchd that is a bit more sophisticated, not as well documented, and probably harder to use. It's apparently still possible to use "cron" to do this on OS X, but it seems that "launchd" is the way to move with the future.

Make a plist file that describes the scheduled task

In TextEdit, put this xml into a Plain Text file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>label</key>
	<string>com.username.relevanttaskname.backup</string>
	<key>ProgramArguments</key>
	<array>
		<string>importantbackup.sh</string>
	</array>
	<key>StartCalendarInterval</key>
	<dict>
		<key>Hour</key>
		<integer>10</integer>
		<key>Minute</key>
		<integer>15</integer>
	</dict>
	<key>StandardErrorPath</key>
	<string>/tmp/importantbackup.err</string>
	<key>StandardOutPath</key>
	<string>/tmp/importantbackup.out</string>
</dict>
</plist>

Of course, you'll need to fill in some of the above with your own info:

com.username.relevanttaskname.backup

change "username" to your username, "relevanttaskname" to a word that describes what is being scheduled, such as "importantfiles" and leave "backup" as it is. Change

importantbackup.sh

to the name of the shell script that you created in the previous steps. In the text between the <dict> and </dict> tags, you can setup a daily time. In this example, the task is scheduled for 10:15 in the morning. Finally, if you want some log files for this task, change the text for these

/tmp/importantbackup.err

and

/tmp/importantbackup.out

using the name of the shell script. These can be really useful for debugging and making sure the backup actually occurred. Because all this happens in the background, you may want some assurance! You can find more info about this xml file format, including different timing options, for launchd in these links:
http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man5/launchd.plist.5.html
http://www.devdaily.com/mac-os-x/launchd-plist-examples-startinterval-startcalendarinterval

Now that you've created this xml file, you must save it to the folder

<hard drive>:/Library/LaunchAgents/ 

and give it the name

com.username.relevanttaskname.backup.plist

This is a weird name, I realize, but it must be the same as the text (plus the .plist extension) that you put between the <label> tags in the text file. When you save, if Text Edit gives a warning about the extension, make sure you select "use .plist". You'll notice this is saving to the root Library, not the user's Library, which means this task will run with root permissions. This needs to happen for the rsync to properly work, otherwise it won't have ssh permission. This was, at least, my experience with this: I couldn't get the script to properly work unless I put the plist in the root Library.

Make the operating system aware of this task

We are almost done. Open a Terminal window and type:

cd ~/Library/LaunchAgents

and then type

launchctl load com.username.relevanttaskname.backup.plist

Restart the computer. DONE. Your computer should now run this shell script daily at the time you specified in the plist, securely backing up your important files offsite.