Sync’ing content between two servers.

There are a number of reasons why it may be an advantage to sync the data in a directory (or the whole disk) from one server to another. This technique has been used to backup one server to another, sync mailboxes or sync the ports tree.

I find this very useful while building a new server to replace an old one. Some time it may take a few days to get the new one built up and testing. Dorring this time the only one is still in production and file are changing on it. What I do is sync the user directories and other dynamic files durring the upgrage, then they are already in sync one the new server is ready to go.

Setting up the SSH keys

The rsync program can use two different protocols to communicate from one server to the other. RSH or SSH. I only use SSH because if it’s security so we will need to get the SSH connection setup and going before we can use the rsync program over it. I have a detailed explination on how to get the SSH keys setup here so this example will be quick.

Log into the server that will have the origonal files that you wish to sync. In my case this would be the older server.

# mkdir -p /root/.ssh
# cd /root/.ssh
# ssh-keygen -q -f ~/.ssh/client.domain.com.key -t rsa -N "" -C "Client key for rsync"

Edit the public key to add the ‘from’ and ‘command’ paramaters as shown here and then copy the whole public key to the end of the /root/.ssh/authorized_keys file.

# cat client.domain.com.key.pub >> /root/.ssh/authorized_keys

Login to the client server (in my case this will be the new server I am building) and test the connection. If you haven’t added a command statement to the public key then you should be able to ssh into thre server without a password.

# ssh -i /root/.ssh/client.domain.com.key server.domain.com

Installing Rsync from the ports

Rsync needs to be installed from the ports on both servers. Login to each server and run the following commands:

# cd /usr/ports/net/rsync
# make install (select the default options as well as the ATIME fix)
# make clean

Setting up the Sync

rsync should be all setup by now. Now we just need to test it. I am going to sync a user directory from the server over to the client machine in order to test and make sure everything is working.

# rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" root@server.domain.com:/usr/home/user /usr/home/

The above command should sync all the files from the “user” directory on the server to the “user” directory on the client. If you run the command again, it should sync only the changes each time.

I generaly create a shell script on the client server that I run periodicaly out of cron to keep the two server sync’ed. Here is an example shell script that I would use.

#!/bin/sh
#
# Sync the password databases
rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/etc/*pwd* /etc/
rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/etc/*pass* /etc/
rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/etc/group /etc/
/usr/sbin/pwd_mkdb /etc/master.passwd
#
# Sync the users home directories
rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/usr/home /usr/
#
# Sync the email files
rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/var/mail /var/
rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/etc/mail/aliases /etc/mail/
/usr/bin/newaliases

That should be enough to get you going. Don’t forget to turn off the sync once once the new server is in place.

If you wanted to backup a server, you can use the following command.

rsync -av -e "ssh -i /root/.ssh/client.domain.com.key" --delete root@server.domain.com:/etc/group /usr/backup/server/