Using Rsnapshot and SSH

This document covers the details of setting up the SSH part of an automated remote Rsnapshot backup configuration, so I do not have to relearn how to do it next time.

It is assumed that the starting point here is a correctly installed and configured Rsnapshot setup, so you will need these packages installed:

rsnapshot
rsync
openssh

I have completed my configuration on a Red Hat Fedora Core 3 server, but I will try to keep this instructional as "distribution generic" as possible. There is much that is not covered here regarding the setup of Rsnapshot because it is covered so well in the Rsnapshot HOWTO. Please grow to know and love that document before reading this one.

Much of what is explained here is explained in a more general way in another document I wrote on Using Rsync and SSH, but it has no Rsnapshot specific information.

In the examples I will be configuring Rsnapshot as the root user on localhost.example.com to do backups as remoteuser for remotehost.example.com.

Choosing the root user to run Rsnapshot from localhost.example.com is more dangerous than using an unprivileged user to do the same, but I think it is the default configuration (for a few good reasons [1]). Choosing the root user on remotehost.example.com would be considerably more dangerous, and should be avoided where possible, so I will not connect as root in these examples.

Creating a SSH key without a password is not something one should consider lightly. If you can use the key to automatically connect and copy important files from a remote host (which is the point of all this), the potential exists for others to do this too. Seriously consider if you want to endure the risk this entails, and protect the key from all eyes if you do.

To generate the passwordless SSH key, use this command or something like it (using 'man ssh-keygen' as your guide):

[root@localhost ~]# ssh-keygen -t dsa -b 2048 -f /root/cron/localhost-rsnapshot-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in /root/cron/localhost-rsnapshot-key.
Your public key has been saved in /root/cron/localhost-rsnapshot-key.pub.
The key fingerprint is:
2e:28:d9:ec:8d:21:e7:ff:73:df:2e:0a:78:fb:d0:a0 root@localhost.example.com

and the two files will be created in the above mentioned locations [2]. The public key (the one with the '.pub' file extension) will be copied to remotehost.example.com and inserted in a authorized_keys file (that belonging to remoteuser, specifically), as can be seen here. The private key file will be used to make the SSH connection from localhost.example.com, and is the key that requires the utmost security.

In a SSH config file you can create a text label for a connection, and assign various configuration options to it, without interfereing with the options set for other connections. The default file for root on localhost.example.com is /root/.ssh/config. Create the file if it does not exist already, and add the following to the end of the file (using your hostnames and file locations in place of the example contents):

Host remotehost-rsnapshot
Hostname remotehost.example.com
IdentityFile /root/cron/localhost-rsnapshot-key

You can use these commands to make sure the file has the correct permissions and contents (again, using your hostnames and file locations):

# if [ ! -d ~/.ssh ]; then mkdir ~/.ssh ; chmod 700 ~/.ssh ; fi
# cd ~/.ssh/
# if [ ! -f config ]; then touch config ; chmod 600 config ; fi
# echo Host remotehost-rsnapshot >> config
# echo Hostname remotehost.example.com >> config
# echo IdentityFile /root/cron/localhost-rsnapshot-key >> config

These configuration options will make sure the host designation in the Rsnapshot configuration file will use the correct key and will not have pass anything to 'ssh' (via the '-i' command line option) to make sure that key use used. The 'remotehost-rsnapshot' host designation can be used on the command line, but the 'rsnapshot' in the name should remind future configuration file editors for what the host and key are used.

To avoid any problems with interactive host key recognition, a host key for the remotehost must be added to the SSH initiator's known_hosts file (usually $HOME/.ssh/known_hosts). On a system configured to add keys automatically to known_hosts upon first connection, after prompting the user to do so, all you must do is connect to the remote system as the user who will be running Rsnapshot ( root in this case):

[root@localhost ~]# ssh remoteuser@remotehost.example.com
The authenticity of host 'remotehost.example.com (2.3.4.5)' can't be established.
RSA key fingerprint is 34:c1:29:42:67:b3:8d:ad:ba:50:34:d1:ab:d5:9a:aa.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'remotehost.example.com,2.3.4.5' (RSA) to the list of known hosts.
remoteuser@remotehosts.example.com's password:
Last login: Sun Apr 9 22:21:09 2006 from localhost.example.com
[remoteuser@remotehost ~]#

It is important to add remotehost.example.com to the known_hosts file for root on localhost.example.com or Rsnapshot will fail because the connection will not be allowed until the question is answered. That file is located here: /root/.ssh/known_hosts. The reason the question is posed is the SSH configuration option 'StrictHostKeyChecking' which defaults to 'ask' in many installations. The easy (and less secure) way of avoiding this problem is to set 'StrictHostKeyChecking' to 'no'. This can be done under the 'remotehost-rsnapshot' part of the /root/.ssh/config config file.

If SSH host keys must be added to the known_hosts file manually, consult the 'man' pages for ssh and sshd for formatting information.

SSH key restrictions are made on a per-key basis, and they reside in the authorized_keys file:

from="10.1.1.1",command="/home/remoteuser/cron/validate-rsync" ssh-dss AAAAB3Nza
C1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4uA+2qx9JNorgdrWKhH
SKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz5tVGfZe6ydlgomzj1
bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP5IaCuYBhuTKQGa+oy
H3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh4oyX/aXEf8+HZBrO5
vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55Kk2rAAABAE/bA402V
uCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5
c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583Kr
cWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzU
mNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4
cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9
MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79mbt1OE8LS9ql8trx8q
yIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn75Cfzhv65hJkCjbiF
7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMWyJNej2Sia70fu3XLH
j2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= root@localhost.example.com

There are 4 fields here, all on a single line, all separated by spaces: restriction, key type, key text, and key comment. Originally, when the key was first generated by "ssh-keygen", there were just the last three fields. The first is added onto the front to restrict what kind of connections can be made with this key. Two separate restrictions are in this example's restriction field to emphasize that the separator between them is a comma character (,) and not a space. You will not be able to connect with a key that has too many, or too few fields, due to a mis-editted key line. SSH will simply fail to authenticate with that key, then move on and ask you for your password [3].

The 'from="10.1.1.1"' part only allows connections to come from a certain IP Address, "10.1.1.1" in this example. The "command=/home/remoteuser/cron/validate-rsync" part only executes the designated command. This may seem very limiting, but you may write the script yourself and allow many different commands. You even have the actual command line given to SSH to work with in the form of the $SSH_ORIGINAL_COMMAND environment variable. An example script is located here: validate-rsync; and other options are included in surrounding footnotes.

The Rsnapshot configuration file is, by default, /etc/rsnapshot.conf. It contains quite a few configuration options, most of them in "option [tab character] value" format. Remove the 'comment' character ('#') to the left of the 'cmd_ssh' option so that part of the file looks like:

# Uncomment this to enable remote ssh backups over rsync.
cmd_ssh /usr/bin/ssh

Confirm that the 'ssh' command is where the file says it should be (with the 'which ssh' command).

Scroll down a bit and make sure the 'ssh_args' configuration option is uncommented and contains the following [4]:

ssh_args -o BatchMode=yes

The only reason I can see for not wanting to do this is if you plan on using Rsnapshot interactively and you want to provide SSH credentials for some connections. That is an unusual situation and probably best addressed with separate and customized 'rsnapshot.conf' file.

Further down in the file, insert a new 'backup' line that looks similar to this:

backup remoteuser@remotehost-rsnapshot:/usr/local/apache/htdocs/ remotehost/

Notice the 'remotehost-rsnapshot' text from the SSH config file, where normally a hostname would be found.

To make sure there are no Rsnapshot related errors in the configuration file, run the following command when editting is completed:

# rsnapshot configtest
Syntax OK

The second line should appear as above if everything is correct.

SSH server options, in the context of this Rsnapshot configuration, are set on remotehost [5]. The SSH server configuration file will typically be found at /etc/ssh/sshd_config and by default contains many of the configuration options set to their default values and commented out. This file controls SSH connections on this server, so be careful not to "secure away" functions you or others actually need.

PermitRootLogin can be used to make a SSH installation more secure in one of two ways. When set to "no":

PermitRootLogin=no

no connections can be made as the root user. When set to "forced-commands-only":

PermitRootLogin=forced-commands-only

connections may be made as root, but only from connections using authorized keys associated with a command in the authorized_keys file (as mentioned above).

The 'AllowUsers', 'AllowGroups', 'DenyUsers', and 'DenyGroups' key words can be used to restrict SSH access to particular users and groups. They are documented in the man page for "sshd_config", but I will mention that they all can use '*' and '?' as wildcards to allow and deny access to users and groups that match patterns. 'AllowUsers' and 'DenyUsers' can also restrict by host when the pattern is in USER@HOST form.

You could create a group called sshusers, add only necessary users to it, put this:

AllowGroups sshusers

into your /etc/ssh/sshd_config file, then give your SSH server a restart and test it out.

Run an hourly backup to test the configuration:

# rsnapshot hourly

If things don't work out, you may have to try running an hourly backup multiple times. To keep from messing up Rsnapshot backups you may already have in place, you may wish to move the current hourly out of the way before testing. Remove (or move) each test before running the next so as not to displace normal hourly backups. If you have a long running Rsnapshot configuration, you may want to comment out some of the backups, or possibly use a separate testing Rsnapshot configuration file altogether.

Notes:

[1] I think Rsnapshot is usually configured to run as the root user so that all files, especially those only accessable by the root user, are backed up without trouble. Many problems, and a considerable amount of fooling around, is avoided by running Rsnapshot as root.

[2] The '/root/' directory is the root user's home directory. To use another user, replace '/root/' where you see it with the location of the (hopefully) unprivileged user's home directory (like '/home/thisuser/' for the 'thisuser' user).

[3] You can see what fails and what succeeds during a SSH connection by using the 'verbose' command line options: '-v', '-vv', or '-vvv' (more v's meaning more verbosity).

[4] Thanks to Martin Shroder for pointing this out.

[5] To affect settings for the SSH client on 'localhost' changes should be made to the '/root/.ssh/config' file mentioned above, or if more global changes are necessary, the '/etc/ssh/ssh_config' file.

[6] This document is really not complete, but hopefully it can be of some help for now.