next: Web Browsers and Plugins on Linux »

Secure Remote Backups via rsync

# Filed on Oct 13, 2005 by Anthony / 4 replies

The Problem: you want to back up an entire (Unix/Linux/BSD) system over the internet, and both the source and destination machines have dynamic IP addresses.

Mark I: rsync, using SSH for the transport, as a normal user

[dest]# rsync -av --delete user@source:/ /mnt/backup/

This doesn’t work because your normal user, user@source, will be unable to read some of the files on the source system, since they are readable only by root.

Mark II: rsync, using SSH for the transport, as root

[dest]# rsync -av --delete root@source:/ /mnt/backup/

If this works, then you are allowing root login over SSH on your source system, which is horribly insecure.

So a normal user account can’t read all the files we want to backup, but we don’t want to allow root to login over SSH...

Mark III: rsync, using the rsync server for the transport

If we use the rsync server (run "rsync --daemon") on the source system, we can run it as root and it can therefore read the entire filesystem.

/etc/rsyncd.conf on the source system:

[wholefs]
uid = 0
gid = 0
path = /
transfer logging = no
read only = yes
hosts allow = 1.2.3.4

Now assuming that the destination system is 1.2.3.4:

[dest]# rsync -av --delete source::wholefs/ /mnt/backup/

This is fairly secure because the rsync server is read-only, and because only 1.2.3.4 can access it.  However it does require opening port 873 on the source’s firewall.

The problem is that the destination machine is on a dynamic internet connection so tomorrow its IP might be 5.6.7.8.  We can use a DynDNS/ZoneEdit hostname and a client like Eponym to name our machines source.dyndns.org and dest.dyndns.org, and they’ll always point to the current IP; then set "hosts allow = dest.dyndns.org".  But the problem now is that rsync does a reverse lookup of the hostname’s IP to verify it; ideally it’d go dest.dyndns.org -> 1.2.3.4 -> dest.dyndns.org, but in reality it goes dest.dyndns.org -> 1.2.3.4 -> 1-2-3-4.foo.my-isp.com.  rsync (rightly) sees this as a potential security problem so it won’t allow access.

Mark IV: rsync, using the rsync server for the transport, tunnelled through SSH

Again, rsync server (run "rsync --daemon") on the source system, but "hosts allow" is now different:

/etc/rsyncd.conf on the source system:

[wholefs]
uid = 0
gid = 0
path = /
transfer logging = no
read only = yes
hosts allow = 127.0.0.1

Now from the destination system, log into the source system via SSH as your normal user account, and create an SSH tunnel with the -L option:

[dest]$ ssh -L 8873:localhost:873 user@source

Now we have a secure encrypted tunnel from port 8873 on dest to port 873 (the rsync daemon port) on source.

[dest]# rsync -av --delete --port 8873 localhost::wholefs/ /mnt/backup/

This setup is very secure because the rsync server only accepts connections from the local system (127.0.0.1), and because we don’t need to open port 873 in the source system’s firewall, since all the traffic is tunnelled through the SSH connection on port 22.

Final Notes

rsync can be run as a normal user on the client (dest) system, but in order to make an exact backup (preserving file permissions) it must be run as root.

In the rsync calls, I omitted any --exclude switches for the sake of brevity.  But in reality when backing up the root of a filesystem, you’ll probably want to:

--exclude /dev
(because device nodes are usually created dynamically)

--exclude /media
(exclude any USB memsticks, pendrives, etc)

--exclude /mnt
(exclude any mounted CDs, floppys, etc)

--exclude /proc
(because proc nodes are created dynamically)

--exclude /sys
(because sys nodes are usually created dynamically)

Please contact us with any questions or comments.

Comments:

01. May 9, 2008 at 05:05pm by Jerret Kinsman:

Thank you very much for this post, it’s exactly what I was looking for!

02. Sep 16, 2008 at 03:57am by xaethos:

Excellent post!  You just saved me a great deal of time.  Now to write some scripts and crons.

03. May 14, 2010 at 03:09pm by Gerry:

Firstly fantastic article. I’ve read a lot of them on the topic of Rsync backups and this is by far the best I have found. Unfortunately I had to come to the solution of using the Rsync daemon to make everything read-only before I was able to track down your article in Google. Oh well. :)

However you say this:
"If this works, then you are allowing root login over SSH on your source system, which is horribly insecure."

I’m guessing what you’re saying here is that it’s bad because a compromise of the destination system would mean a compromise of the source system(s).

But the solution you give, while better still would allow anybody who had compromised the destination system to grab all of the private keys on the source system(s). I’m a linux system admin noob, but wouldn’t this situation be almost just as bad? At the very least doesn’t it mean that the attacker can pretend to be the source machine and connect to anything that it can connect to?

On another note, I discovered a "Mark II.V" while researching this topic. A number of sites recommend using ’sudo rsync’ along with a rsync sudoers restriction as a safer alternative to giving root access. However what they didn’t realise is that their sudoers restriction is meaningless because the user can just use ’sudo rsync’ to read and then overwrite their sudoers file. You may wish to add this to your fantastic article.

04. May 17, 2010 at 01:16pm by AnthonyDiSante:

Thanks for the comment, and the kind words!

Quoting Gerry:

However you say this:
"If this works, then you are allowing root login over SSH on your source system, which is horribly insecure."

I’m guessing what you’re saying here is that it’s bad because a compromise of the destination system would mean a compromise of the source system(s).

I just mean that allowing remote root login, on any system, is a bad idea.  A typical brute-force attack needs to determine both a username and a matching password for the system under attack.  But "root" is a standard built-in username on Unix/Linux systems, so on systems with remote root login enabled, the attack is potentially far less difficult (and/or much faster).  And as a bonus, if the attacker does gain access, he now has root-level access (i.e. he fully owns the machine) instead of merely having user-level access.

Reply to this message here:

Your name
Email (why?)
Website (if you have one)
Subject
search posts:

[ home | archives ]

Client Quotes

I just installed the demo of your product and got it up and running in no time.  I searched high and low for a decent login script and thank God I found yours.
– Adrian F.
I spent ages trying to find a way of making my own log in page for my website - if you're thinking of doing that forget it - don't waste your time!  UserBase is a 1st class product at a very reasonable price.  The software works faultlessly and can be adapted to any situation.  The service that I have received from Encodable is terrific!  I am very very impressed.  Nothing was too much trouble and I am most grateful to Anthony DiSante in particular for all his help and patience.
– Paul S.
Worked like a charm... man, this piece of software is a dream and I really appreciate all your customer service help getting this taken care of.
– Kyle M.
I just want to say you guys really stand alone in that you have a quality product and you provide genuine customer service.  It's sad but those qualities are seldom found separately, much less together.  Thanks again for your time and help.
– Alex S.
Also, I wanted to tell you that I was very skeptical about buying this script.  I've spent a lot of time and money over the past 3 months trying to find a solution that works, but I ended up having problems with so many of the scripts I tried that I was almost to the point of giving up.  But then I came across your script, and it actually does what it's supposed to.  An absolute wow.  A very impressive and powerful script indeed!  Many, many thanks!
– Mike E.
I can't thank you enough, I was up against a deadline that required me to get this up and running in 48 hours and you have probably the best customer service I've ever seen.
– Dan T.
Your scripts/software are the greatest, I mean I really love how customizable they are, how intuitive they are, and so on.  Thanks again, I love this stuff!
– Tucker O.
We searched for a long time for an application to password protect directories and allow file uploads.  Userbase & Filechucker are far superior to anything out there.  Simple yet powerful programming, extremely flexible in configuration, and great customer service.  Thanks for a superb product.
– Kat G.
Thank you VERY much for all of your help.  You've really impressed me.  We have support agreements for other software that costs thousands of dollars / year (just for the support), and most of them aren't as helpful as you have been.
– Keith Y.
There are a lot of these scripts out there, but I think they all pale in comparison to yours.
– Peter W.
The software has some great features, is well presented, runs where others are problematic and will make a good impression on our clients.  We look forward to reaping its benefits!
– Alex H.