Secure Remote Backups via rsync

# Filed on Oct 13, 2005 by Anthony 8 replies

The Problem: you want to back up an entire (Unix/Linux/BSD) system over the internet, and both the source and destination machines have dynamic IP addresses.

Mark I: rsync, using SSH for the transport, as a normal user

[dest]# rsync -av --delete user@source:/ /mnt/backup/

This doesn’t work because your normal user, user@source, will be unable to read some of the files on the source system, since they are readable only by root.

Mark II: rsync, using SSH for the transport, as root

[dest]# rsync -av --delete root@source:/ /mnt/backup/

If this works, then you are allowing root login over SSH on your source system, which is horribly insecure.

So a normal user account can’t read all the files we want to backup, but we don’t want to allow root to login over SSH...

Mark III: rsync, using the rsync server for the transport

If we use the rsync server (run "rsync --daemon") on the source system, we can run it as root and it can therefore read the entire filesystem.

/etc/rsyncd.conf on the source system:

[wholefs]
uid = 0
gid = 0
path = /
transfer logging = no
read only = yes
hosts allow = 1.2.3.4

Now assuming that the destination system is 1.2.3.4:

[dest]# rsync -av --delete source::wholefs/ /mnt/backup/

This is fairly secure because the rsync server is read-only, and because only 1.2.3.4 can access it.  However it does require opening port 873 on the source’s firewall.

The problem is that the destination machine is on a dynamic internet connection so tomorrow its IP might be 5.6.7.8.  We can use a DynDNS/ZoneEdit hostname and a client like Eponym to name our machines source.dyndns.org and dest.dyndns.org, and they’ll always point to the current IP; then set "hosts allow = dest.dyndns.org".  But the problem now is that rsync does a reverse lookup of the hostname’s IP to verify it; ideally it’d go dest.dyndns.org -> 1.2.3.4 -> dest.dyndns.org, but in reality it goes dest.dyndns.org -> 1.2.3.4 -> 1-2-3-4.foo.my-isp.com.  rsync (rightly) sees this as a potential security problem so it won’t allow access.

Mark IV: rsync, using the rsync server for the transport, tunnelled through SSH

Again, rsync server (run "rsync --daemon") on the source system, but "hosts allow" is now different:

/etc/rsyncd.conf on the source system:

[wholefs]
uid = 0
gid = 0
path = /
transfer logging = no
read only = yes
hosts allow = 127.0.0.1

Now from the destination system, log into the source system via SSH as your normal user account, and create an SSH tunnel with the -L option:

[dest]$ ssh -L 8873:localhost:873 user@source

Now we have a secure encrypted tunnel from port 8873 on dest to port 873 (the rsync daemon port) on source.

[dest]# rsync -av --delete --port 8873 localhost::wholefs/ /mnt/backup/

This setup is very secure because the rsync server only accepts connections from the local system (127.0.0.1), and because we don’t need to open port 873 in the source system’s firewall, since all the traffic is tunnelled through the SSH connection on port 22.

Final Notes

rsync can be run as a normal user on the client (dest) system, but in order to make an exact backup (preserving file permissions) it must be run as root.

In the rsync calls, I omitted any --exclude switches for the sake of brevity.  But in reality when backing up the root of a filesystem, you’ll probably want to:

--exclude /dev
(because device nodes are usually created dynamically)

--exclude /media
(exclude any USB memsticks, pendrives, etc)

--exclude /mnt
(exclude any mounted CDs, floppys, etc)

--exclude /proc
(because proc nodes are created dynamically)

--exclude /sys
(because sys nodes are usually created dynamically)

Please contact us with any questions or comments.

Comments:

01. May 9, 2008 at 05:05pm by Jerret Kinsman:

Thank you very much for this post, it’s exactly what I was looking for!

02. Sep 16, 2008 at 03:57am by xaethos:

Excellent post!  You just saved me a great deal of time.  Now to write some scripts and crons.

03. May 14, 2010 at 03:09pm by Gerry:

Firstly fantastic article. I’ve read a lot of them on the topic of Rsync backups and this is by far the best I have found. Unfortunately I had to come to the solution of using the Rsync daemon to make everything read-only before I was able to track down your article in Google. Oh well. :)

However you say this:
"If this works, then you are allowing root login over SSH on your source system, which is horribly insecure."

I’m guessing what you’re saying here is that it’s bad because a compromise of the destination system would mean a compromise of the source system(s).

But the solution you give, while better still would allow anybody who had compromised the destination system to grab all of the private keys on the source system(s). I’m a linux system admin noob, but wouldn’t this situation be almost just as bad? At the very least doesn’t it mean that the attacker can pretend to be the source machine and connect to anything that it can connect to?

On another note, I discovered a "Mark II.V" while researching this topic. A number of sites recommend using ’sudo rsync’ along with a rsync sudoers restriction as a safer alternative to giving root access. However what they didn’t realise is that their sudoers restriction is meaningless because the user can just use ’sudo rsync’ to read and then overwrite their sudoers file. You may wish to add this to your fantastic article.

04. May 17, 2010 at 01:16pm by AnthonyDiSante:

Thanks for the comment, and the kind words!

Quoting Gerry:

However you say this:
"If this works, then you are allowing root login over SSH on your source system, which is horribly insecure."

Iím guessing what youíre saying here is that itís bad because a compromise of the destination system would mean a compromise of the source system(s).

I just mean that allowing remote root login, on any system, is a bad idea.  A typical brute-force attack needs to determine both a username and a matching password for the system under attack.  But "root" is a standard built-in username on Unix/Linux systems, so on systems with remote root login enabled, the attack is potentially far less difficult (and/or much faster).  And as a bonus, if the attacker does gain access, he now has root-level access (i.e. he fully owns the machine) instead of merely having user-level access.

05. Mar 5, 2013 at 08:55am by Tozz:

This design is very bad. With this setup you are giving away free root access to any user on your system.

/etc/rsyncd.conf is world-readable, so anyone could determine your "wholefs" block and start rsyncing everything from localhost.

This could happen by a user that has SSH access, login and then start rsync’ing from localhost. He could rsync your shadow file and start john the ripper. Basicly he can copy your entire filesystem and do whatever he likes.

It can also happen remotely by a badly protected website such as a compromised Joomla installation or an older version of phpMyAdmin.

Just because your rsyncd is read-only doesn’t make it safe. It only prohibits from writing files, but beeing able to read any file (includig password files) is just as insecure.

06. Mar 5, 2013 at 09:23pm by AnthonyDiSante:

Wrong.  The design is not bad at all.  It’s only bad if you take it out of context.  Most people don’t just let random other people have accounts on their PCs.  In fact, for most PCs, the only person with an account on the PC is the PC’s owner himself.  All of the potential risks that you mentioned are irrelevant in this most typical case.

07. Jun 5, 2013 at 04:32pm by jahvascriptmaniac:

Tozz is right, this technique gives full read access to the whole filesystem to anyone who has access to the source server.

This means normal users can read database passwords stored in php scripts owned by other users (therefore gaining read/write access to the victim’s database), for example, or like Tozz said copy and crack the shadow files.

You should only use this method if you 100% trust all users that may log-in on the machine (and if they mutually trust each other).

This should really be noted in the article, as a warning.

08. Jun 6, 2013 at 05:39pm by AnthonyDiSante:

As stated in the very first sentence of the article, the purpose of this guide is to backup an entire system over the internet.  This requires full read access, which is also clearly stated.  To add a "warning" like "warning: a person with full read access will have full read access" would just be silly.  If you allow untrusted users to have access to your system, then you have a whole additional set of concerns that aren’t relevant to the purpose of this article.

Reply to this message here:

Your name
Email (why?)
Website (if you have one)
Subject
search posts:

home | archives ]

Shopping Cart

Your cart is empty.

Client Quotes

The work, the thought and the organization you put into this app is incredible.
– Bruce C.
I looked all over trying to find a simple cgi script.  I found that FileChucker was by far the best.  If you have issues with your hosting service's php.ini max upload size then this is the way to go.  Looking forward to future enhancements.
– Bob C.
Nice script, it's saving the day on our project.
– Aaron W.
I just wanted to say that yours is the first product that I've tested so far that hasn't failed on handling uploads.  This is going to work for a print company, so they are handling nothing but large files and all the other solutions I've tried so far have not been reliable.  So far yours has been 100% successful in my tests.
– Kevin H.
I just installed the demo of your product and got it up and running in no time.  I searched high and low for a decent login script and thank God I found yours.
– Adrian F.
FileChucker is working great...  Clients love it.  Vendors love it.  We love it.
– Gerry W.
The amount of customization in the program is incredible.  I was able to integrate it into my existing page layout relatively simply.  I was also able to easily customize the look/feel to match the current site.
– Jason M.
Do you know how rare it is to have a "canned" shopping cart with the ability to fairly easily do complex pricing options on a single item?  Maybe you do, but if you don't... basically, they don't exist!  I have looked.  Everywhere!  And the few that might even come close to the functionality that you have in CornerStore cost a fortune!
– Tashina P.
Just one word: Fantastic.  10-minute job to plug FileChucker into my app, and it now works a treat.  It's through the hard work by people like yourselves that make my job so much easier.  Congratulations on an outstanding product... Many many thanks.
– Sean F.
Our members think your software is fantastic...  I would recommend your software and your company to anyone.  Thanks for all your help.  It has been a pleasure dealing with you.
– Tommy A.
FileChucker is helping drive the backend of several high profile entertainment sites for people like Shania Twain and Dolly Parton.  We're also using it to drive backend file uploads for a multi-billion dollar banking institution.  It's a great product.  We've tried other "chucking" upload solutions with progress bars using flash and php, but nothing works as reliably as FileChucker.
– Michael W.
Thanks again for a great product and great support - beyond expectations.
– Greg S.
FileChucker is a great drop-in solution for file uploads, and worth every penny of its very reasonable cost.  Encodable's support is excellent to boot.
– Loren A.
I just want to say you guys really stand alone in that you have a quality product and you provide genuine customer service.  It's sad but those qualities are seldom found separately, much less together.  Thanks again for your time and help.
– Alex S.
Thank you VERY much for all of your help.  You've really impressed me.  We have support agreements for other software that costs thousands of dollars / year (just for the support), and most of them aren't as helpful as you have been.
– Keith Y.