Amazon EC2 + EBS and rsync as a quick backup/mirror
A few days ago a client came to me and asked how they could back up a lot of data on a nightly basis that only had a few changes. Many solutions with their current hosting providers were discussed, Amazon’s S3 and Glacier, but, none gave him the flexibility he was after. So, the logical conclusion here was Amazon EC2 + Elastic Block store (EBS).
We created a micro instance of Debian, logged in and added rsync, created a volume large enough to hold his backup with some room for growth and with a little command line magic he was able to mirror his data to EBS. Of course, the instance didn’t need to be running all the time so he would need to start it, run the rsync, shut it down.
After some thought I decided it would be easy enough to write a quick Python script using boto to start the instance, rsync the volume and stop the instance. If he needed access to the instance he could start it manually and log in when needed. Now, his backups could be run via cron on a regular basis.
I put the code on GitHub: https://github.com/cd34/spawncamping-octo-ninja
Most of the instance startup situations are handled and so far it seems robust.
Tags: Amazon ECS, backup, ebs, ec2, rsync