View; MT.commentIds = []; The Importance of Good Backups - ChaseVenters.org

The Importance of Good Backups

| No TrackBacks

I know, I know.

I'm preaching to the crowd. But I suspect I may not be preaching to the converted.

Everyone has heard the mantra since the beginning of their computing lives: back up your important data! But it's incredible to me how many computer literate people still get it wrong. Even when they give the right advice to their novice friends, they fail to implement the correct strategies in practice.

When I was a teenager, I accidentally ruined a friend's semester-long research project. I was helping him install a new hard drive. He didn't need my help, but back then it was still really exciting to get a new drive, to crack open the case and to install new gear. He decided to trust my butterfingers to the task of reconnecting power to his old drive. Hard drive molex power connectors are supposed to be impossible to insert backwards, but I couldn't see into his case to monitor what I was doing, and managed to connect it backwards anyway.

He had no backups, and after paying hundreds to a data recovery expert, he never recovered his project. Sorry David!

More recently, a friend encrypted his laptop hard drive and discovered that he could use Unicode characters in the password. The only problem was that while the UI accepted those characters in the screen where your password is set, it did not accept those characters on the screen where you decrypt and log in. Whoops.

This weekend, my RAID array finally decided to crap out. I'd been operating a RAID 10 made of 4 Seagate 1 TB drives. All of the drives had been through at least one RMA cycle. (They don't make these things as reliable as they used to. But it's definitely not just Seagate.) After dealing with rounds of RMAs, I got lazy and ran the array in degraded mode for months. When you lose drives in a RAID 10, it effectively becomes a RAID 0. A RAID 0 is more dangerous than a bare hard drive, since a failure of any single disk in the stripe will destroy the array.

I'm not totally irresponsible, mind you. I have nightly automatic backups of my most critical data, courtesy of Rsync.net and Duplicity. I even tested the backups after setting them up.

But what I did not test is that all of my expected files were present in the backup. I misconfigured Duplicity and caused it to ignore my most critical directories - those with the source code of everything I've been working on for many years.

Many copies of these projects exist at different places - work, GitHub, etc. But there were some critical projects that were either not up to date, or not mirrored anywhere at all.

You can imagine my swearing and my feelings of panic. Thankfully, after I calmed down and carefully reassembled the RAID array, I was able to recover everything of importance. Nowadays, Linux is pretty resilient when reading from failing media.

Always back up your important data. Do it regularly. Back it up in more than one way. Don't assume your RAID array will save your data. Don't assume your backups will work, or will contain everything you need. Don't run your RAID arrays in degraded mode, don't let your backup process fail and not fix it, and don't forget to test your backups!

Implementing backups is boring. Testing backups is really boring. But some day you might be really glad you did.

No TrackBacks

TrackBack URL: http://www.chaseventers.org/cgi-bin/mt/mt-tb.cgi/49

The data is modified by the paginate script