Backup Strategy

[WhiskeyTango]

I have no idea where to put this, so if I'm in the wrong place, I'm counting on the mods to move me... (apologies in advance)

I'm considering a backup strategy involving a 2 drive RAID 1 array. I'm thinking about buying (3) 2Tb drives, (2) of which will be in the array at any point in time, while the 3rd is in a safe deposit box offsite. On a periodic basis (weekly/bi-weekly/monthly, etc.), I'd pull (1) of the drives in the array and swap it for the one in the safe deposit box, then rebuild the mirror.

Thoughts?

 

[Dao]

I do not think it is a good idea for breaking the array and rebuild it. You may be better off get a USB 3.0 dock and 2x internal drives (or 2x USB 3.0 external drives) and use them as backup. Rotate them and store one as off site backup. You can sync the files over with a script of 3rd party software. In Windows, you can use syncback.

Here is what I have.

- RAID 5 setup (more storage)
- A external USB 3.0 drive for backup
- Another internal drive on the USB 3.0 dock for another set of backup that I keep in my work place. Bring home once in awhile for backup.

 

[nmoody]

I have a main desktop which gets backed up by a Windows Home Server 2011 to a 2TB raid5. That Array also gets backed up to another 2TB raid1. This is a bit costly but does have some nice redundancy.

Stuff that I find supper important gets put in my dropbox which is an offsite backup.

I hear a lot of people on this forum also use Carbonite for an off site backup. This one works well and is reasonably priced.

 

[BuS_RiDeR]

RAID 5 is probably the best way to go... But it requires either an internal RAID controller capable of RAID 5 or an external enclosure with a built in RAID 5 controller; as well as a minimum of 3 (or more) drives (all the same).

See... THIS.

 

[WhiskeyTango]

I'd considered RAID 5, but I'm discounting it a bit for a couple of reasons. The primary one is simplicity. I like the idea that, in the event of a failure, all of the data is on all of the drives, and I can simply plug one of the mirrored elements directly into any computer to retrieve it. With a RAID 5 array, I'd have to rebuild the array first.

I'm open to the non-RAID solution. It's actually where I started... On Windows, I'd used Microsoft's SyncToy as well as Karen's Replicator, to sync files between volumes. I'll look into a Mac solution. I have Delta Walker, which may actually have that functionality...

As for a dock, I've looked at Thermaltake BlacX and NewerTech Voyager. I like that the NewerTech is Firewire, but it seems to get lower reviews. Anything else I should be looking at?

 

[BuS_RiDeR]

One nice thing about RAID 5... you have 3+ drives... One fails, you simply replace it... you lose nothing. Its a very good strategy... But I understand your point...

 

[KmH]

Get the inexpensive book - The DAM Book: Digital Asset Management for Photographers

 

[WhiskeyTango]

Get the inexpensive book - The DAM Book: Digital Asset Management for Photographers

Apparently I'm going to have to breakdown and order a hardcopy of that book... I've been putting it off for a couple of months in hopes that there will eventually be a Kindle, or other e-book version, but alas...

In the meantime, I've read everything I can get my hands on, which means there's lots of competing opinions. I'd love to hear yours.

 

[Garbz]

One nice thing about RAID 5... you have 3+ drives... One fails, you simply replace it... you lose nothing. Its a very good strategy... But I understand your point...

This breaks down for two reasons.
1. RAID is for high availability not for data security. A RAID system has MANY common mode failure problems that make it not even remotely a replacement for a proper backup solution. Simple things such as user error (deleting files), house fire, lightning strikes, etc all take out your entire "redundant" array.
2. RAID5 has not been viable since drives started exceeding around 300GB. Statistically speaking you have half the chance of recovering from a RAID5 hardware failure than a RAID1 hardware failure with data intact. The only reason RAID5 exists is for scalability (i.e. it's much cheaper to make a 5TB RAID5 array than a 5TB RAID1 array. But conversely because of the size of dives it's cheaper to make a 4TB RAID1 array than a 4TB RAID5 array.
On the failure modes, harddisks are likely to die due to common causes. They reach their Mean Time To Failure at the same time, they experience the same heat, the same load, and as such when a RAID member dies the remaining drives in the array are often on the edge of their life. Now replacing the failed harddisk results in almost a full day of continuous THRASHING of the disks. Very many RAID5 arrays fail during a rebuild which is exactly why most places with any kind of valuable data have moved to RAID6 (two parity drives).


Now as to what you should do: A backup needs to be separate from all common mode causes of failure. I.e. not connected to the computer, not stored in your home, etc. Your safety deposit box is a good idea. But rather than swapping the drives and horrendously thrashing them you should do a mirror backup. There's free programs like Synchronicty which can do this for you. So when you bring your backup drive in at the end of the week for a refresh rather than copying some many hundreds of GB over again (which happens on a RAID rebuild) it'll scan the drives, check the last edited dates on files and copy only the new changes over. It's less stress on your drives and the backup time is cut from a day to probably half an hour.

As for my earlier comments on a RAID failure the backup drive plays an important role there too. The first thing you do when you suffer a RAID failure is resist all temptation to rebuild the array. Bring in your backup drive and do a quick backup as described above. THEN use another drive to rebuild the array. That way if your RAID array falls over during the rebuild you still have current copies of all your data.

 

[480sparky]

If all your stuff is sitting on your desk (internal, external, and raid drives), what happens when your house burns down, or you have a tornado, or a flood?

If your work isn't at three physically separate locations, it doesn't exist.

Also, keep in mind safe-deposit boxes are not fire and flood proof. And fire-proof safes will not prevent a hard drive from melting into a puddle of plastic with electronics imbedded in it.

 

[Gaerek]

If you're looking for simplicity...

Drobo

It's a little spendy, but the benefits outweigh the drawbacks if you're looking at keeping your photos safe. Drop a drive in and forget it. They don't even have to be matched (like most RAID implementations do). If a drive fails, simply remove it and replace it with another. I'm planning to purchase a couple of these in the future, but at the moment, a bit out of budget.

If all your stuff is sitting on your desk (internal, external, and raid drives), what happens when your house burns down, or you have a tornado, or a flood?

If your work isn't at three physically separate locations, it doesn't exist.

Also, keep in mind safe-deposit boxes are not fire and flood proof. And fire-proof safes will not prevent a hard drive from melting into a puddle of plastic with electronics imbedded in it.

And I'd like to second this. Having a single onsite backup of your primary data is a good idea, but make sure you have AT LEAST one backup moved off-site. There are several ways of doing this. Carbonite and Mozy both offer good solutions. There's other ways of accomplishing this as well, but those are the easiest.

 

[WhiskeyTango]

The problem I have with Drobo is that it's a proprietary interface. I'm less worried about individual drive failures in a Drobo system, but if the Drobo itself fails I can't recover until I've got a new unit...

The offsite storage of a drive is one piece of what I have in mind.

My primary, i.e. "working" drive is a 2Gb Raid1 array. It is currently backed up to a single 2Gb external drive. I haven't yet implemented offsite redundancy, and thus the question to this group. I have considered a (3) site strategy, but am/was discounting it due to what I perceive to be minimal risk of catastrophic loss at (2) sites. A 3rd drive and a firesafe is pretty cheap, though, so perhaps I need to reconsider.

Also worth noting: I'm a hobbyist who occasionally takes paid work. I'd like "occasionally" to become more frequent, but I'm not trying to "force" that and photography is not/will not be putting food on my table or clothes on my kids anytime soon. I can afford to scale up my backup strategy over time. I simply want to start on the right road.

Garbz, I love the idea of doing a sync between the two backup volumes rather than a full copy over. My only concern with that is that it puts both volumes, along with the primary drive, all in the same location at the same time. Since all of my volumes are the same size, I wonder if I'd be safer to swap the volumes at the offsite location and then sync the stale backup drive to the current working drive. Seems like I'd accomplish the same thing without ever putting all of my backups under one roof...

 

[Gaerek]

People have given you all sorts of ideas and you always go back to your original plan. It seems like you have already made up your mind as to what you want to do. Continually breaking a RAID 1 is such a bad idea, not to mention extremely inconvenient. The less handling you have to do the better. You should only remove a drive after a failure. If your solution requires you to do it more than that, you're doing it wrong and you'll run into more issues down the line. People continually forget the fact that hard drives are 50 year old technology, and although the technology has gotten better and more robust with age, hard drives are still very fragile. A grain of dust inside, and you can potentially ruin the whole drive. Thus, the less handling the better.

You're going to come up with compromises with whatever it is you decide to do. There is no perfect, end all solution. Your concerns about the Drobo are certainly valid, but your solution is far more prone to failure than the actual Drobo failing. Anyway, whatever you decide, I wish the best of luck.

EDIT:

When it comes to backups, here's what you can get.

1. Cheap
2. Simple
3. Robust

Now, choose two. If you want it simple and robust, it's going to cost you. If you want it cheap, but simple, your solution will be prone to failure. If you want cheap and robust, your solution will be complicated and inconvenient. What's your data worth? I don't make money off my photography, but I consider my photos very valuable. If I have to choose (and this is unrealistic, but you get my point) between saving my camera and lens (several thousand $'s) or saving the 16GB memory card full of photos, I'll take the memory card, each and every time. This is why I won't skimp on a backup solution.

 

[WhiskeyTango]

People have given you all sorts of ideas and you always go back to your original plan.

Whoa, hang on. Not at all. My "original plan" was to ditch the 2Gb external in favor of a 2nd array, swapping an array element weekly.

I've definitely taken the advice given to heart. Now my plan is to skip the 2nd array and simply swap out a set of 2Gb external drives.

I'm still VERY open to refinement, criticism, etc. I don't ask questions on subjects that I'm closed minded on. I apologize if I gave another impression.

Your point on spending for backup is also taken to heart. I've got damn near 20k invested in this "hobby" now. I don't mind spending money for a robust backup solution, I simply want to spend wisely and not foolishly.

 

[480sparky]

........There are several ways of doing this. Carbonite and Mozy both offer good solutions. There's other ways of accomplishing this as well, but those are the easiest.

I have a simple method. My work is backed up on two external drives (these are in addition to my 4-drive backup routine). One is kept at a neighbors house. This prevents me from losing everything in case my house burns down, a thief cleans me out, something like that. But it's not fool-proof when it comes to floods, tornadoes, large local disasters like that.

So a second copy is kept at a relative's house 872 miles away. Now, unless we have something like this happen, I think I'm fairly safe. And if we do, then I doubt I'd care about my stuff any more.