hard drive configuration - suggestions?

[pgriz]

I've had enough data and program setup lost over the years. Nowadays, my work computer has two drives, mirrored (raid 1). Same for my home-office. Same for my laptop. Each system has a system image backup on external HD. All data have their own external HD backup. There's a software package on each machine that synchronizes data across multiple platforms. So far, I've lost a contoller, and two hard drives (different machines) in two years, and no loss of data or system. For photography, I use external HD's that connect with either Firewire (1394), or USB 3.0. I've got a bank of 1TB drives that are synchronized on a daily basis. It all takes some care to set up, and I won't claim my setup is very sophisticated, but I do have enough backup for my purposes.

 

[MLeeK]

I am not as thorough at pgriz in my setup, but I am close.
My main hard drive has only the operating system and programs on it. NOTHING is stored there. Not my working files even. I think it's something like 500G. Not a very big drive because I don't need it for the OS and programs.
My SECOND internal hard drive is my working drive. It does have an OS partition on it IN CASE my main fails. On the larger part of that 1TB drive I import all of my files and then back up to an external. Each day I back up to that external after I have worked on any files. Each Sunday that external gets switched with one I store off site. The OTHER hard drive gets updated to include any files/changes from this week and I begin again.
I have a fireproof safe in both locations with a bunch of full hard drives (one copy here, one copy there) that go back over about 7 or 8 years. There are some CD's in there needing to be tx'd to hard drives still as well.

My Christmas present to me this year is a cloud system that will be off site so I don't have to do the trading out crap. It's a pain. However, I have lived through a fire and lost everything. Not business stuff, but all of my personal stuff and it's not fun. So, I deal with the off site headache. Each day I'll upload to the cloud and switch out externals with one I only keep in the safe. No more having to go to the other office for backup.

 

[Garbz]

The chances of both failing at the same time I would assess as very slim. Of course I would always recommend a CD/DVD back-up as well, but the OPs question I believe related to suggestions for the use of multiple drives.

The inherent problem with RAID is that identical drives typically feature identical MTTF numbers, and have typically been thrashed for an identical amount. The chances of both drives having a hardware failure are actually much higher than you think. This is one of the reasons RAID5 is not recommended for any system more than a couple of hundred GB, a second drive typically will fail while rebuilding a degraded array. RAID1 suffers the same problem. If you want to rebuild your RAID array after a harddisk failure you inevitably end up completely thrashing the working drive which may be on it's last legs.


In comparison the OP's idea of rotating drives in and out of service is great. Bonus for storing the spare drive offsite. It means you not only have an offline backup that will be resistant to viruses, deletions, and if you keep it offsite it's also resistant to tornadoes, fires, floods and theft, but the idea of rotating drives solves another issue of seized bearings by keeping them exercised.


RAID1 should never be used where a backup solution is needed. It should be used for availability. Ask yourself the question "What happens if the computer goes completely tits up right now?, or what happens if my house burns down?" If the answer is you lose everything then you need an offsite backup solution, if your answer is you loose the last months worth of new work then you need a RAID array.

The way this would work is that if a drive fails you instantly do an incremental backup of the last months worth of data. THEN you attempt to rebuild the array. Too many people make the mistake of rebuilding the array as the first step and end up disappointed.

 

[Gaerek]

I agree. To a certain extent. "RAID 1 (mirroring without parity or striping), data is written identically to multiple drives, thereby producing a "mirrored set"." Sounds to me like an excellent application of two identical HDDs. The chances of both failing at the same time I would assess as very slim. Of course I would always recommend a CD/DVD back-up as well, but the OPs question I believe related to suggestions for the use of multiple drives.

Not even to a certain extent. RAID 1 just mirrors two drives. What happens if you accidentally delete a file you were working on? Whoops, it's gone now (assuming you cleared your recycle bin/trash can)! What happens if you're working on a jpeg and you accidentally click 'save' instead of 'save as' (or in some other way, modify the original jpeg rather than modifying the copy)? Sorry, it's gone now. RAID stands for Redundant Array of Independent Disks. It's just redundancy. The sole purpose of RAID is to help counteract a hard drives true nature of failing at exactly the wrong moment. Essentially, a drive fails and you don't lose all your data. That is it, it is not backup. A real backup solution will allow you to retrieve files that you've deleted or modified in a non retrievable way. RAID is a good idea as part of your entire backup scheme, but not as the only backup solution.

Let me give you an example from just yesterday. I got a call from a frantic accountant who had an excel spreadsheet that pretty much logged all the work she has done for the year. The information would literally take weeks to recreate since she'd have to go pull up all sorts of hard copy forms and manually add them to a new spreadsheet. What she had done was somehow selected the entire sheet then deleted all the data. Then when she went to go click undo, she accidentally clicked save instead. So now all she had was a blank spreadsheet. If our only backup had been a RAID 1 or 5, she would be screwed because I wouldn't have had any backups to pull an older copy from. Luckily, an incremental backup had been completed the night before, I was able to restore the file, and she lost all of about 30 minutes of work.

This is why RAID is not backup, not even to a certain extent. It is redundancy.

EDIT: Thought I'd add that if a file somehow gets corrupted, the corrupted version gets mirrored as well. If you had a backup to pull from, you could get the non-corrupted version back.

 

[tirediron]

Okay... what did I miss? I've read and re-read the OP, and I can find no mention of "back-up" but rather requests for suggestion of multiple HDD. I offered one which I feel to be a reasonable implementation, and having worked with very large arrays in RAID 1 configuration, I will stand by my assertion that simultaneous failure is unlikely. I agree with Garbz' point that if one fails, the other is likely not far behind it, but given how cheap magnetic storage is these days, I would suggest replacement of all storage media at significantly less than MTBF. I also firmly believe that a RAID 1 or similar type configuration is a valid part of a back-up plan. NOTE: I do not for a second suggest that it is a back-up plan on it's own, but rather one facet of a plan.

 

[Gaerek]

Okay... what did I miss? I've read and re-read the OP, and I can find no mention of "back-up" but rather requests for suggestion of multiple HDD. I offered one which I feel to be a reasonable implementation, and having worked with very large arrays in RAID 1 configuration, I will stand by my assertion that simultaneous failure is unlikely. I agree with Garbz' point that if one fails, the other is likely not far behind it, but given how cheap magnetic storage is these days, I would suggest replacement of all storage media at significantly less than MTBF. I also firmly believe that a RAID 1 or similar type configuration is a valid part of a back-up plan. NOTE: I do not for a second suggest that it is a back-up plan on it's own, but rather one facet of a plan.

Here's the problem. His reason for using multiple drives is one of data protection. Sure, he never once mentioned the words backup, but the implication is that he wants a full on solution for data protection. RAID is an integral part of any solid data protection plan, however it is not a solution, in and of itself. It's not even a good suggestion for the use of his multiple drives without offering some kind of backup plan as well. It would be easy to mislead people into believing that RAID is a data protection solution unto itself. All I was trying to point out is that if you're looking for away to keep your data safe, RAID is not the right overall solution. RAID protects your data from a specific (albeit common) type of failure, but in no way protects your data from any other kind of failure. Believing that RAID will protect your data is extremely dangerous and will almost inevitably end with some kind of data loss.

I want to make it clear that I'm not anti-RAID by any stretch. However, it needs to be a single part of an overall data protection scheme, not the data protection scheme itself.

 

[nfp]

Just a suggestion but I think you wasted a lot of money for something that will make it harder than it has to be. You would have been better off buying an external inclosure and 2 RAID drives that would mirror automatically. No worries, always a backup. Then all you would have to do is plug it in once every so often (however often you want to back up - preferrably daily) and copy the folders over you want to back up. I don't see the need for partitioning; some laptops have a separate bay for a second harddrive. If yours does, buy another HD and install it. Will be a better solution in the long run.

 

[dakkon76]

RAID is a great high-tech solution for backing up data... but is overkill when it comes to backing up some pictures. What RAID is great at is load-distrubuting for extremely high-traffic applications that require fast read/write times... like a Web or Data server.

I guess if you're looking at doing a RAID array of any type, you should ask yourself if you're going to be able to recover the data yourself if one of the drives fails. If you're going to have to bring it to someone to recover the data off of it rather than get it back yourself, then it's probably not the right solution for you. Sticking to a simple backup that exists on a couple of different drives and keeping one of them off-site is an extremely simple, and extremely effective plan. You just have to make sure to keep rotating them, so that you don't go to plug in a drive 10 years from now and find that it doesn't spin up anymore.

The bottom line is, I think a lot of people like saying that they backup their data on a RAID array because it sounds bad ass. In reality, having some cheesy Western Digital backup utility sync your external drive while you sleep is just as effective, and actually MUCH easier for most people to recover if something ends up failing.

 

[Gaerek]

The bottom line is, I think a lot of people like saying that they backup their data on a RAID array because it sounds bad ass.

Let's be honest, RAID is quite badass, but only for what it was designed to do. I cringe when I see people use the words RAID and backup in the same sentence. RAID is redundancy. It's to mitigate the effects of a drive failure and to provide some other types of benefits such as performance increases (in some cases) and load balancing (in other cases). It is not a backup. It doesn't work like a backup. If you screw up and delete/modify+save/etc., your data is GONE.

Besides, if anyone wants RAID-like performance in an easy to use product, I usually point them to Drobo. Way easier than setting up a RAID yourself, and if a drive fails, all you do is pop out the bad one, and put in a new one and you're good to go.

 

[dakkon76]

The bottom line is, I think a lot of people like saying that they backup their data on a RAID array because it sounds bad ass.

Let's be honest, RAID is quite badass, but only for what it was designed to do. I cringe when I see people use the words RAID and backup in the same sentence. RAID is redundancy. It's to mitigate the effects of a drive failure and to provide some other types of benefits such as performance increases (in some cases) and load balancing (in other cases). It is not a backup. It doesn't work like a backup. If you screw up and delete/modify+save/etc., your data is GONE.

Besides, if anyone wants RAID-like performance in an easy to use product, I usually point them to Drobo. Way easier than setting up a RAID yourself, and if a drive fails, all you do is pop out the bad one, and put in a new one and you're good to go.

So... basically everything I just said? Ok.

 

[Garbz]

Okay... what did I miss? I've read and re-read the OP, and I can find no mention of "back-up" but rather requests for suggestion of multiple HDD.

Which is a way of asking should the drive be configured as just another drive, a mirrored array, a stripped array, or a backup drive

RAID is a great high-tech solution for backing up data...

errr ok if you insist, just remind me not to trust you with any of my data

 

[Gaerek]

The bottom line is, I think a lot of people like saying that they backup their data on a RAID array because it sounds bad ass.

Let's be honest, RAID is quite badass, but only for what it was designed to do. I cringe when I see people use the words RAID and backup in the same sentence. RAID is redundancy. It's to mitigate the effects of a drive failure and to provide some other types of benefits such as performance increases (in some cases) and load balancing (in other cases). It is not a backup. It doesn't work like a backup. If you screw up and delete/modify+save/etc., your data is GONE.

Besides, if anyone wants RAID-like performance in an easy to use product, I usually point them to Drobo. Way easier than setting up a RAID yourself, and if a drive fails, all you do is pop out the bad one, and put in a new one and you're good to go.

So... basically everything I just said? Ok.

Sorry, should have made it clear I was agreeing with you. I wasn't referring to your use of RAID and backup in the same sentence.

 

[matt.garnett]

My main problem here is he spent money on 2 EXTERNAL hard drives. Internal hard drives are faster and cheaper, what's the deal? And RAID is not just a "high-tech" solution to back things up (because let me tell you, it sure as hell doesn't back stuff up!) or a fancy way to make it faster. I have 3 1TB Samsung Singpoints in RAID 1 and a 60GB SSD to hold all my applications on and moving files is still slow pretty sometimes. But that doesn't have to do with my hard drives. The thing is, is you can only increase your speed so much with external hard drives because you're restricted to how fast the connection between your computer and the hard drive is. The best way you can back you're data up is to get a program like Norton or something and back up all the files you want in safe keeping, to one of your external drives.

 

[Dao]

My main problem here is he spent money on 2 EXTERNAL hard drives. Internal hard drives are faster and cheaper, what's the deal? And RAID is not just a "high-tech" solution to back things up (because let me tell you, it sure as hell doesn't back stuff up!) or a fancy way to make it faster. I have 3 1TB Samsung Singpoints in RAID 1 and a 60GB SSD to hold all my applications on and moving files is still slow pretty sometimes. But that doesn't have to do with my hard drives. The thing is, is you can only increase your speed so much with external hard drives because you're restricted to how fast the connection between your computer and the hard drive is. The best way you can back you're data up is to get a program like Norton or something and back up all the files you want in safe keeping, to one of your external drives.

Actually USB 3 drives are quite fast. If not, go with eSATA. If you prefer lower cost of internal drive, you can get a dock that support eSATA or USB3, or both if any. With that, you can dock your internal drive.

Also, in real world server environment, it MAY need a fast backup solution because the server has a backup window. Like it need to finish the backup job within a set time each day. You do not want to start another new backup job before the previous one finish it's job. Or the server need to finish it's backup job before the peak hours.

But in photography type environment, that is not the case. It may take longer for the first time backup since you need to copy all your images. But once you have the full backup, you just need to backup the new changes. And so, backup speed is not too critical here.

I am using external USB3 dock allow me to dock an internal drive to it. And it is quit fast.