Best (Cheap) RAID / Storage Solution?

[Bifurcator]

I will say the same about your U.S. conspiracy theories and leave it at that.

I have no conspiracy theories! Not even one.

 

[Dao]

usayit. I agree with you. It is common that a lot of people misunderstand RAID technology. I'd been working in Data Center environment for more than 8 years, and I always come across with clients or prospects think RAID technology will give them reliability so that they do not need backup. The fact and the matter is, it only give them an option that they do not need to shutdown the server while it is in a recovery mode or fix it later (off peak hours). Will RAID increase the Reliability of the drive system? Yes it will, see this. (of course, I am not talking about RAID 0) However, by no mean it can prevent data loss.

As for the RAID 1, it is still very popular. Simple and works. If someone is not using expensive hardware RAID card, RAID 1 is the best approach. Software RAID or peusdo HW RAID (like most of the RAID that came with motherboard) is better to use RAID 1 unless you do not care about speed.

Also, if the system crash and you need to recover data from the dirve/drives, it is easier to do it with RAID 1.

 

[Bifurcator]

better to use RAID 1 unless you do not care about speed.

You mean unless you DO care about speed tho right? RAID1 is zero speed increase.

http://www.anandtech.com/storage/showdoc.aspx?i=2101&p=3 <-- read'em and weep.

 

[Dao]

You mean unless you DO care about speed tho right? RAID1 is zero speed increase.

I do mean "DO NOT" since Software RAID 5 (or RAID 6 if possible) will use too much CPU processing power, so the degrade in overall performance of RAID 1 is a lot less than RAID 5 with software RAID especially in rebuild mode.

If you use a true hardware RAID card that has a build-in processor to handle the RAID function is a different story.

If someone would like to increase the speed of a RAID set, they will usually stripe 2 of them to create a RAID 0 on top of the current RAID sets. i.e. RAID 1+0, or 5+0

 

[Bifurcator]

Yeah, that's kinda true for chipset-only RAID5. But since you mentioned RAID0 just before that I thought you were considering it as well. RAID0 scales the best while at the same time maximizing the storage space to number of units ratio. And it consumes no more CPU usage than RAID1 which of course you must know does not increase read or write speed (performance) at all.

1+0 is OK but kind of a waste of money and electricity. I at least contend that for a home user full unit mirroring - redundant parallelism - is about as useful as tits on a bull - they might be nice to have but the advantages are just about nil in the real world.

I can see periodic backups to something like a USB or Firewire drive tho.

Anyway I thought you were saying: "better to use RAID 1 [than RAID 0] unless you do not care about [file access and I/O] speed." which is of course false and what usayit and I were discussing.

 

[Moglex]

When a drive breaks 90% of the time it corrupts data for a few weeks or months first. In a RAID1 configuration all of that corruption will be copied over to your other drive(s).

You really do talk some unmitigated nonsense, don't you?

There is no path from one disk in a RAID array to another short of reading the data into the computer at which point any corrupt data will be identified.

So one drive going faulty will NOT cause corrupt data to be written over other drives.

OTOH, you are correct in saying that RAID1 is all but useless for the home user.

 

[Moglex]

How do you know if the drive was broken? In a RAID1 you'll know because all the read errors that happen while it busted will corrupt the image of the mirror. So you have an operational device that you can see is obviously trashed.

More complete nonsense, I'm afraid.

You seem to be claiming some sort of expertise in this field but you also do not appear to have a clue about the general systems architecture of even basic disk technology.

In case anyone has been misled by what you've written, a few basic facts:

1) Before a packet of data is sent to a disk the OS calculates some check data from the data to be sent. This is written with the user data.

2) The drive itself calculates check data that is written with the data sent from the OS.

3) The drive checks the data to make sure it has written the data correctly.

4) Upon reading the data, the drive checks that the check data tallys with the user data - if it doesn't it raises an error.

5) The RAID system (if in use) will check that both drives have returned the same data.

6) The OS will make its own check to see that the check data matches the user data retrieved.


So having one faulty drive will not corrupt the entire array. The array will not be allowed to function if its integrity cannot be assured.

We are agreed that RAID1 is pointless for end users (unless they have some very unusual application), but it is nowhere near as useless as you make out when used in the appropriate environment.

 

[Moglex]

I can see periodic backups to something like a USB or Firewire drive tho.

But you have already indicated that you believe that a faulty drive can go on writing bad data to a disk without detection even in a RAID1 array.

So what on earth would be the point of backing up that data (in your world)?


Of course, in actual fact, although a faulty drive can write bad data to a disk without flagging an error, as soon as that bad data is accessed the fault is discovered which is why:

1) RAID1 arrays do work.
2) You don't corrupt your backups with corrupt data from a faulty disk.

 

[JerryPH]

RAID1 Sucks nuggets! In a home computing environment it's ONLY for the extreme anal retentive who doesn't understand RAIDs at all. I seriously believe that. So do most of the IT old-timers I know.

When a drive breaks 90% of the time it corrupts data for a few weeks or months first. In a RAID1 configuration all of that corruption will be copied over to your other drive(s). Drives do break suddenly but it's rare and that's ALL RAID1 is good for. It's a total waste of money. You are buying twice the hardware and getting absolutely nothing for it except a higher power bill and probably more ambient room noise.

Personal Computing + RAID1 = SUCKS!

I'm not sure that you understand how RAID1 (mirroring), works based on what you said there.

In a software RAID1 configuration (lets say XP/Vista/Server scenario) if the primary drive dies, your machine doesn't boot. In a hardware RAID1 scenario, depending on the RAID hardware manufacturer, it may just beep, poke you in the shoulder, send emails, scream at you that one drive is damaged and keep the system up and running by automatically switching over to the "good" drive. In either case, if the secondary drive dies, you are notified one way or another (beeping, popup message on screen, etc...).

In a RAID setup, information is NOT continually copied from one drive to another any other time other than during the creation of the mirror (unless you happen to have some 1970's ESDI RAID1 controller?).

If info on 1 drive is corrupt BEFORE the mirror, and you create a mirror, THEN and only then is corrupted data written to the mirror. All other times, data in the cache (RAM) is written first to the primary drive and then that same data from RAM is written to the secondary drive (in an IDE or SATA setup) or written concurrently simultaneously in a SCSI setup.

Honestly, for a home user, a RAID1 external solution is probably the best bang for the buck. It means that they have 2 copies of whatever they copy to their external array. It also means that they don't have to purchase a unit with more than 2 drives and for archival purposes, do not need BLAZING speed to get the data there... just turn on the drive(s), copy the data there on a regular basis (once a week perhaps if they take that many pictures?) and turn them off right afterwards, even further increasing the lifespan (MTBF) of the hard drives.

Doing it this way means you are spending a lot less and accomplishing your goals... which is to find an effective and economical means of protecting your data.

Of course, to date, there are no realistic 100% effective solutions that last forever, so it doesn't hurt to hedge your bets by diversifying media. I use a SAN and a couple of 1 and 2 TB external drives, but I also save on my local hard drives (my local HDs on most of my computers are RAID0 of 3-4 drives depending on the computer... zero redundancy, but great for speed), and DVDs. It all depends on how much you value your data and how much you want to spend. One cannot get it much better in terms of least $ spent and effectiveness than an external 2 drive RAID1 setup.

 

[Moglex]

I'm not sure that you understand how RAID1 (mirroring), works based on what you said there.

In a software RAID1 configuration (lets say XP/Vista/Server scenario) if the primary drive dies, your machine doesn't boot. In a hardware RAID1 scenario, depending on the RAID hardware manufacturer, it may just beep, poke you in the shoulder, send emails, scream at you that one drive is damaged and keep the system up and running by automatically switching over to the "good" drive. In either case, if the secondary drive dies, you are notified one way or another (beeping, popup message on screen, etc...).

In a RAID setup, information is NOT continually copied from one drive to another any other time other than during the creation of the mirror (unless you happen to have some 1970's ESDI RAID1 controller?).

If info on 1 drive is corrupt BEFORE the mirror, and you create a mirror, THEN and only then is corrupted data written to the mirror. All other times, data in the cache (RAM) is written first to the primary drive and then that same data from RAM is written to the secondary drive (in an IDE or SATA setup) or written concurrently simultaneously in a SCSI setup.

Thanks for backing up my assertion that some of the posts above contain complete nonsense.

Honestly, for a home user, a RAID1 external solution is probably the best bang for the buck. It means that they have 2 copies of whatever they copy to their external array. It also means that they don't have to purchase a unit with more than 2 drives and for archival purposes, do not need BLAZING speed to get the data there... just turn on the drive(s), copy the data there on a regular basis (once a week perhaps if they take that many pictures?) and turn them off right afterwards, even further increasing the lifespan (MTBF) of the hard drives.

Doing it this way means you are spending a lot less and accomplishing your goals... which is to find an effective and economical means of protecting your data.

snip

One cannot get it much better in terms of least $ spent and effectiveness than an external 2 drive RAID1 setup.

I don't want to end up going around in circles but I still cannot see how a RAID1 setup can possibly be better than backing up to a cycle of 2 or more external hard drives.

I have a system whereby all new or changed data on the target drives is written to the external drive. This dramatically speeds up the process compared with just writing everything out (of course, you can do that to a RAID array as well). I have three drives which are used in turn with one always being kept physically separate.

The advantage of this scheme is that even if my system were to be stolen, burned beyond repair in a fire, or otherwise fatally damaged I would still have all my vital data. (Recent data is kept on a web based store until it has migrated to all backup drives).

The only advantage I can see to a RAID solution is that you don't need to keep track of which backup to use or move the off-site one around but these tasks seem very minor given the vastly greater security achieved.

 

[Dao]

I think it all comes down to if the cost for the RAID 1 solution worth the money. And it also varies from person to person.

Someone can get 2 or more of those RAID1 USB/eSATA storages and feel happy about it. However, some others may be tight in budget and can only afford one USB drive. Either case, it's still a solution.

Off Topic:
For those who like to play around with technology, have some older systems laying around and you as well as someone you can trust that have a decent speed broadband connection, you may want to take a look at OpenAFS. You maybe able to achieve automatic files repulications between 2 sites with the distributed Network Filesystem.

 

[Dweller]

Anyone know of a good online backup service that I could get to mirror my 500GB for cheap?

A thread on another forum I was reading this morning was discussing Carbonite, and people seem to really like it. I understand there are coupon codes that can be found online as well. They claim unlimited storage is available. I am looking at something like this myself since my current backup system still has everything in the same location.

PC --> server with data mirroring (Windows Home Server) --> tape drive --> (something offsite, likely Carbonite or something similar)

I have never lost a single photo despite some hardware loss along the way. I have dodged that bullet once, and I don't want to have to worry about it anymore.

 

[Dweller]

Off Topic:
For those who like to play around with technology, have some older systems laying around and you as well as someone you can trust that have a decent speed broadband connection, you may want to take a look at OpenAFS. You maybe able to achieve automatic files repulications between 2 sites with the distributed Network Filesystem.

Wow, thanks I was talking about something like this here a while back. I am not surprised to see a solution exists.

 

[usayit]

I don't want to end up going around in circles but I still cannot see how a RAID1 setup can possibly be better than backing up to a cycle of 2 or more external hard drives.

Backup is data protection.
Mirror is fault tolerance.

Backup scheme only doesn't provide the convenience of uninterrupted service. Once a drive fails, you have to restore the data from your latest image.

Mirror only scheme only provides fault tolerance. Once a drive fails, you have the convenience of continuing with your work. Replace the failed drive ASAP. If corruption (doesn't happen often) or both drives fail, you are out of luck. Data loss.. game over.

For a backup and mirror scheme, leverages both. Once a drive fails, you have the convenience of continued service from your machine. If a both drives fail or corruption, you have restores available from backup images. No interrupted service, no/limited data loss.

Place an importance on Backup. Mirror is the first line of defense against a disk failure. If data is important but RTO is not, implore backup scheme. If data is not important but RTO is, implore fault tolerance scheme. If data and RTO is important, implore both. If data and RTO is VERY VERY IMPORTANT and very valuable, purchase full RAID 5 or 6 with enterprise data protection.

RAID 1 IS the most simplistic and best bang for the buck for redundant disks... RAID 5 and 6 are of course superior but at a cost of additional hardware (software RAID 5 or 6 is highly CPU and resource heavy). People who say RAID 1 is expensive need to look at the hardware requirements for implementing full RAID 5-6. RAID 1 is no where the cost of RAID 5.

 

[Moglex]

Backup is data protection.
Mirror is fault tolerance.

Backup scheme only doesn't provide the convenience of uninterrupted service. Once a drive fails, you have to restore the data from your latest image.

Correct. But the OP wanted a cheap scheme and as an end user is very unlikely to want to double the cost of that solution to get high availability.

Mirror only scheme only provides fault tolerance. Once a drive fails, you have the convenience of continuing with your work. Replace the failed drive ASAP. If corruption (doesn't happen often) or both drives fail, you are out of luck. Data loss.. game over.

Correct.

For a backup and mirror scheme, leverages both. Once a drive fails, you have the convenience of continued service from your machine. If a both drives fail or corruption, you have restores available from backup images. No interrupted service, no/limited data loss.

Correct, but the OP said he wanted a cheap solution.

Anyway, this does not answer my objection.

Honestly, for a home user, a RAID1 external solution is probably the best bang for the buck. It means that they have 2 copies of whatever they copy to their external array. It also means that they don't have to purchase a unit with more than 2 drives and for archival purposes, do not need BLAZING speed to get the data there... just turn on the drive(s), copy the data there on a regular basis (once a week perhaps if they take that many pictures?) and turn them off right afterwards, even further increasing the lifespan (MTBF) of the hard drives.

Doing it this way means you are spending a lot less and accomplishing your goals... which is to find an effective and economical means of protecting your data.

snip

One cannot get it much better in terms of least $ spent and effectiveness than an external 2 drive RAID1 setup.

You have not explained how RAID1 can possibly be better at protecting your data when it only handles failure of a drive and does nothing to protect you against theft, fire or other physical damage whilst external backup protects against all four eventualities.

The above quote strikes me as entirely wrong. You certainly can get much better protection for your data by getting a couple of cheap USB disk caddies. You protect against the whole gamut of dangers mentioned for about £20 more than the cost of the drives.