Image Organization - Storage

[The_Traveler]

A hard drive docking station may be useful in the very near future.

I backup, and have for several years, with a simple relatively low tech way.
I have an internal data drive and 2 external 1 tb backups drives with the externals connecting thru USB 3 ports.
Using Syncback SE from 2BrightSparks.com ( a totally great, infinitely flexible piece of software) , my data drive is backed up to these two separately (as is my wife's laptop across the network) automatically about 4:30 every day. (if my pc is asleep, it gets woken up for the backup and put back to sleep when its done)
The 1 tb drives are approaching 80% each and I am looking at transitioning to new 2 or 3 tb drives, one internal and two external in a drive cradle like

SATA QuickPort Duo USB3.0 -v.2- SHARKOON Technologies GmbH

or

EN-CAHDD-U3 Series - Cavalry

They support the infamous JBOD standard (Just a Bunch of Disks)

 

[TCampbell]

Please explain RAID use in terms of these articles where the emphasis is on two things:

RAID 5 is inherently dangerous because of the likelihood of data fail in large drives
RAID is not for backup​

The RAID5 delusion ZDNet
RAIDfail Don t use RAID 5 on small arrays ZDNet
RAID 5 Data Recovery National Leaders - Affordable Recovery
Dangers of RAID 5 Array with SATA Drives xByteIT

The articles are a bit alarmist in nature. All this was hashed out in the 80's about RAID 5. And these aren't really arguments so much against RAID 5... it's more of an argument against using SATA disks. Also... I noticed the authors fail to state an alternative that doesn't have the same problem using SATA disks.

One article, for example, suggests the use of RAID 10. There's no such thing as "RAID 10". It's actually "RAID 1+0". RAID 1 + 0 means that if you have 4 disks (and you MUST have a minimum of four disks to do this) then you take disk 1 and mirror it to disk 2. You take disk 3 and mirror it to disk 4. These are essentially two volume "groups". You then stripe across the two groups putting all even blocks in the first group and all odd blocks in the second group.

E.g. the four disk contain:

[TABLE="class: brtb_item_table"][TBODY][TR][TD]Disk 1[/TD][TD]Disk 2[/TD][TD]Disk 3[/TD][TD]Disk 4[/TD][/TR]
[TR][TD]block 0[/TD][TD]block 0 mirror[/TD][TD]block 1[/TD][TD]block 1 mirror[/TD][/TR]
[TR][TD]block 2[/TD][TD]block 2 mirror[/TD][TD]block 3[/TD][TD]block 3 mirror[/TD][/TR]
[TR][TD]block 4[/TD][TD]block 4 mirror[/TD][TD]block 5[/TD][TD]block 5 mirror[/TD][/TR][/TBODY][/TABLE]

If each drive is a 1TB drive, then you get a total of 2 TB of useable space.

Now lets suppose a disk fails... we'll pick disk 2. You're only marginally more protected than RAID 5 with a URE because if disk 1 fails or has a URE then you've still lost the whole filesystem.

Good hardware RAID controllers, however, will verify that their writes succeeded by doing a validation of the blocks as they are written (to ensure they can be read).

The GOOD part of the advice in the articles is that you shouldn't have just one copy of your data. If you care about it at all... then have AT LEAST one backup. I have TWO backups of everything and, in some cases, more.

A friend of mine from some years back worked for a data-recovery business. These are the guys who can recover the data from your hard drive even if it fails completely (smoke coming out of the box... controller is fried... motors won't work anymore... doesn't matter). They physically crack open the disk and remove the spindles, then re-mount them into a special machine, carefully align the heads, and get to work recovering your data. But when you use such a company, you're basically paying for a skilled worker for whatever their hourly rate is, plus benefits, plus overhead, plus profit... for the entire duration of the time they work on recovering your data. As he explained to me... there's basically no such thing as needing to have your hard drive recovered and having it NOT exceed $1000 for the service (and possibly substantially more.) Meanwhile there are some good inexpensive backup programs and an external hard drive can be as cheap as $50. This literally is the "ounce of prevention..." cliche.

I make a weekly "bootable" backup. This is basically a clone of my internal drive but if the internal drive ever does fail (and on a previous computer... it did just that) I literally had no downtime. I booted from the external bootable clone and kept on going while I placed an order for a replacement drive. Then cloned my data back onto the replacement drive when it arrived.

I also use the Apple "Time Machine" backups -- which backups anything that's changed every single hour of the data (after a week it collapses the hourly backups into daily backups to save space... and after about a month it condenses daily backups into weekly backups to save even more space... and it does this for as long as it can until it runs out of disk space and then purges the oldest backups automatically before starting new backups. This allows me to restore to almost any point in time.

Both backups saved my bacon at one point. One when my internal disk failed hard. The incremental saved me when I had a drive that didn't fail fast... it started corrupting blocks and I didn't know it. Those corrupt blocks were getting backed up. It was a few weeks before I tried to go back and open an image that I detected had been corrupted. So I went to restore it from my "cloned" drive and discovered that enough time had passed that the "clone" literally had a backup of the already-corrupted file (so that was useless). But since I had the Apple incremental backups... I just had to go back in time to find a point where the file was NOT corrupted to recover the data.

I STRONGLY advocate in favor of backups that are "just" backups (do NOT use your backup for extra storage space. That path leads to trouble.)

 

[Heather Koch]

After reading more of your guy's posts, I kept seeing the word "backup," well something hit me... I can just buy more storage for my computer which is less than the hard drive I bought, so I will return that and add 6 times as much storage!

Thanks everyone for you help, I really appreciate it.

 

[KenC]

After reading more of your guy's posts, I kept seeing the word "backup," well something hit me... I can just buy more storage for my computer which is less than the hard drive I bought, so I will return that and add 6 times as much storage!

Thanks everyone for you help, I really appreciate it.

If by "more storage for my computer" you mean a bigger or additional hard drive in the computer, then you're not really backing up because if your computer gets fried, there goes everything. My apologies if I misinterpreted this statement - just wanted to be sure.

 

[Buckster]

After reading more of your guy's posts, I kept seeing the word "backup," well something hit me... I can just buy more storage for my computer which is less than the hard drive I bought, so I will return that and add 6 times as much storage!

Thanks everyone for you help, I really appreciate it.

If by "more storage for my computer" you mean a bigger or additional hard drive in the computer, then you're not really backing up because if your computer gets fried, there goes everything. My apologies if I misinterpreted this statement - just wanted to be sure.

While that's possible, it's not certain. When your computer "gets fried" it could be just the power supply, the motherboard, the video card, one or more of the hard drives, etc. It doesn't necessarily mean that EVERYTHING, including all hard drives, got toasted, and it's quite possible that the hard drive(s) survived.

On the other hand, it's also worth mentioning that just because you have separate hard drives that are outside the computer's main system unit, that doesn't make them necessarily immune to the same surge damage either, especially if they're plugged in on the same circuit. It's not unknown for a power surge out on the main lines to "fry" a bunch of appliances or electronics in several homes on the same circuit even.

In any case, being that it IS possible, take precautions. Plug all electrical appliances that you wouldn't want "fried" into good surge protectors, rather than straight into the wall. There are now very good surge protectors you can get that replace the circuit breakers in your house as well.

 

[KenC]

Good point. I wonder if I should unplug my home hard drive when not in use. It's actually plugged into a second surge protector so I can switch it off easily, but one never knows I suppose ...

 

[W.Y.Photo]

I own a couple 2 terabyte harddrives.. one I use around the house and for transporting data and one that sits on the shelf as backup. I update these drives on a yearly basis since their spinning disks are very likely to fail, especially the one I use as a mobile drive.. Updating yearly also helps as I can increase my storage space as I increase the amount of data I am storing.

 

[TCampbell]

There are ONLY TWO types of hard drives in the world. Those that HAVE failed... and those that WILL.

If you value your images. Have a backup. A real backup.

 

[Rgollar]

I swear by readynas units. I have (6) 8tb drives in a raid 5 configuration. With this setup if one drives goes bad you just swap the bad one out. I have been using these units for 12 plus years and have had only one drive go bad. No big deal just swap it out. I will say to go with a hard drive that is approved for nas units as the are built for them. I just hope I never run out of space as I just upgraded all my hard drives and I have 37.5 TB free. But I never thought my last setup with 18 TB would run out of space wrong again.

 

[Heather Koch]

After reading more of your guy's posts, I kept seeing the word "backup," well something hit me... I can just buy more storage for my computer which is less than the hard drive I bought, so I will return that and add 6 times as much storage!

Thanks everyone for you help, I really appreciate it.

If by "more storage for my computer" you mean a bigger or additional hard drive in the computer, then you're not really backing up because if your computer gets fried, there goes everything. My apologies if I misinterpreted this statement - just wanted to be sure.

Really all I wanted/want is more storage. My computer literally has 2.xx GB left on it. Its slow and won't hold much more.

Though all of your advice is great and I have read and looked into it all. If you think about it, nothing is guaranteed. So at this point I can only afford so much and will get the extra storage for the computer. I have searched around for the best option and found something good, I hope.

 

[W.Y.Photo]

After reading more of your guy's posts, I kept seeing the word "backup," well something hit me... I can just buy more storage for my computer which is less than the hard drive I bought, so I will return that and add 6 times as much storage!

Thanks everyone for you help, I really appreciate it.

If by "more storage for my computer" you mean a bigger or additional hard drive in the computer, then you're not really backing up because if your computer gets fried, there goes everything. My apologies if I misinterpreted this statement - just wanted to be sure.

Really all I wanted/want is more storage. My computer literally has 2.xx GB left on it. Its slow and won't hold much more.

Though all of your advice is great and I have read and looked into it all. If you think about it, nothing is guaranteed. So at this point I can only afford so much and will get the extra storage for the computer. I have searched around for the best option and found something good, I hope.

Please atleast get a backup external harddrive. 2 TB is now around $100 that's very cheap compared to the value of every image you've ever made.

 

[bhop]

I have a couple external drives connected to my MBP. I don't keep much of anything on my Macbook except for apps and anything i'm currently working on. It runs smoother and more reliably without stuff on it from my experiences. When i'm done with projects, i'll move it to my external. I use the second external as a backup once a week (or that was the plan anyway.. it's more like once every couple months.. gotta get back on that)

 

[Heather Koch]

Ya'll were right!!!

I bought the 1tb seagate portable hard drive and have already returned it because I thought the extra 8gb of memory would do the trick. To much of my surprise, I didn't do enough research and failed. SO my father reminded me we already had a 1TB hard drive we used for our WII system. Its a seagate freeagent desk, which isn't portable but will do the trick and at free of cost!

Needless to say, 1 weeks worth of research doesn't do much unless you really know what you're doing... And wasted time order and returning items