Saved digital images go bad

keiththirgood · Feb 20, 2012

Over the years I've found a number of saved digital images go "bad". They acquire stripes and other defects, sometimes minor, often major. These defects occasionally happen right at transfer from the card to my hard drive, at other times they have been opened at least once (often many times) and suddenly they are bad. This happens with jpegs and with Raw files as well.

I would upload a screen shot to show you what I mean, however, the system will not let me do so.

Any idea what might be going wrong and how to prevent it?
Keith

Bossy · Feb 20, 2012

Darn that system. Check out this thread on how to post images

Josh66 · Feb 20, 2012

They've been corrupted somehow.

Multiple back-ups on multiple media types is really all I can think of to prevent it...

Rephargotohp · Feb 20, 2012

Are you using Lightroom? sometimes in Lightroom the rendered preview will go bad even though the file itself is still good. You can delete the cache on that file and a new preview will be rendered

keiththirgood · Feb 21, 2012

I'm using photoshop and Preview on a Mac. However, the program doesn't matter. You can see the damage in icon mode without opening the file.

keiththirgood · Feb 21, 2012

Bossy,
The thread you sent me to says, regarding uploading photos directly"
"From the new page the first task is to make an album for your photos to go into, for ease of categorising your photos in the gallery. Click on the "Create new Album" link just above the main bars in the page. In the new window that opens you can enter a title, a short description and also set the privacy settings for the new Album. By default they are not set to private."
However on my browser, only "My Photos" "My Favorites" and "no messages" show up on that bar. There is no reference to "Create a New Album" or anything else that refers to albums.

MLeeK · Feb 21, 2012

failing hard drive is usually the culprit. It starts with a few sectors and blows from there.

keiththirgood · Feb 21, 2012

Interesting idea, however, my files are kept on a redundant raid drive, so that each file is written to two separate drives. It would be odd that the drives would fail on the same files at the same time.
There is over a half terabyte of files on the raid, and only the photo files are showing any issues.
I'll look at the raid with hardware tools to see if any significant sectors have failed.

KmH · Feb 22, 2012

Oops!

Garbz · Feb 24, 2012

keiththirgood said:
Interesting idea, however, my files are kept on a redundant raid drive, so that each file is written to two separate drives. It would be odd that the drives would fail on the same files at the same time.
There is over a half terabyte of files on the raid, and only the photo files are showing any issues.
I'll look at the raid with hardware tools to see if any significant sectors have failed.

Raid provides redundancy against hardware failures. It does not protect against corruption during access or replication in which case your files would be hosed on both drives. Also has it just shown up in picture files because it's obvious? You'd be amazed at the amount of corruption you can get on a computer before you start actually having problems. A document gets corrupt, you lose a letter or two and it comes up as a typo. An image becomes corrupt it shows up. A video becomes corrupt and you may not even notice the dodgy frame.

480sparky · Feb 24, 2012

Garbz said:
Raid provides redundancy against hardware failures. It does not protect against corruption during access or replication in which case your files would be hosed on both drives. ...........

I thought that was one of the pros of a raid array.... it can be set up to check one drive against another and find errors & etc.

bazooka · Feb 24, 2012

This has happened to me and I found that when I ran a memory check, at least one of my memory modules was bad so I replaced all of them. The corruption was happening at the time of the transfer from one machine to another.

Download memtest86 and burn it to a disk or put it on a usb drive and then boot from it. Run the test.

If the test shows errors, it may be the memory, or it may be the motherboard. Memory is generally cheaper and always easier to replace.

keiththirgood · Feb 24, 2012

The files were fine when I transferred them to my raid drive. The first thing I do after any photo file transfer is to go through the files one at a time to make sure they are okay. So the files have gone bad, just sitting there on the hard drive. They have not been written or re-written since they were checked after being written to the raid drive. I don't think the drive arbitrarily re-writes files as they sit there.

I haven't had the time to do a check of the hard drive, and any of the other mechanicals, but will do so tonight.

I'm still not clear what Garbz meant regarding raid drives. The corruption of the file is not happening when the files are being written to the raid. I check each file after it is transferred. The corruption is happening as the files are sitting quietly on the raid drive. That means two copies, which had been proofed have gone bad at the same time. How would it be corrupted on both drives at the same time, if it was not corrupted during transfer?

Garbz · Feb 25, 2012

480sparky said:
Garbz said:

Raid provides redundancy against hardware failures. It does not protect against corruption during access or replication in which case your files would be hosed on both drives. ...........

Click to expand...

I thought that was one of the pros of a raid array.... it can be set up to check one drive against another and find errors & etc.

On the raid hardware layer yes, how about the bus layer? Ram problems? CPU problems?

Actually that springs something to mind. keiththirgood is your computer otherwise stable? and by stable I mean ROCK SOLID, no app crashes in months, definitely nothing like a bluescreen or even basic image corruption while playing a game or something. Data corruption can occur in many places which is exactly why things like Error Correcting Code memory comes in to play. Often some specific things can trigger it which is why programs like Furmark which render a complicated image and then check the image to ensure it looks as expected exist.

As I mentioned there's nothing that could corrupt photos that couldn't also corrupt any other file on your computer. So let us know if there's any oddities.

keiththirgood · Feb 25, 2012

When you ask if my computer is rock solid, it gets a bit complicated. The files are stored on my server, and my partner and I access the files from any of four other computers. These are all Macs, so they fundamentally run on the Mac version of Unix, which makes them very solid.

I'm not sure about things like error correcting code, and furmark as they sound like Windows things.

I ran Tech Tools which checks all sorts of hardware related issues. The only test I didn't run was the surface scan, as it estimated it would take around 15 hours to run. (Each raid drive is one terabyte and the bigger the drive the longer a surface scan will take.) I'm going to let the surface scan test run tonight, as I don't need the server Sunday morning. If I find anything, I'll report back.