Best Archive Format for RAW? (zip, rar, gzip, 7zip, etc)

[MushinVin]

Hi,

I'm new to this forum and new to digital SLRs in general, but I have a simple question. Is there a compression format, as in ZIP/RAR/etc., that is considered to be the best compression format for archiving RAW? I'm doing some work with a friend/colleague of mine and transferring multiple GBs of data every few days to and from each other's computers is getting a bit tiresome. I have a Nikon D40, so my flavor of RAW is NEF, though I do shoot in RAW+Basic (NEF + Low Quality JPEG), so I wonder if that would make a difference in the end as far as compression goes.

I have access to pretty much any compression format out there, zip, rar, 7zip, tar, etcetera etc.

Thanks in advance,

-MushinVin

 

[ann]

i keep 4 copies of every raw file, changing the ones i work with to tiff, then making copies of that as well.
works for m e

 

[craig]

I archive my work in DNG format. Compression is a killer all I can say is do not save your final copies in jpeg format. I have seen how bad it gets and it is not pretty

Love & Bass

 

[rufus5150]

You can compress RAWs with any loss less compression format (gzip, zip, 7zip, rar, etc) as they compress the data stream only, not the actual image contents. Don't save in JPEG format, but if you're concerned about space, you can use any archiving compression format or method you like.

 

[Bifurcator]

Probably 7zip compresses smallest. No?

It can compress MP3 and MP4 video and JPEGs (often embedded) and even compress ZIP (Huffman) compressed TIFFs down quite a bit whereas the others you mentioned there can't.

 

[Garbz]

If you're after compression many formats include built in lossless compression. The Nikon compressed-NEF format is NOT lossless regardless of what the marketing mumbojumbo says. The guys who reverse engineered the format did the maths and found you lose quite a bit of information relevant to post processing, but not simply displaying the image.

The key word there is reverse engineered. If you are serious about archiving you wouldn't dream of using the RAW format of your camera. As it is they currently need to be reverse engineered. Will you be able to open them in 50 years when you can see you D40 only in the museums? I can still print from 50 year old negatives, so pick something with a fighting chance like

TIFF - Which has built-in compression, is supported by every program, is lossless, supports any number of bits, and is infinitely expandable (there's pixel width limits on JPEG did you know that?)

or DNG - Which hasn't been adopted very well yet, but IS an open standard. So there's no reverse engineering, meaning the format is more likely to stand the test of time. At least that's the theory. Uptake has been slow, but it is happening.

 

[MushinVin]

I archive my work in DNG format. Compression is a killer all I can say is do not save your final copies in jpeg format. I have seen how bad it gets and it is not pretty

Love & Bass

Yeah unfortunately I have seen how bad jpeg artifacts can get lol. Coming from a web design background (of sorts), I know how necessary compression is for the web to work, but I don't plan on using it as a means of storing my images.

You can compress RAWs with any loss less compression format (gzip, zip, 7zip, rar, etc) as they compress the data stream only, not the actual image contents. Don't save in JPEG format, but if you're concerned about space, you can use any archiving compression format or method you like.

Ah, ok. I'm really just interested if there is a compression format that is known to be effective for RAW/NEF/DNG/TIFF archiving. Maybe I should hit up some wikis on those compression formats to see if they mention RAW. Though in my googling before posting this thread I didn't find anything very relevant to my original question.

The key word there is reverse engineered. If you are serious about archiving you wouldn't dream of using the RAW format of your camera. As it is they currently need to be reverse engineered. Will you be able to open them in 50 years when you can see you D40 only in the museums? I can still print from 50 year old negatives, so pick something with a fighting chance

Right now I'm keeping the RAWs (and JPEGs) of every picture I take, which takes up quite a large amount of drive space (luckily I have a good 500 GBs left, and I'm planning on buying a 1TB external just for images), though I figure I'll eventually go through and delete all the images I don't want anymore, or convert them to DNG/TIFF and delete the NEFs.

there's pixel width limits on JPEG did you know that?

Do you mean pixel aspect ratio? As far as limits, I didn't know there were any with JPEGs, good to know.

or DNG - Which hasn't been adopted very well yet, but IS an open standard. So there's no reverse engineering, meaning the format is more likely to stand the test of time. At least that's the theory. Uptake has been slow, but it is happening.

I've been hearing more and more about DNG, and it's looking to be a pretty good format to work with. I'll have to do a little research on it, as well as TIFF.

I'm not sure if I stated my question properly in my original post (and I apologize if was misleading), but what I really mean to ask what the best file archive format would be for NEF/RAW/DNG/TIFF? I think rufus5150 knew what I was really trying to ask.

Thanks again everyone, I didn't really expect to get so many replies ^_^.

-Vin

 

[sameer]

I'm new to this forum and new to digital SLRs in general, but I have a simple question. Is there a compression format, as in ZIP/RAR/etc., that is considered to be the best compression format for archiving RAW?

Try rawzor, uses special algorithm and works great on my nikon and fuji files.

www.rawzor.com

 

[MushinVin]

Try rawzor, uses special algorithm and works great on my nikon and fuji files.

www.rawzor.com

Thanks for the link, I'm checking it out now, and it seems to be what I'm looking for.

#youtube_inline_div { position: absolute; display: none; border: 1px solid #999; background: #ececec; padding: 1px; text-align: center; z-index:99}