Archiving in Raw vs. DNG

[icassell]

OK ...

I've been working in Bridge/PS for some time and finally broke down and bought LR2 a few months back. Now I've finally decided to start using it and picked up a copy of Lightroom 2 by Nathaniel Coalson as an introduction. He makes a very cogent argument for importing everything into LR2 as Adobe DNG files and then dumping the RAW file.

I'm interested in knowing if there are good arguments AGAINST this practice.

Oh and let me apologize in advance if there are other similar threads. I did a search and couldn't find one specifically on this question.

 

[Derrel]

Well, the only person who I know has written AGAINST the practice of importing raw files into DNG is noted outdoor photography expert, digital technology writer, and Nikon expert Thom Hogan. He maintains that currently, there is still some room for improvement in raw image processing that DNG conversion removes. He also believes that it is not a good idea to have DNG become the defacto recording format for cameras, and that the manufacturers' raw formats can wring a bit more out of their sensors than DNG can.

I know that DNG conversions are roughly 20 percent smaller than many raw files in their native format, and with things like Fuji's huge 24.5 megabyte Extended Dynamic Range .RAF files, the conversion to DNG really cuts the size down....but it also removes the potential to use the BEST raw conversion software, which is Fuji's very own HyperUtilities suite,which still produces a better raw conversion than Lightroom,Aperture, ACR, SilkyPix, or RawTherapee.

Hogan believes that the DNG 'standard' is still not quite as advanced as it could be. Given how cheap storage is, I see no need to erase raw files,even if they have been cnverted to DNG files for smaller archives.

 

[Garbz]

I see no reason to store two files if DNG has support for native embedding of the original data. Sure you end up with a 1.8x file size but you essentially get the best of both worlds. An open standard likely to be able to be read (see thread about guy who bought D300s and can't actually open his RAW file), and you get to keep the original data if in the future there are improvements to be gained. Falls in nicely with the "given how cheap storage is" argument.

The only problem really with RAW that DNG is trying to address is lack of openness. NASA couldn't read it's own luna landing tapes. What choice do we have amongst a world of corporations who build obsolescence natively into every product. No one could with a straight face tell you that in 60 years, a propriety format which had to be reverse engineered at the time by everyone except the manufacturer has a higher chance of being read than a natively open documented format free to implement by anyone with time to write some code.

I see where Thom Hogan is coming from, but my counter argument is that this data is only relevant at the time of processing. As someone who processes an image once and then never touches it anymore for anything other than viewing or printing I actually don't see the point in keeping RAW files at all, when JPEG can display all the visible data in an sRGB file anyway.

So yeah the images may come out better if I reprocess them in 10 years, but they'll still look crap compared to my produces Nikon D5, and then really do I have the time to rework 250000 photos?

 

[manaheim]

Wow, nice comments both Derel and Garbz.

I am someone who winds up going back to his images quite a bit. Images I really like I wind up re-tweaking repeatedly over years, so I don't want to lose an fidelity at all so I've always been a bit leery of DNG.

I've been concerned about not being able to process those NEFs at some point, but I keep copies of the valid software that works (including very old operating systems going back to 1994 and even a few old PCs). One way or another, I'll be able to convert them to whatever becomes the primary solution however many years from now... and I figure I'll likely see that change coming well before it happens anyway, so I'll do the conversion when I do.

This is just a summary of my approach to it, however... it means nothing more than this is what I have come up with that I am personally comfortable with.

 

[icassell]

Hmmm .... clear as mud .... :lmao:

Thanks for comments, everyone.

I have to make a managerial decision here ... I don't think I'm going to convert my 50K RAW images to DNG, but going forward I have to make a decision ... there are good arguments on both sides of the fence ...

 

[Garbz]

hehehe good luck. You got me thinking manaheim and that's always trouble.

My first computer is only 15 years old, may still run if I can get the BIOS to recognise a replacement hard disk. I wonder if Windows 3.1 will run on modern hardware, certainly most of my old games don't run under Windows 7 (at all thanks to a drop of 8/16bit code support).

Kinda makes you want to grab a computer and store it in some sensitive archives somewhere, just in case.

 

[Overread]

Garbz you might find this website helps feed your addiction:
GOG.com

though that said old games still can have problems - and display worse on newer computers than old - as software and hardware changes.

 

[manaheim]

I ran Windows 3.1 on a Pentium Pro once. That's the last time I tried though.

And maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaan did that sucker FLY.

BTW, even when things don't work there are often emulators for older OSes, mainly because people love to play their games.

 

[Garbz]

True. Lets hope emulators will run old photoshop. I see that as the only way to keep these archived files in the long run.

Adobe CameraRAW is now only a few years old and there's a HUGE list of files each proprietary and different from the next already: Adobe - Adobe Photoshop CS4: Digital camera raw file support If this keeps up then in 20 years the camera support component of the software alone will be many times the size of the actual program :S, something will give.

 

[manaheim]

True. Lets hope emulators will run old photoshop. I see that as the only way to keep these archived files in the long run.

Adobe CameraRAW is now only a few years old and there's a HUGE list of files each proprietary and different from the next already: Adobe - Adobe Photoshop CS4: Digital camera raw file support If this keeps up then in 20 years the camera support component of the software alone will be many times the size of the actual program :S, something will give.

Yeah, true... though I wonder... I mean, at what point would you wind up being in a TRUE "oh crap I can't use this anymore" kind of situation?

In truth, you could probably download ACR and the cam definitions every couple years and just keep that on a hard drive somewhere... then you can always just go back and convert your old files if you hit that "Oh $hit" point where suddenly Adobe drops support for your older cam.

 

[Moglex]

True. Lets hope emulators will run old photoshop. I see that as the only way to keep these archived files in the long run.

Adobe CameraRAW is now only a few years old and there's a HUGE list of files each proprietary and different from the next already: Adobe - Adobe Photoshop CS4: Digital camera raw file support If this keeps up then in 20 years the camera support component of the software alone will be many times the size of the actual program :S, something will give.

Yeah, true... though I wonder... I mean, at what point would you wind up being in a TRUE "oh crap I can't use this anymore" kind of situation?

In truth, you could probably download ACR and the cam definitions every couple years and just keep that on a hard drive somewhere... then you can always just go back and convert your old files if you hit that "Oh $hit" point where suddenly Adobe drops support for your older cam.

I would have thought that the number of times any normal person would really need to return to a RAW image and reprocess it would be pretty close to zero.

Saving in a TIFF with lossless compression would get you all you need in virtually every case (for most people, every case).

After all, with film you never got the chance to reprocess (although there were some rather gruesome 'rescue' processes you could apply to some negs), even one day after first processing.

 

[manaheim]

I would have thought that the number of times any normal person would really need to return to a RAW image and reprocess it would be pretty close to zero.

Saving in a TIFF with lossless compression would get you all you need in virtually every case (for most people, every case).

After all, with film you never got the chance to reprocess (although there were some rather gruesome 'rescue' processes you could apply to some negs), even one day after first processing.

Define "normal person".

As my skills have grown and my eye has adjusted over the years, I've gone back to many images and either drastically improved the final product, or found images of note that I simply didn't see before.

Plus it's just a matter of the whole "negatives" concept. Imagine if you couldn't pull your negative again (were you a film shooter) because some technology changed.

Unless I'm mistaken... the tiff doesn't have the broad exposure and color range that a RAW does...? Right? (Honestly, I've worked relatively little with tiff since I usually save in JPG for short term needs or PSD for long term ones)

 

[Moglex]

I would have thought that the number of times any normal person would really need to return to a RAW image and reprocess it would be pretty close to zero.

Saving in a TIFF with lossless compression would get you all you need in virtually every case (for most people, every case).

After all, with film you never got the chance to reprocess (although there were some rather gruesome 'rescue' processes you could apply to some negs), even one day after first processing.

Define "normal person".

Fair point.

As my skills have grown and my eye has adjusted over the years, I've gone back to many images and either drastically improved the final product, or found images of note that I simply didn't see before.

I can't argue with your experience but have there really been that many times that you've needed to go back to what is effectively the latent image and change the processing? If you had a correctly 'developed' image that hadn't been modified in any way would that not have been as good?

Plus it's just a matter of the whole "negatives" concept. Imagine if you couldn't pull your negative again (were you a film shooter) because some technology changed.

But getting rid of the RAW (and keeping a processed but unmodified losslessly compressed image) is not getting rid of the negative, it's more akin to getting rid of the latent image but keeping the negative.

Unless I'm mistaken... the tiff doesn't have the broad exposure and color range that a RAW does...? Right? (Honestly, I've worked relatively little with tiff since I usually save in JPG for short term needs or PSD for long term ones)

It's true that you would lose something moving from RAW to any lossless 'developed' format.

What I'm saying is that what you give up by moving to a non-proprietary format is only a very small amount of flexibility ( for TIFF, microscopic for DNG) but what you gain is the guaranteed ability to be able to get at your data.

It may well be that in 50 years time it's impossible to get at music stored in apple or sony proprietary formats (or indeed anything crippled with DRM) but you can pretty much guarantee that you'll be able to access MP3 or FLAC.

I think the options are:

1) Store in proprietary and risk not being able to process the image at all at some point in the future.

2) Store in non-proprietary and risk losing a tiny amount of re-processing flexibility

3) Store in both and accept the extra storage cost.

 

[itznfb]

I've actually started storing files as jpg. I'm the type that doesn't like to delete anything.... even completely missed and out of focus shots. (I'm trying to break that habit). After I've processed the "keepers" I run ViewNX and convert all the raw files to jpg. If I need to go back and re-process something. Oh well, I'll have to make due with the jpg.

 

[Garbz]

The other major assumption is that Adobe is still around in 50 years. They may get eaten up by Google, when one day Google consumes every technical thing on earth.