Seeking Archival Workflow Recommendations/Ideas

[Derrel]

I think the simple answer is to convert your processed raw files to.jpg format which is easily readable a whole host of computer software.

That's what I proposed at the start. Is this what you do? I'm inclined to save a 16-bit TIFFs, although it's unclear if that format actually provides any advantages for a "final" image.

Well I understand your desire to read XMP files, they are not really a picture, and pictures are what you are in the business of saving.

At some level, XMP files are different from other formats. At another level, they are the same—bits on the computer that can be converted to images. You do need the associated RAW file, but RAW+XMP+the right software will produce images just as well as other image formats. While I disagree with your characterization, the point is moot since it is not an archival format (at this time).

The 16-bit .TIF is hard to deal with; it is LARGE, significantly larger than an 8-bit .TIF... i have saved a number of spotted, corrected "master files" as 16-bit .TIF, mostly in the 1995-2000 era..today? No, no longer. The 16-bit .TIF is just soooooo large a file!

In 95% of cases, I prefer to have a finished, edited, large, low-compression JPEG as an indicator of how a photo ought to look. I have always preferred looking at a finished picture, as opposed to looking at a negative.

In the future, editing raw data will likely be faster and easier than it is now, and one's editing skills tend to grow over time. TO me, archiving the raws and finished JPEGs is good enough.

 

[smoke665]

The 16-bit .TIF is hard to deal with; it is LARGE,

While TIFF is the old standard I switched over to PSD about a year ago as it is the default file format and the only format, besides the Large Document Format (PSB), that supports all Photoshop features like layer styles, layer folders, snapshots, custom channels, and editing histories. When saving a PSD, you can set a preference to maximize file compatibility. This saves a composite version of a layered image in the file so it can be read by other applications, it also maintains the appearance of the document, just in case future versions of Photoshop change the behavior of some features. Including the composite also makes the image much faster to load and use in applications other than Photoshop, and may sometimes be required to make the image readable in other applications.

 

[JBPhotog]

The 16-bit .TIF is hard to deal with; it is LARGE, significantly larger than an 8-bit .TIF... i have saved a number of spotted, corrected "master files" as 16-bit .TIF, mostly in the 1995-2000 era..today? No, no longer. The 16-bit .TIF is just soooooo large a file!

In 95% of cases, I prefer to have a finished, edited, large, low-compression JPEG as an indicator of how a photo ought to look. I have always preferred looking at a finished picture, as opposed to looking at a negative.

In the future, editing raw data will likely be faster and easier than it is now, and one's editing skills tend to grow over time. TO me, archiving the raws and finished JPEGs is good enough.

What ever works I guess, the main reason I wouldn't save just a final edit JPEG would be for resizing and printing. Ultimately sharpening should be done as per the final size required and as a last step in the edit process, so baking in a sharpening algorithm for a JPEG and resizing later introduces artifacts. Embedding a colour profile is part of the workflow dependant on output, JPEG's by default output in sRGB which may not be preferable if one is looking for the widest colour gamut.

Yeah, I know, PSD's and Tiff's are way bigger but we have seen storage prices drop and bigger drives available in the market place so it can be overcome with relative ease. As for saving absolutely every file, well its best one is judiciously efficient when making that decision so those 1's and 0's go farther.

 

[Derrel]

I agree with most of your points JB Photog, but as I said, " TO me, archiving the raws and finished JPEGs is good enough." meaning.. IF a file is needed in the future, with the raw file and the finished JPEG as a guideline, it is a trivial matter to re-edit: as I wrote, " In the future, editing raw data will likely be faster and easier than it is now, and one's editing skills tend to grow over time. TO me, archiving the raws and finished JPEGs is good enough."

One never know the future..but since I made my first raw Nikon files in 2001, editing raw dat ahas become _much_ easier, and faster, and entirely new directions have become common. Layer editing, and frequency separation, automatic mask creation, and automated digital fill light, clarity slider, content-aware fill in the clone tool, de-haze...all are now common tools that can do in minutes what USED to take a long, long time, and which required a lot of skill.

Instead of saving a 128-megabyte, 16 bit .TIF of one file as my guide/reference/catalog entry/stock library representative for a particular image, I have decided that 1) a large, low compression JPEG from 2 to 8-10 megabytes is a roughly 12- to 15-times more economical way to store the image, and the image is smallish, for easy FTP or even e-mail transmission 2) saving the RAW file is about 6-12 times more economical in terms of disk space than is a 128-megabyte, 16 bit .TIF and 3) Sharpening and re-sizing are as you wrote, best done according to end use of the file.

I do not save only the edited .JPG...I still have all the .NEF and .RAF and .CR2 files I have made since 2001..now, with near 20 years' worth of raw data, and with cameras having shot up from 2.7 MP, to 6 MP, to 10 MP, to 12 MP, 16 MP, 24 MP. 36 MP,and now 42 to 50 MP in some cases, the adage that "storage is so cheap" is no longer really 100% true.

As @smoke665 mentioned, how much raw data one saves MIGHT be related to how much other data one saves..a huge.PSD or .TIF file occupies the space on average, of around 10 or more RAW files, hence my decision in roughly 2003 to move away from the .PSD and .TIF formats, and to save roughly 110 megabytes per-image on storage.

 

[smoke665]

how much raw data one saves MIGHT be related to how much other data one saves..a huge.PSD or .TIF file occupies the space on average, of around 10 or more RAW files. hence my decision in roughly 2003 to move away from the .PSD and .TIF formats, and to save roughly 110 megabytes per-image on storage.

I've found my PSD compressed files significatntly smaller then a TIFF. By doing some housekeeping, deleating unecessary layers where possible, using adjustment layers rather than duplicating a layer and applying image adjustments, applying layer masks, and cropping oversize layers, I can reduce them even further. Of course it depends on the amount of editing, but 2-3 times a raw file size is more in line.

I know Ps is always the go to for heavy editing, but with each new upgrade to Lr, combined with diligent "in camera" work, I'm finding less reason to open in Ps. Lr's addition of Profiles and the ability to change the opacity of them was a game changer for me.

 

[JBPhotog]

FWIW, Once I have edited a photo in PS I am loathe to want to do it all over again at a later date. I may tweak a layer mask or apply a new adjustment down the road but to start all over from scratch, no thanks.

 

[freixas]

Thanks, everyone. The discussion has got a lot more interesting.

A couple of points:

• I actually own ON1. I can't convert my XMPs because they only support import of a Lightroom catalog.
  
• It's worth reading their notes on how they import Adobe adjustments. It's not straight-forward nor complete.

I currently have Adobe Camera RAW for CS6, ON1 2019, RawTherapee, and trial copies of ACDSee, Affinity, and Capture One. I selected some images that I had processed with ACR long ago and then tried to reproduce them using the other tools just by referencing the final image. I found the exercise very difficult and never succeeded. It might have had an easier time if I had picked an image with just some basic adjustments—or if I had inspected the RAW adjustments to see what I had done.

RAW file editing is interesting. I don't know how other packages do it, but with my CS6 ACR, all the adjustments are very small. I was surprised to see that masks aren't actually created as an 8-bit layer (which is what I assumed and what I believe Photoshop does). Instead a mask is stored as vector information—the brush parameters plus the path. Since the XMP file is a text file, you can just open it up to see what gets recorded.

Sadly, Capture One's masks seem to be bit masks. This may allow better precision at the expense of more space.

RAW file processing has indeed come a long ways. The thing I dislike is that the long ways that it's come is completely proprietary. Switch software and watch all your work disappear. When I own the software, it's less of an issue. I can keep CS6 for as long as it will run and retain access to any RAW files I've processed. Same with ON1 or Capture One, but not with Adobe CC.