Seeking Archival Workflow Recommendations/Ideas

[freixas]

I suspect this is one of those questions that really has no good answer. What would you recommend as a good method for archiving your images? For 5 years? 10? 100? I am not talking about backup—you can backup a file in a proprietary format that may not be accessible in 10 years.

The best I've come up with is to save the RAW files and save a TIFF or lossless JPG file showing what the final image should look like. For those things I still modify in Photoshop, I would also save the PSD file, which has become something of a standard format. Unprocessed images, of course, only get saved as RAW files.

If I wanted to re-work an image in the future, what I did might depend on what software I had. If I had a PSD and a program capable of opening PSD files, I could start there. If I had a tool that could read the RAW file adjustments (i.e. XMP file or whatever), I would use that. Otherwise,
I would have to start from the original RAW (and whatever tool I would have for reading/modifying RAW files), using the final TIFF or JPG as a guide for the things I wanted to retain. Any better ideas?

This is not what I have been doing and I'm thinking maybe relying on my XMP files wasn't a great choice. It does save a lot of disk space.

 

[Tim Tucker 2]

Things change.

The way we take images, the way we store them, the way we edit them and the way we view them have all changed since I started photography. It is a simple truth.

But also the possibilities have changed about what we can do, what is popular, what floats our boat, what represents a good image...

The truth is that I still use a 1958 film camera with a 100+ year old lens yet can still access all the images. I can still access all my old film images from when I started photography and do it in the modern digital format.

So what are you asking? Is it, can I remain static. edit my images for this time and my taste at this time for ever? Because history has shown that even photographs from the dawn of photography are still accessible and still being published today. Old formats have not become redundant, (apart from Beta-max).

 

[Derrel]

https://forums.adobe.com/thread/2598410

A web search on "future-proofing an archive" lead me to the above.

 

[smoke665]

Old formats have not become redundant,

Old formats may not have become redundant, but the ability to read has. Have any data stored on reel to reel, how about 5.25 floppies? If you did it's unreadable because of the obsolescence/incompatibility of today's equipment. You can still view low tech paper prints, as long as our eyes continue to work, but in the new age, of 1's & 0's without the equipment to decipher it, you're pretty much SOL. Who's to say there will even be equipment around 10 years from now that can read a CD/DVD, or even a hard drive as we know it today? Technology is rapidly changing, and old tech goes by the wayside just as fast.

I for one have come to believe that it's a fallacy to seek to store anything digital for more then a couple years at best. IMO it makes more sense to utilize the storage options available now either HD or SSD, and to make a practice of upgrading and saving it to the latest technology every few years. At least that way, you're somewhat assured that you'll always be able to access it.

 

[Soocom1]

OK. I deal with archiving aspects daily.
First and foremost, make sure that whatever system you use, that you can access it in the years to come.
Ergo: If you have a standalone hard drive, it will need to be updated and the files transferred from one HD to another in years time.
remember the first tower HD's worked with W95. Still running that format? Jpegs saved in 1998 are still accessible today, but you should create a system that allows for file transfer every 5-10 years or so. That's the simple fact.

Secondly: I cannot stress this enough. Security! Double or triple copies of the original RAW files and worked Jpegs can be a life saver, but they also need to be routinely transferred in the same time frame. The system you use should be encrypted and double copies kept in different locations. This is so if there is a flood, fire, meteor strike alien invasion, or whatever, you can still access the other copie(s).

Remember that older systems started with 8 Bit and moved to FAT 16, FAT 32 and is now FAT 64 NTSB, and other type formats. Keep them up to date and store in the cloud somewhere with multiple secure copies. the ability to access them. Ergo: passwords etc.

Putting up reminders on calendars especially on smartphones is a really good idea. Set up a reminder for 5-10 years future and the calendar should carry on from one phone to the next or calendar system like Outlook.

Hard copies are another aspect but that's a whole different system to explore. But remember, if you kick the pervervial bucket, your family should have some kind of access to the files.

 

[Tim Tucker 2]

Old formats have not become redundant,

Old formats may not have become redundant, but the ability to read has. Have any data stored on reel to reel, how about 5.25 floppies? If you did it's unreadable because of the obsolescence/incompatibility of today's equipment. You can still view low tech paper prints, as long as our eyes continue to work, but in the new age, of 1's & 0's without the equipment to decipher it, you're pretty much SOL. Who's to say there will even be equipment around 10 years from now that can read a CD/DVD, or even a hard drive as we know it today? Technology is rapidly changing, and old tech goes by the wayside just as fast.

I for one have come to believe that it's a fallacy to seek to store anything digital for more then a couple years at best. IMO it makes more sense to utilize the storage options available now either HD or SSD, and to make a practice of upgrading and saving it to the latest technology every few years. At least that way, you're somewhat assured that you'll always be able to access it.

Old *image* formats have not become redundant, though as you point out old digital storage media has. You are quite correct to point out that you have to back-up storage media from time to time, and maybe even re-save in a different format. Though I don't believe that 5 1/4" floppies were current at the dawn of digital photography and that modern cloud storage is quite adaptable, it's funny how it's the march of digital technology that brings redundancy where it didn't exist before...

 

[freixas]

Sorry, I didn't want to get into a discussion of methods of preserving digital data. Let's just assume that the data has somehow been preserved—my question is about accessing it.

My CS6 no longer accepts RAW files from newer cameras and I was thinking of buying a new camera. Lately, for many images I only have the RAW file and the XMP sidecar. It occurred to me that this might not be the wisest approach (it saves a ton of space, though). If I were to enter the Adobe subscription world and continue doing this, I would have to keep paying Adobe just to maintain access to my images.

Other packages are somewhat better in that one can buy a permanent license (like I have for CS6). Let's say I go with Capture One and continue with RAW + adjustments. If PhaseOne goes belly up, I can still run their software (and access my images) until the software no longer runs on my system. Then it all turns to dust. Then again, maybe I can rely on future virtual machine support for my current OS and hardware.

The RAW + (optional) PSD + TIFF or JPG approach might be a good fallback. It eats up more disk space, although I only have to do this for the images I process. I would still keep the adjustments file—the rest would be insurance.

With the exception of switching to ON1, which now has the capability of converting an Adobe XMP (with adjustments) into an ON1 file, switching your RAW tool means losing access to the adjustments (or, rather, not having them in your new tool). It would be nice if there were an archival format for RAW adjustments, but each package tends to have a different approach—I don't see any developers having any desire to create or use a standard format.

 

[smoke665]

Old *image* formats have not become redundant

Whether by obsolescence or necessity all things digital will eventually become historical novelties. I read somwhere that at our current rate of advancement computer capabilities are doubling every 12 to 18 months. Storage in the cloud is a way of life already for the cell phone selfie millienial, and 3d holographic images are here already. Once that takes hold on the consumer level, I believe its only a matter of time before all image formats as we currently know will become obsolete. I mean why would you want a two dimensional flat image of your newborn when you could have a three dimensional, holograph with sound and movement.

That's why I believe a more short term option with updates and conversion to the newer technology as it becomes available makes more sense then trying to store a digital image for eternity. Otherwise you'd have a better chance of viewing them a 100 years from now by printing them on archrival paper and storing them in a cool dry place.

 

[freixas]

That's why I believe a more short term option with updates and conversion to the newer technology as it becomes available makes more sense then trying to store a digital image for eternity.

The main issue I've brought up in this thread is the proprietary RAW adjustments files. How would you "convert" these to newer technology?

Until I started to consider alternatives to Adobe, I hadn't thought much about exporting my processed images to standard formats. In fact, I assumed that other RAW photo programs would be able to read my XMP files. A lot of my images now exist only as RAW+XMP. I don't think I'm alone in this.

I was curious if there were people who had given this problem some thought and wondered if anyone had come up with anything better than what I had proposed when I started this thread.

I'm sure Adobe loves this, but the pain of losing access to all RAW adjustments creates a barrier to switching packages. Maybe this barrier will be lowered in the future if more vendors follow ON1's approach of porting Adobe's adjustments (of course, it is much less likely that anyone will port ON1's adjustments, so choosing to use ON1 brings back the issue that RAW adjustments are proprietary, subject to obsolescence and not a good archival choice).

 

[Derrel]

I think the simple answer is to convert your processed raw files to.jpg format which is easily readable a whole host of computer software.

Well I understand your desire to read XMP files, they are not really a picture, and pictures are what you are in the business of saving.

Yes it is a lotta' work migrating to new storage media every few years, or every decade , But that is what preserving archives is all about. The other option is maintaining physical machines that can read the media. If all of your data is stored on reel to reel or eight track or cassette tape, it makes sense to maintain Machines that can read that media.

 

[smoke665]

The main issue I've brought up in this thread is the proprietary RAW adjustments files. How would you "convert" these to newer technology?

Actually the question is "What will the technology of the future be?". Until you know, anything you do is best guess. Look at the current release of Ps and the 3D formats it now supports. If you save a file as a JPEG, consider there's already a JPS format out. I think the new Hologram files are fbx. The formats are expanding rapidly with no real standard yet.

Without fail all the other guys in some way or another always compare themselves to Adobe. Love them or hate them Photoshop has become a proprietary eponym when referring to a photo editing software, much like Kleenex is to tissue. My personal feeling is that any new technology of the future is going to include some method of converting Lr or Ps files. To do otherwise would be severly hampering their market right out of the gate.

That doesn't mean that you can sit on your hands waiting, skipping updates and conversions over time because at some point your files will become obsolete. I know the the subscription model really flys in the face of many, but personally I consider it a convient way to stay abreast of updates as they occur. Even most of my purchased business software has been on a subscription plan for several years.

As I said earlier, for now my storage plans are more on the order of less then 5 years, with rotating upgrades, and a sound backup plan that includes duplication.

 

[freixas]

I know the the subscription model really flys in the face of many, but personally I consider it a convient way to stay abreast of updates as they occur.

I happen to be a composer as well as a photographer. I have a "subscription" to Sibelius. If I stop paying them money, I stop receiving updates. I still own the software and can continue to use it. Most of Adobe's competitors also offer a subscription plan, but you can opt to purchase the product.

With Adobe, this is not an option—the moment you stop paying them, you lose access to anything in their proprietary formats. I'd love to stay with Adobe, but I don't want to support their extortionate business model.

I do hope that other developers do to the XMP file what they's already done to the PSD file.

 

[JBPhotog]

Unfortunately anyone shooting digital must also stay aprised of the current media technologies short of losing the ability to access them. Just think of all the formats that have gone extinct in 20 years, many were touted as "archival" at the time.

Everyone has their own methods of workflow, for me I keep the Raw files, processed PSD's in 16 bit and final JPEG's. They get dumped on hard drive for storage. FTR, I use CS6 so I am not on a subscription. I also use Nikon Capture NX-D for file conversion of my NEF's

 

[freixas]

I think the simple answer is to convert your processed raw files to.jpg format which is easily readable a whole host of computer software.

That's what I proposed at the start. Is this what you do? I'm inclined to save a 16-bit TIFFs, although it's unclear if that format actually provides any advantages for a "final" image.

Well I understand your desire to read XMP files, they are not really a picture, and pictures are what you are in the business of saving.

At some level, XMP files are different from other formats. At another level, they are the same—bits on the computer that can be converted to images. You do need the associated RAW file, but RAW+XMP+the right software will produce images just as well as other image formats. While I disagree with your characterization, the point is moot since it is not an archival format (at this time).

 

[smoke665]

At some level, XMP files are different from other formats. At another level, they are the same—bits on the computer that can be converted to images. You do need the associated RAW file, but RAW+XMP+the right software will produce images just as well as other image formats. While

My understanding is that XMP was created by Adobe, but has now become an industry standard. If you save your XMP as a sidecar , not embedded , some alternative software like ON1 can already read it. Some can only read bits of it. In any case, I think that having that XMP as a sidecar would be a good first step for conversion in the future. Just be aware that constantly writing to an XMP sidecar will slow Or down considerably.