File management - RAW, tiff, & JPEGS (oh my)

Jon_Are · Jul 28, 2009

I'm in the early phases of reorganizing my photo files. Because I've been only shooting in RAW for less than a year now, I have some issues as to how I'm going to approach file management with multiple formats.

So, say I've shot, I dunno, a bachelor party in RAW (don't get excited, it's just an example :mrgreen

. Then I edit the keepers into tiffs. Then I want to print some, so I convert them to high quality JPEGS. Then I want to post some online, so now I have some low quality JPEGS.

This, to me, has the makings of an organizational nightmare. I could easily end up with four versions of the same image.

For filing purposes, do I keep ALL the images in one folder? Or, should I make a subfolder for each format? Any other ideas?

Will the various versions of each image remain grouped together within the folder? I'm trying to avoid having them scattered about, even within one folder.

If anyone has a simple, effective solution, I'd love to hear it.

Thanks!

Jon

Big Mike · Jul 28, 2009

This, to me, has the makings of an organizational nightmare. I could easily end up with four versions of the same image.

That is pretty standard. Heck, you might even end up with more than that if you start saving files for different print sizes & ratios.

My file structure is that I put the RAW files into a folder with the date and/or name of the shoot. Above this structure are folders for personal, shoots and weddings. I also have separate folders for the year. So for example, if I uploaded today, it would be ...personal/2009/07-28-2009/

My old workflow would involve processing the RAW files (in RawShooter Essentials) and outputting them into a sub folder called 'converted' or 'work'. These would usually be PSD or TIFF but sometimes just JPEG. Then I would edit those files in Photoshop...and if I was going to work on them further, I would save them with all the layers intact...these files were HUGE.
Then I would create other folders for print & web, then save the chosen images in the appropriate folders with the appropriate edits done to them.
I would often go back later (maybe a couple months or longer) and delete the working files because they were so big. I did keep all the RAW files but with my new workflow, I've been deleting a lot more of them, because really, I'm never going to use most of them.

My new workflow is much better IMO. I use Lightroom. LR is a completely different system in that it works in a non-destructive way. You 'import' the files which basically tells LR where the RAW file is, and what it looks like. From then on, all the changes/edit you do in LR are saved only in a small side-car file. None of the changes are made directly to the image file...so it's non-destructive. You can use LR to output any of the files that you will need later. For example, I can take 100 images and create 400x600 web images, then take the same 100 images and output files for 8x10 prints. Sure, you would still end up with multiple copies of the files but only when you actually need them.
You can output TIFF or PSD files to work on in Photoshop, or you can take them directly into PS from within LR...and then back to LR.

I'm still working on what will work best for me...but I really like using LR compared to using PS for everything.

Jon_Are · Jul 28, 2009

I'm a new Lightroom user, still learning.

Say I take 10 RAW images, store them all in a single folder (let's call it 2009/7-28-09/Michigan).

I work on 5 of them in LR, then save them as tiffs.

I convert 3 of these tiffs to high resolution JPEGS, then save.

I save the other 2 tiffs as low rez JPEGS, then save.

Will my 2009/7-28-09/Michigan folder now contain all 20 files (10 RAW + 5 tiff + 3 high rez JPEG + 2 low rez JPEG)?

Or will it show just 10 files, along with some sidecar (.xmp) files?

Jon

msf · Jul 28, 2009

Im trying to fine tune my workflow as well. But I dont have LR.

My workflow is pretty simular to mike's old one.

I have different folders for personal, portraits, wedding, event, etc.
Then inside each of those I have a folder for the shoot, I name it like 2009-07-28 such and such wedding.
then inside that I a bunch of sub-folders, one for unselected raw, one for selected raw, one for tiff from raw, one for psd from tiff, one for tiff from psd *this is my final file and I use this to crop, make web versions, etc.

The hard drive space does seem to add up pretty fast.

Idealy I would like to have 2 external hard drives, and mirror one on the other. My problem has been Ive had some picturse on the laptop hd, some on the external, and some on the desktop hd. im trying to avoid putting them on internal hd's now and just use external's.

I havent been putting them on dvd like I should be, because I dont fully trust dvd's. I think Id rather have them on hd's, and keep the hd's in a safe place after they get full, and ge ta couple new hd's. hd's are cheaper per gb than dvd's now anywase. And take up less space.

Big Mike · Jul 28, 2009

Jon_Are said:
I'm a new Lightroom user, still learning.

Say I take 10 RAW images, store them all in a single folder (let's call it 2009/7-28-09/Michigan).

I work on 5 of them in LR, then save them as tiffs.

I convert 3 of these tiffs to high resolution JPEGS, then save.

I save the other 2 tiffs as low rez JPEGS, then save.

Will my 2009/7-28-09/Michigan folder now contain all 20 files (10 RAW + 5 tiff + 3 high rez JPEG + 2 low rez JPEG)?

Or will it show just 10 files, along with some sidecar (.xmp) files?

Jon

You are still in the linear file flow...you don't need to be.
Here is my suggestion for your example.
You Import all 10 RAW files into Lightroom. You use one of the methods for rating them. There are stars, colors & flags. I delete the duds and flag the great ones. If you use shortcut keys and auto advance, you can do this very quickly. This is done in the Library Module. I might also add Keywords for future reference.
Then I will take the keepers (your 5 images) and put them into a collection, this makes it easier to keep them organized through Lightroom.
I then go into the Develop module and click on the collection I just created.
I will edit them with LR's tools and maybe bring them into Photoshop via the 'Edit in' command...but I bring them back to LR.

Here, you created Tiff files...but you don't need to. Just keep the files in LR.

Select the 3 files and 'output' them....choosing the settings you want for resolution, file type etc. You can even save those settings as a preset, so next time, you can recall those exact settings with a single click.

Select the other 2 files and 'output' them, but with the different settings you want. Again, you can save a preset for the output settings.

When you output the files, you can choose which folder to put them in, or create a new folder for them.

Even when you use Lightroom, it still helps to have a file structure with separate folders....although it's not strictly necessary.

--
The problem I see with your sample workflow is that once you save the images as Tiffs...you have applied the RAW settings and locked them into the file.
If you skip that step and just keep working on the RAW images in LR...then you always have the option to go back or change something, with the full capabilities of a RAW file.

Jon_Are · Jul 28, 2009

Mike, I appreciate you taking the time to answer these questions. In fact, here's a big thank-you: :thumbup:

Then I will take the keepers (your 5 images) and put them into a collection, this makes it easier to keep them organized through Lightroom.

This was my original thought, but I then figured I wouldn't have to bother with collections; I'd just open the (well-organized) file from within LR.

I will edit them with LR's tools and maybe bring them into Photoshop via the 'Edit in' command...but I bring them back to LR.

Are these brought back into LR auto-magically? Or must I manually re-import them?

Finally, here's something I'm still not getting. If I have 5 variations of the same image (hi-rez, lo-rez, etc.), how can I find each of them if they're not actually separate files? In other words, you say to keep going back to the RAW file...but what if I want to re-open that lo-rez JPEG I saved last week? It is actually a separate file, isn't it?

All of which circles back to my original thought: do most people create different folders for RAW, HR JPEG, LR JPEG, etc.?

Or, do they have a hodgepodge of different formats and different images from a single shoot in a single folder?

Jon

Jon_Are · Jul 28, 2009

Idealy I would like to have 2 external hard drives, and mirror one on the other.

This is exactly what I'm going to do, msf. In fact, I just returned from Target where I picked up a 1TB external drive for $99. I already have a 500 GB external. I'm going to make the 500 a dedicated photo drive, and back it up to the 1TB drive.

Once I figure out my new file system, that is.

Jon

Big Mike · Jul 28, 2009

Are these brought back into LR auto-magically? Or must I manually re-import them?

When you use the 'Edit in' from LR...when you take it into Photoshop and then save it, it automatically gets imported back into LR. You have the option of replacing the 'before PS' image or keeping it. I'm not sure what is best here, but you can stack the new one on top of the older one. I think the file that comes back into LR might actually be a TIFF file...so it's probably best to do all your exposure, color edits beforehand.

Finally, here's something I'm still not getting. If I have 5 variations of the same image (hi-rez, lo-rez, etc.), how can I find each of them if they're not actually separate files? In other words, you say to keep going back to the RAW file...but what if I want to re-open that lo-rez JPEG I saved last week? It is actually a separate file, isn't it?

The way I do it...you don't 'find' those other copies of the images in LR. I use explorer.
I do put them into separate folders when I output them though.
For example, lets say I output some files for uploading to the web...I'll do that with the output command and have it create the files in a folder that I create/specify. Then when I go to upload them, I'll find them in that folder. I will use those files for uploading etc, but I'll probably never 'open' them in anything.
If I want to make changes, I'll open LR (going back to the original) and make changes, then output again. This way, no changes are lost.
For example, if you wanted to make a change to the image, after you had uploaded the images...you could edit the small JPEG and upload again...but then your 'master' image doesn't have the change...or maybe you would have to edit the web file and the print file, so that it's the same when you go to print it. But by going back to LR, you change the 'master' image and can then easily replace any of the print/web files.

This is especially good when you get into different crop ratios. For example, if you were going to print an 8x10, you would normally edit the image, the crop it and save it like that. If you did any further editing, you would only be editing the cropped section. But if you later decided that you wanted to print an 8x12 or a 5x7...you would likely have to go back to your working/original file and re-edit it because your edited 8x10 file might not have enough room because you have already cropped it.

With LR, you are always editing the whole image, even if you crop it. And you can always set the crop to one thing, output some files, then change the crop and output another set of files with a different crop ratio.

Sorry, that might have sounded a little convoluted...but the point is that if you are always editing the master file, you shouldn't have to worry about doing things repetitively.

Of course, if you want to create a copy, to try a couple different edits, you can create a virtual copy in LR...still keeping the same original RAW file.

itznfb · Jul 28, 2009

i attached a screen shot of how i have my system setup. i hate software that tries to manage my files so i now use my own organization and Bridge > Photoshop. I tried lightroom for a while but hated it.

i use an import directory to dump the contents of my SD cards. under that directory i store by date and the shoot description. i then have an editing folder where i copy to for PP. i save from that directory 3 copies; 1 full size, 2 resized for sharing posting online and 3 signed with mgroberts photography and copywrite.

i then have more specific directories underneath.

the editing, finalfullsize, finalresized, finalsigned and import are all separate mounted volumes from my NAS. then i file syncronization to a 2TB WD MyBook Mirror edition.

redtippmann · Jul 28, 2009

I just make a folder for the shoot that I just did, the next folder would be labled card1 or card2 for what memory card it was on and then the original pictures on that folder. then the edited ones go into a folder thats in the shoot folder titled FINAL.

IgsEMT · Jul 28, 2009

I'll be a devil's advocate here... what is wrong with JPGs?
You processed the job, convert it to High Quality JPGs, keep those for printing, Your LQ JPGs after you loaded them up - delete from harddrive.
If you really like TIFFs, load those onto CD/DVD and archive it.

Big Mike · Jul 28, 2009

Nothing wrong with JPEGs as a final product. But the problem with JPEG is that it's a compression/lossy format. Every time you save a JPEG file, it losses something. Probably not noticeable the first few times, or maybe even a couple dozen times if you keep it set to high quality...but the more sound workflow is to use a lossless format through your workflow.

msf · Jul 28, 2009

You dont want to save a jpg more than once, because the file looses quality each time it gets saved. And unless you save every possible print size combination as a jpg, you may have to refer to a cd/dvd later, which can get time consuming, compared to just accessing it off of the hard drive. And cd/dvd's have been known to fail.

IgsEMT · Jul 28, 2009

Nothing wrong with JPEGs as a final product. But the problem with JPEG is that it's a compression/lossy format. Every time you save a JPEG file, it losses something. Probably not noticeable the first few times, or maybe even a couple dozen times if you keep it set to high quality...but the more sound workflow is to use a lossless format through your workflow.

I am talking about JPG as a final product. I'm sure it is a matter of preference and style but after you shot, edited, got client's approval, printed and delivered the job and it was approved again is there a need to keep anything but final jpgs?

B/c I'm refering to this

So, say I've shot, I dunno, a bachelor party in RAW (don't get excited, it's just an example :mrgreen. Then I edit the keepers into tiffs. Then I want to print some, so I convert them to high quality JPEGS. Then I want to post some online, so now I have some low quality JPEGS.

are you gaining anything in quality by converting RAWs to TIFFs then to HighQ JPGs and then printing them vs RAWs to highQ JPGs and print?

Big Mike · Jul 29, 2009

I am talking about JPG as a final product.

No problems 99% of the time. I send JPEGs to the lab for printing and if a client gets a file, it's a JPEG. The exception might be a job for a designer or something where they want to further process the file, in which case I'd give them what they contracted for...RAW, TIFF etc.

I'm sure it is a matter of preference and style but after you shot, edited, got client's approval, printed and delivered the job and it was approved again is there a need to keep anything but final jpgs?

That depends. Storage memory is pretty cheap, and having something on hand is better than deleting it just to save some hard drive space. Your final JPEG might be OK for your final usage right now, but what if sometime in the future, you want to do something else with it. It would likely be beneficial to have all the data available to you, and not just the JPEG file. Also, many photographers say that their editing/processing style changes over time, so they like to go back to some of their old files are reprocess them for different/better results.