Editing massive RAW files

[CdTSnap]

Hi team

I'm trying to solve a little issue with the huge RAW files from my A7R2's

Basically I just want to speed up my workflow for next wedding season, my computer is a beast, its not that at all im just trying to get it even faster. I really want to get it as fast as possible.

I have over 40 weddings next summer and I want to edit them fast without lag god dam Sony's lol amazing files but way over kill for weddings.

One thought I had was importing all the RAWs to Lightroom then exporting them as DNG's with a max size of 6000px on the long edge then re-import with 1:1 previews and smart previews etc

What you think? Any other ideas?

 

[Derrel]

On my older fuji S2 Pro and S5 Pro d-slr cameras, their .RAF file format was very bloated, due to the way those cameras Up-rezzed 6MP captures to 12MP captures"converting those raw files to .DNG formatted raw files saved a LOT of file space!!! It was a MAJOR difference...

As I recall, some of the .RAF files from the S5 would be 24 megabytes in Fuji's.RAF format, but would slim down to 10 to 12 megabytes as .DNG files.

I dunno...I would be tempted to take say, a collection of 100-150 raw files, and TEST-convert them with the software apps and hardware you want to use; if there is a substantial savings of processing time or space, it might be a good and a worthwhile procedure.

Yeah....I think 6,000 pixels on the long axis ought to be amply big for most uses.

 

[bratkinson]

For me, my biggest editing time saver is simultaneously producing both RAW and JPG files in the camera (I don't do videos, ever). As an aside, my camera has 2 card slots, so the RAWs go to the faster card and JPGs to the slower. I also have an instant backup in case of a card failure.

How do the JPGs save me time? By allowing me to delete the 'losers' before I waste 1 nanosecond more bringing them into lightroom for editing. Using any photo viewer, I quickly examine each shot and if need be, zoom in on it to ensure that most if not all subjects in the image have their eyes open, no unflattering facial expressions, no obese people in profile, over/under exposed, missed focus, etc. In another window on the screen, I have the files shown in list format, and simply delete the losers in that window...usually 3-4 at a time holding down the <cntl> key to select multiple items. I could delete the RAWs directly, but instead delete JPGs. Once done, I simply 'eyeball match' file names between the RAW and JPG folders and delete the loser RAWs before importing the winners into Lightroom. I do it as a two-step process in keeping with my 'always keep a recent backup' (max 1 hr) retired computer consultant mindset. It paid off when I accidentally wiped out two days worth of editing on a project trying to make space on my SSD this past March. I lost about 5 minutes work.

For me, shooting at church events (no weddings – too much stress) I take many photographs (AKA spray & pray). For a multi-evening annual event, I shoot 400-500 frames per evening...oftentimes more. Many of those are group shots, such as a stage play or choir groups, so I have lots of eyes and faces to look at in each frame before I declare it a 'keeper'. Needless to say, much of my editing time is yea/nay decisions rather than Lightroom time.

You indicated you have a beast of a computer. I certainly hope so, because NOTHING slows down editing more than a mediocre to fast computer. I build my own, and am currently running an 8 processor overclocked, 4.6ghz liquid cooled computer with 16gb RAM...I'm toying with going to 24 or 32gb. The big reason is Lightroom.

#1 is that Lightroom can and does use all 8 processors, especially during the import and export operations...although it doesn't push them to maximum clock speeds. So, faster is always better...more processors, better yet.

#2 is having all Lightroom import from, export to, catalog, intermediate, and work files on SSD. This is especially true if you have less than 8-12gb RAM as it has to continuously swap in and out RAM to SSD when constricted in RAM. I suspect they bypass Windows swapfile to improve performance. Windows and all its work files and swap files are on SSD as well.

#3 is that Lightroom never modifies the original file. What it does is that it writes every action you took on every image out to one of its 'save' files. Every time you click, it appears to sequentially read through the list of everything you did on that image to determine where the end of that list is, then adds the record of that click. It must also force the computer to perform the disk write action rather than leave it in the buffer until full, as well. The more clicks, the more reads, the longer it takes.

Why doesn't it simply store everything in RAM? So that in case of a power failure or computer crash, you lose at most 1 click of activity. I proved that this is the process about 2 years ago while editing a large quantity of very degraded slides for a friend. Even using a 100 PSI blast of air from a compressor on the slides while they were still in Kodak Carousel trays failed to remove much of the 'baked on' dust (and some mold) of 40-50 years storage. Using a bulb-style blower with a brush after the air blast still left significant amounts of dust. I had no choice but to remove the dust one click at a time using 'heal' vs using any kind of 'dust and scratches' filters as using a 3-5 pixel radius seriously degraded the images. After maybe 30 clicks, I noticed some slowdown between clicks. After 100 on an image, it was reaching the annoying level of slow. The total number of images being processed also affects overall speed, not just importing and exporting. I did resort to 1 pixel dust and scratches clean up and major cloning in Photoshop Elements afterward.

I learned to do my processing in 100-150 image 'chunks' to keep faster speeds and to provide a good 'time for a break' after several hours at the computer. The 100-150 'chunk' size is also due to Lightrooms' keeping the all images worked on in RAM. With 16gb, the 100ish size is about where my 16gb RAM gets filled and it starts having to swap in and out. It also provides a convenient 'backup point' where I copy what's done to both a hard drive and to a thumb drive that's in my pocket in case the house burns down before I finish the project.

Speaking of RAM, it's always advisable to do a reboot of your computer before starting Lightroom. That way, minor Windows 'left over' pieces (fragments) of RAM from other programs get eliminated thus ensuring maximum RAM for Lightroom.

I'm still a Lightroom novice, not having 'graduated' to presets, etc. But I do a number of 'standard' adjustments 'up front' and then SYNC everything to the first slide. That includes minimal clarity and saturation adjustments as well as lens correction and verticals. As I don't have hyper-critical bride and groom customers, I don't have to produce 'perfection' in every image. Except for the pictures I think are really good, the old 'close enough is good enough' for me.

 

[BrentC]

One of the fastest way to speed things up is, if you don't have one, get an SSD. The bottleneck is usually the harddrive.

 

[Vtec44]

I shoot with a D800 and limit myself to about 25-30 weddings a year. That's about all I can handle.

LR temp files are set to be on a SSD drive, import the RAW files, create smart preview, then move the RAW files to a subfolder so LR uses only smart preview. It loads and edit much faster. Just before I export, I move the RAW files back, then export them to about 4000px on the long edge.

 

[ronlane]

You might consider using Photomechanic to cull the images. This is something that sports photographers have been using for years. It is so much faster than LR and you can mark the ones that you want to edit in it and only import those into LR (while you are doing something else). I used it at the end of football season and have seen some youtube video's about how some wedding photogs are starting to use it as well.

 

[CdTSnap]

I discovered it's a 4K issue. Editing on a Full HD monitor is fine. Much faster


Sent from my iPhone using ThePhotoForum.com mobile app

 

[Light Guru]

I discovered it's a 4K issue. Editing on a Full HD monitor is fine. Much faster

Sounds like you need a better graphics card that can handle things better.

 

[CdTSnap]

I discovered it's a 4K issue. Editing on a Full HD monitor is fine. Much faster

Sounds like you need a better graphics card that can handle things better.

Haha nah not possible there. I have the GTX 1080ti, i7 7700K

It's a lightroom issue. It's perfectly fine on Capture One

 

[Light Guru]

not possible there. I have the GTX 1080ti, i7 7700K

It's a lightroom issue. It's perfectly fine on Capture One

Ahh you left those bits of information out. Yet another another issue I have heard about with Lightroom.

 

[table1349]

Lightroom is a waste if you have Capture One.

 

[CdTSnap]

Capture one is definitely better for editing raw images, as in clean processing but it sucks for wedding photographers. Bulk editing and organising etc.