Challenge: Noticeable differences between RAW and a jpeg edited in 16 bit mode?

[Ysarex]

....Conclusion: There shouldn't really be any technical reason why it is more useful to edit a RAW than to edit a jpeg after first converting it to 16 or 32 bit.

The technical reason is that the camera JPEG processing software isn't adequately capable of rendering the photographer's intent and, after the fact with the raw data discarded, it is then usually impossible to realize the photographer's intent from the JPEG file.

In a response further back in this thread you said this: "And if you are thinking about color and light in the field (which you should be), then you shouldn't need very much flexibility. The out of camera jpeg should already be pretty much where you want it, and whatever minor (10-20%) corrections you may want to do will not be sufficient to cause visible color posterization.

There are tons and tons of settigns in your camera for setting up a proper white balance. There's custom compensation, there's pre-set WB for different standard lighting conditions, and there are options to calibrate using cards in the field. You have all the tools you need, most of which only need to be set once for an entire shoot."

This is where you're completely missing it. If you can coax a JPEG from the camera software that is close to the photographer's intent, and I grant that this is in limited circumstance possible, then I'll agree that a camera JPEG once converted to 16 bit can tolerate light repair and remain serviceable (wince). The problem is that what you're suggesting as an approach to taking the photo isn't workable over a sufficient range of conditions -- at least not for me. And those tons and tons of settings available in our cameras are sub-crude and entirely inadequate to the task given what we know is possible once we have the raw file. So in part it's a case of because we can. We're always going to be pushing to go as far as we can. Using the camera processing software is like tying one hand behind your back, then tying your shoelaces together and trying to run.

I am curious as to how I might be proven wrong, in reasonable situations where you didn't completely fail your photo?

You asked for a challenge, here you go. I took this photo:

I'm happy with that photo. That's what I saw (photographer's intent) and what I expected to take away when I clicked the shutter. I am using a Canon camera and the day I unboxed it I set those tons of settings to null and the picture style to faithful as you have suggested. Here's the camera JPEG for that image:

And here's a link to that JPEG at full-res so you can download it and go to work processing it: Cannon_wetland The processing goal of course is to realize the photographer's intent and produce the same image I produced from the raw file. You can post it back here when you're finished.

You may want to argue that I could have used those tons and tons of camera settings to coax the camera software into producing a result closer to my intent and I might have better luck then trying to repair the JPEG. I know better -- not in lighting conditions like this.

Your very limited proposition that a JPEG once converted to 16 bit can tolerate light repair is valid but it makes me wince. I've never seen a camera processed JPEG that I couldn't repair and improve. I spent years chasing after this very proposition since I've always wanted a camera to carry with me everywhere. When digital cameras became shirt pocket size I started buying them and using them in precisely this way. They only produced JPEGs and I would work to get the best JPEG possible and then repair it in Photoshop. I went through 1/2 a dozen of those cameras constantly frustrated. I'd buy one and then when it bit me (impossible to repair JPEG and I knew a raw file would have worked) I'd ship it off to a niece or nephew and eventually get another one. I finally compromised on a slightly larger camera that would save raw files. I'm happy now and my production rate has soared with no more frustration.

Joe

 

[Gavjenks]

Your intentions here, clearly, were to 1) reduce the contrast of the actual scene, and 2) up the saturation. Simple histogramming of your jpeg versus your intended shows a huge, two-peak-with-valley-in-the-middle mega contrasty pattern on the jpeg, versus a fairly even distribution in the intended.

The saturation is not a big deal, because upping saturation does not degrade data (there's already plenty of hue information in the jpeg. They're just boring hues, but the data is there, and thus doesn't have to be invented or stretched causing banding).

The contrast, however, IS a big deal. In order to follow my suggested process, you should have shot this with a custom picture mode of [0 -4 0 0] ([sharpness contrast saturation color tone]).

Did you do that? If not, then this is not a valid challenge photo, because I never claimed to be able to make whatever you want happen without any consideration of settings when taking it. In other words, you threw out the data that I needed and kept a bunch that I didn't need, when with the right settings, you could have done the opposite and made it possible to match.



Most people don't routinely do what I am suggesting (not necessarily because it's wrong. Possibly just because it requires more thinking in exchange for time, and because it is not the default setting). Thus, it is very unlikely that any existing photos are going to be valid challenge photos here. If you don't routinely use picture style and white balance in camera before a shoot to push the jpeg as far as possible in the right direction, then none of your old photos will work for demonstration purposes. You'd have to go take a new photo with optimal settings, and then post that as a challenge to match to the RAW processed version.

 

[o hey tyler]

Upping saturation can introduce banding or posterization. If that's not degradation, I don't know what is.

 

[amolitor]

Possibly just because it requires more thinking in exchange for time, and because it is not the default setting).

This makes no sense at all. You're essentially proposing that we do something in-camera rather than out of camera. You're also waving away all the technical issues with an airy "well, they're not visible, anyways" which is at best questionable.

Since when is doing something in-camera faster than doing it out of camera?

 

[dbvirago]

I was having a similar discussion today. Each file type has its own uses, I get that. BUT can anyone tell me why a wedding photographer would shoot solely in JPEG?

The reason I have seen is the large number of images they shoot and the short time to process. However, since a couple of clicks can convert the RAWs to JPGs, I'm not sure I get it either.

For me the reason was simple. I shot JPGs the first six months or so after getting a DSLR until the day I came home from a full day of shooting landscapes with the WB set to Tungsten.

 

[DBA]

The idea is that RAW slows down your workflow and your burst speed.

So for those 500 images, editing them in RAW might take 2-3 extra hours compared to editing them in 16 bit photoshop from jpeg. During those 2-3 hours, you could have been out in the field taking another 100 photographs, of which many more than 1 would have been keepers.

False and false. Nothing about RAW slows down your workflow. You can preset your import settings from RAW just like you can set your in camera JPEG settings. Heck if you use the manufacturer's RAW converter it will actually read the camera picture settings out of each individual file and apply them on import without any loss of data. Also 2-3 hours in the field taking photos of nothing because I miss my photo opportunities while messing with menus in the camera does not sound like time well spent to me.

Oh and RAW doesn't slow down burst mode. It only slows down continuous shooting speeds after the buffer is full which rarely happens and never was a problem even when I was firing off a ludicrous number of frames at a motorsport event.

What Garbz said. :thumbsup:

Unless you're uploading images [online] directly from your camera, I don't see how it'll slow down your workflow (maybe a minute or two while transferring them to your pc).

 

[Gavjenks]

Upping saturation can introduce banding or posterization. If that's not degradation, I don't know what is.

Upping saturation is a linear change. taking an array [1 2 3 4 5] and adding 3 to all the numbers = [4 5 6 7 8]. The sample rate is the same, there is zero loss of precision, and it is entirely reversible. This would be true even if you edited it at 8 bit. if you get banding from saturation changes, then you must be doing something fancier than just upping saturation linearly. Are you perhaps upping saturation with a curve algorithm or similar? OR are you upping saturation to ceiling causing clipping and then reducing it again (silly to ever do, but would create posterization)?

Since when is doing something in-camera faster than doing it out of camera?

Opening a jpeg OOC in photoshop takes about 1 second. Processing a RAW into a jpeg even with automatic settings can take 10-20 seconds depending on the software you use (and for a full memory card of photos, that can add up to hours of difference. Your camera does this, however, while you're out shooting, and it is also hardware accelerated for that specific task). More importantly, if you are shooting RAW in the first place, you obviously intend to at least some of the time do something OTHER than automatic processing, otherwise there was zero reason to use up all that extra memory space, etc. So you're also spending time fiddling with custom conversions some % of the time, just to get it into a photoshop compliant format. That could be anything from minutes to hours depending on what proportion of your RAWs you insist on hand crafting into rasterized form. Especially since processing time on doing any edits in a RAW editor usually takes 2-5x longer in my experience than doing an equivalent edit in photoshop, even in 16 bit.

Setting aside the question of whether or not that helps you achieve a better photo, the fact that it slows you down is a pretty objective one.

The reason I have seen is the large number of images they shoot and the short time to process. However, since a couple of clicks can convert the RAWs to JPGs, I'm not sure I get it either.

Why would you use up all that extra space and burst speed and tie up your computer for longer if all you're going to do is the exact same thing your camera already does for you without any of those drawbacks?

Well, you said why: so that you can correct your mistake if you have the wrong white balance for an entire shoot (something that a cursory glance at the histogram on your LCD for the first couple of photos would have highlighted just as well). If that level of extreme convenience to not have to consider your color or lighting appeals to you, then sure, shoot RAW. I've suggested that all along. I'm simply saying that in MY opinion, you should be carefully considering your light and color and contrast anyway, regardless of the format you shoot, if you're going to come up with good photos as often as possible, so setting these settings correctly should come for free as a side effect of the mental work you should already be doing.

"well, they're not visible, anyways" which is at best questionable.

This is a challenge thread. Says right in the title. Go out and shoot a photo in both RAW and jpeg, using the optimized settings as I have discussed to make the jpeg most closely match your desired artistic intent. And then process the RAW as you see fit, and give the jpeg to me and I will attempt to match it. Whether the differences become observable or not is an empirical question.

(note: it would be most fair if you gave me the settings you used in your RAW converter too. This is a technical possibilities challenge, not a "can Gavjenks guess what edits you did" challenge)

 

[amolitor]

I am utterly uninterested in taking your challenge, and actually, so is everyone else. Your challenge is, as near as I can tell, 'Go do this complicated thing that's a pain in the ass and provides no benefit whatsoever, and prove to me that it's not almost as good as what you already do'. Why on EARTH would anyone want to do that?

 

[Gavjenks]

Also 2-3 hours in the field taking photos of nothing because I miss my photo opportunities while messing with menus in the camera does not sound like time well spent to me.

Setting your picture style and white balance is something you would do only once for a given lighting situation (e.g. set it once when you start in mid day, then maybe reset at sunset, or if you switch to indoors, etc.), outside of extraordinary circumstances.

And since you should already be reconsidering ALL of your settings anyway when you switch to a new lighting situation, this does not require any greater presence of mind than you already need. It's just one more step added to the routine "recheck everything" you already have to do when switching environments.

Oh and RAW doesn't slow down burst mode. It only slows down continuous shooting speeds after the buffer is full which rarely happens and never was a problem even when I was firing off a ludicrous number of frames at a motorsport event.

Depends on your camera body. Maybe if you have a fancy pro grade full frame with 10 FPS already or something, and some huge buffer, it's not an issue. On my Rebel T2i, though, the difference is very noticeable.

Shooting jpeg, i was able to fire at full speed for something like 50 shots in a row before I gave up and assumed it was just infinite.

Shooting RAW only, my camera stopped after 7 shots and had to sit there and think for like 2 seconds. Then each subsequent photo was about 1 per second instead of 4, with thinking in between each time.

I am utterly uninterested in taking your challenge, and actually, so is everyone else. Your challenge is, as near as I can tell, 'Go do this complicated thing that's a pain in the ass and provides no benefit whatsoever, and prove to me that it's not almost as good as what you already do'. Why on EARTH would anyone want to do that?

Your strawman version is not interesting, no.

The actual thing I am describing however, is. "Go do this thing that requires a bit of extra complexity, but not as much as dealing with RAWs, and let's see if it has any impact on image quality."

At WORST, this is equally as complex as dealing with RAWs. Even in optimal circumstances for your side of the argument, where somebody simply auto converts every single RAW to jpeg without touching their keyboard (using it only as a fallback safety net for catastrophic mistakes), it still equates to pushing a couple extra buttons vs. pushing a couple of other extra buttons.

Even then, it is useful to know about and keep in mind in order to choose which set of strengths and weaknesses is best for a given situation. "Do I need burst speed/amount or am I worried about filling up my memory or do I not have easy access to a computer with RAW software on it to clear out my cards? Jpeg with in camera optimized settings set once per lighting situation." "Do i not need those things right now and prefer to have a layer of extra security? RAW."

 

[Benco]

Your intentions here, clearly, were to 1) reduce the contrast of the actual scene, and 2) up the saturation. Simple histogramming of your jpeg versus your intended shows a huge, two-peak-with-valley-in-the-middle mega contrasty pattern on the jpeg, versus a fairly even distribution in the intended.

The saturation is not a big deal, because upping saturation does not degrade data (there's already plenty of hue information in the jpeg. They're just boring hues, but the data is there, and thus doesn't have to be invented or stretched causing banding).

The contrast, however, IS a big deal. In order to follow my suggested process, you should have shot this with a custom picture mode of [0 -4 0 0] ([sharpness contrast saturation color tone]).

Did you do that? If not, then this is not a valid challenge photo, because I never claimed to be able to make whatever you want happen without any consideration of settings when taking it. In other words, you threw out the data that I needed and kept a bunch that I didn't need, when with the right settings, you could have done the opposite and made it possible to match.



Most people don't routinely do what I am suggesting (not necessarily because it's wrong. Possibly just because it requires more thinking in exchange for time, and because it is not the default setting). Thus, it is very unlikely that any existing photos are going to be valid challenge photos here. If you don't routinely use picture style and white balance in camera before a shoot to push the jpeg as far as possible in the right direction, then none of your old photos will work for demonstration purposes. You'd have to go take a new photo with optimal settings, and then post that as a challenge to match to the RAW processed version.

Most people don't do that because there's no need to, if you shoot in raw then none of that matters. Let's approach it from a different angle: what's the advantage of shooting and editing in JPG over raw? I really don't see any, if you do that you have to go to way more trouble to achieve results that might nearly be as good.

 

[amolitor]

I mostly shoot JPEGs. JPEG is fine stuff, I use it a lot.

When I want to move more of my decision making in to post, then I shoot RAW. That's what RAW is for, it's for deferring decision making and (as a special case of that) recovering from errors.

Claiming that JPEG is "just as good" as RAW is simply wrong, though. It's good stuff, it's really quite good, make no mistake. I'd really have to work at it to cook up a test case with interestingly visible issues, and I'm a pretty technical boy. I pretty much know where JPEG buries its bodies. But I could do it. I know I could do it because I understand the technologies involved. I don't have to drop a small cannonball and a large cannonball from the tower in Pisa to prove that they'll fall at the same rate, and I am certainly not going to climb up a great big tower with two cannonballs and a stopwatch just because some guy on the internet asks me to. Neither do I need to fiddle around with custom picture settings and so on.

My camera actually does a bang up job making JPEGs when I set it to "standard" mode and let it do the metering. No custom picture modes involved. They're not *quite* as good as what I can get with RAW and post processing, though.

 

[Gavjenks]

Most people don't do that because there's no need to, if you shoot in raw then none of that matters. Let's approach it from a different angle: what's the advantage of shooting and editing in JPG over raw? I really don't see any, if you do that you have to go to way more trouble to achieve results that might nearly be as good.

I've outlined them multiple times:
1) RAW takes up 5x more memory on your card than jpeg does (16 bit is 2x + the light compression of OOC jpeg). Doubly problematic if you're on vacation and don't have a computer with you with proprietary software for converting RAWs (thus, can't clear out your card at the end of the day).

2) RAW dramatically reduces the number of photos you can take in a burst on low-mid range camera bodies (in mine, RAW alone will limit you to 1 FPS after just 7 shots, and RAW+jpeg will limit you to 1 FPS after just TWO shots. Jpeg allows 4 FPS continuously seemingly forever, by comparison)

3) RAW cannot be used OOC even if you dont want to edit it at all. Websites and printers can't read it, so you are obligated to process it through photoshop or whatever other RAW software. This is a disadvantage if you are taking normal photos of friends or something, for instance, and are not trying to win the pullitzer prize in photojournalism. Oftentimes while just out and about taking photos, I will have about a 30/70 mixture of funny snapshots that I want to share with somebody on facebook as a joke vs. real images I intend to be beautiful. With jpeg, I can pick out the silly ones and post without processing. With RAW, I'd have to painstakingly process them all first OR sit there and constantly switch back and forth between formats in camera, OR use up even more memory and cripple my ability to take burst shots with RAW + jpg.

4) For those images that you do want to process regardless, RAW takes much longer to load into photoshop than jpeg. At best this is annoying (if automated/uniform conversions) and I have to go eat a sandwich or something instead of editing photos like I want to, while it processes slowly. At worst, this wastes tons of my time, if I am actually attempting to do photo-by-photo optimized RAW conversions and am sitting at the keyboard the whole time.


The next two are more subjective musings than cold hard disadvantages, but:


5) In my opinion, the safety net of RAW encourages a photographer to be a little lazy and not worry about lighting or coloration in the field. Lighting, color, and exposure are all part of composition! And if you don't consider them carefully, then you won't even take the same photos that you would have if you did. i don't mean that you would have used different settings. I mean that if you don't consider these things, you will not even end up pointing your camera at the same scenes as if you do. And usually, what you do end up pointing at will be substandard composition. No amount of post processing can ever fix poorly conceived composition.

IF it were the case that RAW was the only way to achieve technical clarity and vividness in a print, then this would be a temptation that you'd just have to suck it up and learn to avoid. But IF you can actually achieve the same quality with jpeg (point of this thread is that I believe you can), then the ability to force yourself to not give in to that temptation to be lazy is a very nice feature. And if you're thinking about these things anyway as you shoot, then the picture style settings, etc. will be second nature side effects, and not a hassle at all.

6) Depending on how much you take editing of RAWs into your own hands, you may be shooting yourself in the foot. This varies depending on how you use them. If you edit RAWs in high quality, custom made RAW converters by major brand name software companies, then you're probably fine. But if you attempt to do basic things like noise reduction and sharpening yourself using sub-optimal software, you are almost guaranteed to end up doing worse than the professional Canon or Nikon engineers do in modern cameras, unless you spend half an hour fine tuning every little thing specifically to each image and have years of practice. In short: RAWs used naively by amateurs who don't use the right tools can lead to worse results than even a jpeg OOC.

 

[amolitor]

So shoot JPEG. What a good idea! Virtually everyone shoot JPEG some of the time.

Just don't pretend that there isn't any advantage to RAW, eh?

 

[Gavjenks]

So shoot JPEG. What a good idea! Virtually everyone shoot JPEG some of the time.

Just don't pretend that there isn't any advantage to RAW, eh?

There is certainly not always an advantage in every instance.

But in some situations, there pretty much objectively is. For instance, if I'm spending all day shooting action sequences of athletes where the interesting part happens very rapidly (like pole vaulters or gymnasts), and I'm in a gymnasium with consistent lighting, using a normal person camera under $3000, then it is pretty much flat out wrong to be using a RAW. RAW would limit my bursts to the point where I will simply miss action that I need to capture. It will fill up my cards due to shooting all day, requiring me to invest unnecessary money in a bunch of memory. I have no reason to require the flexibility of adjusting exposure in post, because the conditions are not rapidly changing, and I can just nail it all in camera within 5 minutes of walking in the door. And working with the huge volume of images would be unnecessarily unwieldy when i get them on a computer, for no real advantage.

For situations like that, it is important to know how your in camera settings work for WB and picture style, to correctly use a more advantageous jpeg format.

In many other situations, it is more ambiguous than that, and in some, RAW is clearly superior, i admit.

 

[o hey tyler]

There is certainly not always an advantage in every instance.

But in some situations, there pretty much objectively is. For instance, if I'm spending all day shooting action sequences of athletes where the interesting part happens very rapidly (like pole vaulters or gymnasts), and I'm in a gymnasium with consistent lighting, using a normal person camera under $3000, then it is pretty much flat out wrong to be using a RAW. RAW would limit my bursts to the point where I will simply miss action that I need to capture. It will fill up my cards due to shooting all day, requiring me to invest unnecessary money in a bunch of memory. I have no reason to require the flexibility of adjusting exposure in post, because the conditions are not rapidly changing, and I can just nail it all in camera within 5 minutes of walking in the door. And working with the huge volume of images would be unnecessarily unwieldy when i get them on a computer, for no real advantage.

For situations like that, it is important to know how your in camera settings work for WB and picture style, to correctly use a more advantageous jpeg format.

In many other situations, it is more ambiguous than that, and in some, RAW is clearly superior, i admit.

Ok, no one cares. Shoot how you want. Others will do the same. Get over it.