Challenge: Noticeable differences between RAW and a jpeg edited in 16 bit mode?

[Ysarex]

Okay here you go. Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally, and put them on imgur:
imgur: the simple image sharer

Click back and forth from image #1 and image #2 at the top of the screen.

Or look below (these are smaller for some reason than the imgur ones). My version from the jpeg on the left, raw version on the right.

View attachment 45656View attachment 45657

Nice try, but you clipped the highlights. I did not clip the highlights. Clipping the highlights is photo felony #1. I NEVER clip the highlights. Helen saw it right away, "featureless patches in the clouds."

In your defense the highlights were clipped in the JPEG when you got it so the best you could do is reduce some of that clipping to "featureless patches" since there is no fix for blown highlights. Even though you did that you still clipped the highlights for example on the top of the white hand rail. You talk like you know something and I was anticipating you would reject the JPEG once you noticed the highlight clipping and we'd have to get into a discussion about exposure. It's worth stressing here that since the highlights are not clipped in my version then they must not have been clipped in the raw capture therefore the exposure was good. It was in fact textbook nailed.

In your original post you came to this faulty conclusion and posted it as a challenge, "Conclusion: There shouldn't really be any technical reason why it is more useful to edit a RAW than to edit a jpeg after first converting it to 16 or 32 bit." In my response I told you why your conclusion was faulty. I'll repeat myself, "The technical reason is that the camera JPEG processing software isn't adequately capable of rendering the photographer's intent and, after the fact with the raw data discarded, it is then usually impossible to realize the photographer's intent from the JPEG file."

That was high contrast lighting, but nothing really extreme. It's sunny sidelight just starting to shift to backlight. I knew from experience that this level of moderate stress was all it would take for the camera JPEG processing software to fail. I told you that as well in my original post, "The problem is that what you're suggesting as an approach to taking the photo isn't workable over a sufficient range of conditions -- at least not for me. And those tons and tons of settings available in our cameras are sub-crude and entirely inadequate to the task given what we know is possible once we have the raw file."

And the camera software did fail with your recommended settings applied. If we crank up the stress level by notching up the lighting contrast even more, the camera software will fail even more. I like to be able to work effectively under those more difficult conditions. Using your methodology you have to sit on the sidelines. You could argue now that you had the option in this example photo to reduce the exposure and help the poor camera software not clip the highlights. I really hope you have enough sense not to try and make that argument.

In defense of shooting JPEGs: there are those of us in the varied sub-disciplines of photography who have to shoot camera JPEGs (sports/action, journalism). For those specialists the controls in the cameras to assist in achieving a best possible result are invaluable and important tools. I'm glad those tools exist for them and I can adjust my expectations appropriately when I view their work. I hope the camera manufactures continue to improve those tools for their benefit.

But right now those tools place an artificial and unnecessary limit on what we can do when we don't for example have to meet a publication deadline. Why would you chose to tie your shoelaces together before you start a race?

Assuming you have a JPEG with all the info you need and can effect a light repair having converted it to 16 bit, will it hold up reasonably well? Yes. Getting that JPEG is however another matter and your estimation of what the tons and tons of controls in the camera provide is way off mark. Judobreaker told you that in the very first response to your post and as far as I'm concerned he settled it right then and there. Now you have empirical proof.

Joe

 

[amolitor]

The JPEG pulled together by the OP suggests to me that he or she simply doesn't see technical minutiae as well as some of us. There's nothing wrong with this, it doesn't make the OP evil. JPEG is specifically designed so that the average viewer can't see that stuff. It does mean that stuff that bugs YOU or ME isn't going to bug the OP.

I am almost certain that an image much more like the one from the RAW could be pulled out of the JPEG, but you gotta be able to *see* the stuff before you can make the stuff happen.

Hence my other thread.

 

[o hey tyler]

Well, that clears that up.

Everyone on the forum other than GavJenks: 1

GavJenks: 0

 

[TruckerDave]

I guess we need something to bicker about since the Nikon v. Canon thing has been beat to death.

Perhaps we should debate Chevy v. Ford.

Or, Coke v Pepsi....vhs v betamax...mac v windows.....iphone v android.....

 

[Gavjenks]

Okay here you go. Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally,

These two images look radically different. Was that your intention? What I take to be yours seems to lack shadow detail, and has quite ugly artifacting on the small awning on the extreme left of the frame. The colors seem less appealing, as well, but that might be simply that you like more saturation.

All the saturation is added, so that is not the fault of jpeg editing, as I could have simply added less. The difference is only about 2% in most areas of the image... I added more than I would have normally for sure in an attempt to try to match his photo. I'd say 2% is a pretty good match. Try loading up both and eye droppering them yourself.

The sharpening (artifacts) was also added, and I used more than I would have personally, again to match the photo. Notably the bricks. I did sharpening as a layer, non destructively, so I could have simply gone back and erased the awning part upon noticing artifacting, which I didn't. And it would have fixed that. Admittedly, sharpening is the trickiest part. However, even with a RAW, if you want best quality, you need to learn to do some difficult sharpening techniques ANYWAY, because the raw converters will not do things like apply separate levels or types of sharpening to areas with different artistic sharpening needs in the same image.

As for highlight and shadow clipping, here are the luminosity histograms to show that the clipping is not really as bad as all that in any of the pictures, and not very different either. Almost identical highlight clipping profiles:


Nor do I see much difference in the actual photos. I can see more details like reflected leaves in the RAW, but that's because I played them down on purpose due to considering them environmental pollution (it's a house, not a tree). And the clouds may have the illusion of being more clipped, because I made them whiter (rather than what i thought to be a less desirable blue cast and wimpy appearance). Seeming additional clipping in the clouds is an illusion of them being white, I think. Histograms don't lie. Nor is it that the histogram is measuring two different things. Spot checking the clouds in particular with eyedroppers and local histograms reveals neither being very clipped at all (the railings are closer than the clouds are).

Also if you examine both clouds very closely, the RAW version's does not actually have more "steps" or shades to it than the jpeg version. The transition is not smoother. It's just shifted a little bit differently, and again, more noticeable due to being bluer, so that you have color and luminosity cues together.



And please note again: This is not a challenge of "can gav guess the settings you used when you don't tell him the settings (which I asked for btw)." We aren't testing my eyes or my color or artistic judgment. We're testing the technical limitations of jpeg editing. And the technical clipping is indeed no worse than the version made from a converted RAW. Nor is there color banding, etc. The two photos are not IDENTICAL, because I had to guess a lot of things (and also, he seems to have done a lot of edits other than just converting from RAW, such as keystoning for perspective, etc. So even more unnecessary guessing). But they are pretty damn close, enough so IMO for purposes of pointing out that you can do the same SORTS of things to the jpeg without getting posterization or other major issues. Most complaints here seem to be based on artistic complaints that arose from me guessing things or in some cases attempting to improve slightly.

 

[amolitor]

Oh, sorry. I didn't look at the histograms, I just looked at the pictures to see how similar they were. Now I know.

 

[Gavjenks]

Also btw for those looking at the histograms: the reason that the raw and the jpeg both pile up on the left side so much more than the original jpeg is because of the weird solid black border that he included in his image for some reason, and that I copied in attempting to match it. So most of those are human-added feathered solid black pixels, not really part of the image per se.

Oh, sorry. I didn't look at the histograms, I just looked at the pictures to see how similar they were. Now I know.

The histogram proves that the data still exists, which is the point of this thread. The point is not to prove how good of an art forger I am, or how well I can guess editing steps that were not told to me, or artistic intent that was not told to me.

Regardless of whether you like my version better, the data still exists. It is not actually clipped any more than the raw version, and it is not any more posterized. So if you prefer subtler, bluer clouds like he had, then you can simply make them bluer, and then lower the lightness a bit and play with the curves until you get the more visible texture you desire. The information is there to allow you to do that without ending up with any more posterization than his version has.

it's just that apparently, based on people's reactions in this thread, the diffrence between 238/238/238 and 233/233/233 is not as visible to the human eye as the difference between 220/220/245 and 215/215/240. It is equally as visible to a computer, though, and the computer should be perfectly capable of shifting it over to where you can see it, without any loss of data, if that's what you want. I just thought it didn't look as good.

 

[Helen B]

I just saw featureless patches in the white clouds (that's not the same as clipping) and poor shadow detail. When I select the same patch of what I see as featureless (by putting both images into the same document and making the relevant parts coincident, then making a selection) and look at the histogram it tells me exactly the same story. The raw histogram is broader and has a greater standard deviation (12 against 2.4 for the area I selected). Technical issues aside, it is visually obvious that there is more subtle gradation and hue in the highlights of the raw-originated file, and the appearance is what matters. Isn't it obvious to you?

 

[Benco]

I just saw featureless patches in the white clouds (that's not the same as clipping) and poor shadow detail. When I select the same patch of what I see as featureless (by putting both images into the same document and making the relevant parts coincident, then making a selection) and look at the histogram it tells me exactly the same story. The raw histogram is broader and has a greater standard deviation (12 against 2.4 for the area I selected). Technical issues aside, it is visually obvious that there is more subtle gradation and hue in the highlights of the raw-originated file, and the appearance is what matters. Isn't it obvious to you?

Clearly it's not. :banghead:

 

[Gavjenks]

Isn't it obvious to you?

1) It is obvious, yes.
2) I like it better with less gradation and bolder whites. The raw version clouds look "wimpy" to me. I did this on purpose, whether subconsciously or not.
3) Even though the raw version has more visible gradation, in the brightest parts of the clouds, the gradation only occurs in 3-5 distinct steps/shades. I actually think this looks a little bit posterized. Making it brighter in my opinion also makes it look better by masking the posterization a bit (people can't tell differences as well at brighter ends of the range, even if its the same number of steps in a computer), without looking completely blown out. "nearly but not quite blown out" looks better to me than "gradation, but in visible steps." At least for white objects like clouds. Apparently I'm in the minority on that.
4) How do you measure standard deviation? That's a neat trick. And if it is indeed that much different, then maybe there was some significant data loss after all. It didn't look nearly that different to me from the graphs, but I don't actually have real statistics like that. Just eyeballing the curves.

If you tell me how, then I will try to run some statistics on the raw converted image versus the original jpeg for the whole cloud region and post them. That would be more in line with the technical claims in this thread, while avoiding issues of artistic taste.

 

[amolitor]

I don't understand at all now.

You're saying that you can take the JPEG version of a photo and make a picture out of it that's different that the one Joe made out of the corresponding RAW file? I don't think anyone's gonna argue with you on that point.

I truly don't understand what on earth the thrust of this argument can be. I thought you were saying that you could get the same results out of JPEG as out of RAW, if only you were careful in exposing the JPEG, and in processing it afterwards. Now you're apparently not saying that? Or what?

 

[Ysarex]

....And the technical clipping is indeed no worse than the version made from a converted RAW.

Blatantly wrong. Shadow clipping is a misdemeanor at best. Highlight clipping is a felony. There's one thing worse than highlight clipping and that's filling in those blown highlights with a flat dead tone. Here's a threshold test of the camera JPEG that shows the highlights that the camera software clipped.



Your version of the photo and your histogram doesn't show the same clipping in the highlights because you filled in those blown areas with a solid tone -- that's the felony that comes with a mandatory life sentence. Blown highlights are ugly -- filling them in with a solid tone is butt ugly.

There is no highlight clipping whatsoever in my version of the photo. From my perspective that was the point of this exercise. The camera software couldn't deliver a JPEG under moderately stressful side lighting. That JPEG from the camera was damaged BEYOND REPAIR by the camera software. You shouldn't have even made the attempt to process it! To suggest that what you did filling in those highlights compares well to processing the raw file in which those highlights are not clipped is laughable.

AND Let's not forget that this was moderately stressful sidelight. The lighting contrast can get much higher and from this point on your camera can only fail worse. Your single option shooting JPEGs in similar lighting is to reduce exposure which is an even worse idea than trying to repair camera JPEGs. I chose this scene and lighting contrast because I expected the camera failure would be minor and I was curious to see how you'd react. What do you do under higher contrast conditions?

You've been granted that, up to the limits of the camera software's ability to deliver a JPEG that isn't badly damaged, some light repair with the photo converted to 16 bit produces serviceable results. You can work like that if you want to, knock yourself out. I'm going to continue to work in a manner that permits me to photograph a whole lot of stuff that you can't without trying to convince yourself that pasty filled in highlights really do look OK. You've got an answer to your original post. Don't make me go outside and take a backlit photo for you.

Joe

 

[Ysarex]

The histogram proves that the data still exists, which is the point of this thread. The point is not to prove how good of an art forger I am, or how well I can guess editing steps that were not told to me, or artistic intent that was not told to me.

Your histogram proves you filled in the clipped highlights with a solid tone -- the featureless patches (I think posterization is a common word for that) that Helen is referring to. I just posted a threshold test for the original JPEG that clearly shows the highlight clipping caused by the camera. (I wouldn't call what you did forgery.)

Regardless of whether you like my version better, the data still exists.

The non-photographic dead tones you created exists and it looks out of place and just butt ugly. The original JPEG with it's clipped highlights is there for anyone to examine.

It is not actually clipped any more than the raw version, and it is not any more posterized.

There is no highlight clipping in the version I processed from the raw file and I didn't fake in any flat tones to fill in the clipping.

Joe

 

[480sparky]

You guys argue all you want:

I'm gonna go take some pictures......

 

[Gavjenks]

Yes okay there are like 30 pixels of clipping when you mask it like that. I really disagree that that amount of clipping you show in the clipping mask you made above is anything very significant to worry about. I am sitting there myself straining to look at that exact part of the image, knowing full well what I'm supposed to be seeing that's so "butt ugly" and am unable to even MAKE myself see it as bad looking at all. It's a white cloud, 0.5% of it is pure white, and the areas around that part gradient smoothly into that pure white, not abruptly . So what?

The camera seems to have done a pretty darn good job in general in choosing a sampling range from the RAW in order to meet your request of very low contrast

However:

The lighting contrast can get much higher and from this point on your camera can only fail worse.

This is a fair point. Based on the fact that it had any clipping here, I imagine that if you used 0,-4,0,0 settings on that first photo you took of the backlit tree, it would indeed have failed pretty badly even at those maxed out negative 4 settings.

I'll try to test it later when I get home with my camera, with a bright lamp in front of a black window shade or something, but I suspect that you're right, and it'ss going to be fairly bad.



If so, then yes, that would lead me to the conclusion that shooting RAW is helpful in normal circumstances practically. However, this doesn't seem like it SHOULD be the case theoretically. Rather, it is a failing of Canon's camera settings that you can't make them strong enough to meet your desires. I.e., it only lets you have some small proportion of the influence that you should have over the RAW->Jpeg sampling pattern, for no apparently good reason.