Affinity

[AlanKlein79 18]

I believe AI is correct in that instance, I've encountered that before. It occurs in the unique case where you use PureRaw or PL to create a linear DNG to pass to another app for continued processing.

If instead you use PL to complete the processing of the image then PL does a really good job of highlight reconstruction. Here's an example:

First the candidate image displayed in RawDigger to verify it meets the criteria for highlight reconstruction. I've highlighted the stats and we can see both the Red and Blue channels have zero clipping. The slight Green channel clipping is highlighted in the image in the sky above the garage. Because the Red and Blue channels are intact the clipped areas of the Green channels can be reconstructed from the Red and Blue data.



Next let's have a look at no attempt to perform highlight reconstruction. Here's the image in Affinity. That's a show stopper for me. I can't proceed to use Affinity's raw develop module with this image.



And here's that photo processed in DXO PL-9 with the highlight reconstruction very nicely accomplished. As noted I've previously encountered the specific case you identified with the linear DNGs and as I recall there were some pretty upset folks who I imagine have let DXO know how unhappy they were -- maybe they'll address it in a future update.

When you brightened the final picture, it made the leaves too orange. The color doesn't look natural. I think the original picture was underexposed maybe accounting for the issue. Can you reduce the saturation? Nice shot.

 

[Ysarex83 77]

When you brightened the final picture, it made the leaves too orange. The color doesn't look natural. I think the original picture was underexposed maybe accounting for the issue.

The exposure for the raw file was perfect -- any deviation from the exposure I set would unnecessarily degrade IQ.
The final result can certainly be processed to accommodate different tastes, so yes I could subdue the saturated orange leaves.

Can you reduce the saturation? Nice shot.

 

[AlanKlein79 18]

When you brightened the final picture, it made the leaves too orange. The color doesn't look natural. I think the original picture was underexposed maybe accounting for the issue. Can you reduce the saturation? Nice shot.

In the first shot, it looks like you underexposed the foreground so you wouldn't blow out the sky. You tried to capture it all but the range was too many stops due to front lighting and a shady foreground. So the foreground seems muddy. When you raised the brightness, the foreground just didn't hold up. These kind of shots are really troublesome. Some use grad ND filters or avoid them altogether, as I do often. They're very frustrating.

 

[Ysarex83 77]

In the first shot, it looks like you underexposed the foreground so you wouldn't blow out the sky. You tried to capture it all but the range was too many stops due to front lighting and a shady foreground. So the foreground seems muddy. When you raised the brightness, the foreground just didn't hold up. These kind of shots are really troublesome. Some use grad ND filters or avoid them altogether, as I do often. They're very frustrating.

That image was used to illustrate a specific technical problem. I presented a sequence of three images not to show a processing sequence but to illustrate the specific task of highlight reconstruction and the fact that Affinity doesn't do it and PL-9 does do it.
The first image shows the photo displayed in RawDigger which is an analysis tool not an image processor. All that matters there is that the photo is a proper candidate for highlight reconstruction -- it is.
The middle image shows Affinity's inability to perform highlight reconstruction and once I had gotten to the point where that was obvious I didn't bother to do any additional processing.
The third image was meant to illustrate that PL-9 does accomplish the highlight reconstruction and once I had that result there was no reason to keep working on the photo, I did what I needed to illustrate the issue.

So yes, the scene lighting is basically mild backlight which can be a problem for a camera image processor and which will have more than average dynamic range. The camera I was using is capable of more than 9, nearly 10 stops of DR. I set an exposure to make full use of the sensor's recording capacity and I nailed that. Here's the raw file histogram indicating a best-possible perfect sensor exposure (slight highlight reconstruction required). You can count the stops down from 4000 and the scene presented around 9 stops of DR -- within the ability of my camera.

That type of scene lighting does not present any difficulty for me and I treat it exactly as I would any other scene setting the same exposure for everything that I photograph (if handholding is possible): Set the brightest diffuse highlight at the sensor saturation threshold and click. It's quite rare that I need more than the ten stops of DR this camera can provide.

So I went back to PL-9 and picked up where I left off and finished processing the photo -- did the retouch, straightened it up and got the tone and color where I want it. And PL-9 does a good job of highlight reconstruction.

 

[AlanKlein79 18]

Here's a question about the process of capturing the full range that maybe you can help me understand. Even though a camera captures the full range or a particular scene, what if it was captured at the darker end of the histogram so you have to increase brightness let's say two stops during editing to make the final appear at the correct brightness? Would the image look the same if you captured it at the higher two stop originally, assuming all stops could be captured at either setting? Or would the darker captured image lose color accuracty? Would the image capture at lower brightness become muddier when raise two stops in editing? Other issues?

 

[Ysarex83 77]

Here's a question about the process of capturing the full range that maybe you can help me understand. Even though a camera captures the full range or a particular scene, what if it was captured at the darker end of the histogram so you have to increase brightness let's say two stops during editing to make the final appear at the correct brightness? Would the image look the same if you captured it at the higher two stop originally, assuming all stops could be captured at either setting? Or would the darker captured image lose color accuracty? Would the image capture at lower brightness become muddier when raise two stops in editing? Other issues?

I fully retired when Covid struck in 2020 -- that was my last year in a classroom of students. To help them understand how a digital sensor works I'd tell them a FF sensor camera can typically capture around 12 stops of usable data. Then I'd say, to make that real let's create a visual and I'd go to the laser printer in the room and take the stack of 8.5 X 11 paper. I'd put one sheet on the floor and tell them that's the first stop captured -- it's the dark shadows in the scene. How many pieces of paper for the next stop? Someone would always say one, maybe someone would say two. Then I'd explain that one stop more or less is either twice and much or half as much. Twice the one piece of paper on the floor is two pieces of paper and I'd lay those down. How many for the next stop? The answer is four -- double the two we just laid down and so I'd lay down four more pieces of paper. All along I was aiming my pieces of paper toward the door. Next stop is eight pieces of paper and I'd walk over and open the door. Then I'd ask do you think the hallway is long enough? How many pieces of paper are we going to need? Any math majors here? What's 2^12 and I can tell you the hallway isn't long enough and I don't have enough paper.

So your thinking is correct. If we describe the scenes we photograph as encompassing a range of stops which makes sense since we're photographers, then our sensors record less data for the low end stops and more data for the high end stops. It's exactly like raising ISO. Digital sensor's can't change their light sensitivity. When we begin to raise ISO we begin to collect less data for the scene -- we begin leaving the high end of the sensor unused, forcing the entire scene to be captured in the lower end. ISO then lightens our image to compensate but because we've collected less data the photo's IQ suffers compared to what it could have been if we had utilized the sensor's full capacity. We get more noise and lose color and detail.

Take a photo at base ISO and hopefully your camera will expose to use nearly all of the sensor's recording capacity then raise the ISO five stops and take the same photo again. Your camera will reduce exposure five stops and you're going to collect a lot less data; compare the IQ of the resulting two images. That's what you're describing. Now, as the technology has matured over the years we've gotten better and better at compensating for that loss of data. Nonetheless you're going to get better IQ at base ISO (using the whole sensor) than at ISO 3200 or 6400.

 

[AlanKlein79 18]

I fully retired when Covid struck in 2020 -- that was my last year in a classroom of students. To help them understand how a digital sensor works I'd tell them a FF sensor camera can typically capture around 12 stops of usable data. Then I'd say, to make that real let's create a visual and I'd go to the laser printer in the room and take the stack of 8.5 X 11 paper. I'd put one sheet on the floor and tell them that's the first stop captured -- it's the dark shadows in the scene. How many pieces of paper for the next stop? Someone would always say one, maybe someone would say two. Then I'd explain that one stop more or less is either twice and much or half as much. Twice the one piece of paper on the floor is two pieces of paper and I'd lay those down. How many for the next stop? The answer is four -- double the two we just laid down and so I'd lay down four more pieces of paper. All along I was aiming my pieces of paper toward the door. Next stop is eight pieces of paper and I'd walk over and open the door. Then I'd ask do you think the hallway is long enough? How many pieces of paper are we going to need? Any math majors here? What's 2^12 and I can tell you the hallway isn't long enough and I don't have enough paper.

So your thinking is correct. If we describe the scenes we photograph as encompassing a range of stops which makes sense since we're photographers, then our sensors record less data for the low end stops and more data for the high end stops. It's exactly like raising ISO. Digital sensor's can't change their light sensitivity. When we begin to raise ISO we begin to collect less data for the scene -- we begin leaving the high end of the sensor unused, forcing the entire scene to be captured in the lower end. ISO then lightens our image to compensate but because we've collected less data the photo's IQ suffers compared to what it could have been if we had utilized the sensor's full capacity. We get more noise and lose color and detail.

Take a photo at base ISO and hopefully your camera will expose to use nearly all of the sensor's recording capacity then raise the ISO five stops and take the same photo again. Your camera will reduce exposure five stops and you're going to collect a lot less data; compare the IQ of the resulting two images. That's what you're describing. Now, as the technology has matured over the years we've gotten better and better at compensating for that loss of data. Nonetheless you're going to get better IQ at base ISO (using the whole sensor) than at ISO 3200 or 6400.

Thanks for the explanation. However, let me ask it a different way. Leaving ISO and the aperture the same and just changing the shutter speed so that one shot is two stops faster, to prevent blowing out the highlights in the sky. It would gather less light and fewer photons in the darker areas then if you shot it at the correct shutter speed for the foreground. So when you raise the brightness while editing, wont you get less faithful color and more noise in the foreground, everything else being equal?

 

[Ysarex83 77]

Thanks for the explanation. However, let me ask it a different way. Leaving ISO and the aperture the same and just changing the shutter speed so that one shot is two stops faster, to prevent blowing out the highlights in the sky. It would gather less light and fewer photons in the darker areas then if you shot it at the correct shutter speed for the foreground. So when you raise the brightness while editing, wont you get less faithful color and more noise in the foreground, everything else being equal?

Yes, but what's your alternative -- you reduced overall exposure for a good reason. Blown highlights can really ruin a photo. So you have to choose here -- a photo with blown highlights or a photo with lower IQ in the midtones and shadows than the photo with blown highlights. Me? I hate blown highlights and it's a rare day that I'll let them clip.

What's the camera going to do? The camera will chose screwup for almost all the photos you take if you let it. In the backlight case the camera chooses to nuke the highlights into oblivion. Here's extreme backlighting; blue sky, sunshine and shooting right into the light. In order to not blow the highlights I had to set a -1.7 EC. There are thin clouds in that blue sky just above the horizon and I captured them. The camera, applying an additional 1.7 stops of exposure would have nuked those clouds and blown the sky. The camera JPEG below is what came from the exposure I set that retained the highlights. It's there so you can see what I did to process the raw file. You'll have to imagine the camera's screwup. So I lifted the shadows -- I wanted that big tree on the right to be visible with detail and texture. Is it noisier than it would be in the camera's blown highlight version? Yes, of course, but my highlights are not blown and for me that's the right choice.

Now let's flip the lighting around. Backlighting is not the typical photo for most folks. People much more commonly photograph frontlit scenes and scenes with lower contrast.

What's the camera going to do? The very thing that you're concerned about here and it's going to do that for lots if not most of everyone's photos. Let the camera select the exposure which will for the most part produce a good SOOC JPEG and the camera will underexpose the sensor. The very problem you've identified only it's doing it without good reason. In the backlit scene there was a good reason to reduce the exposure; don't blow the highlights. Now in a frontlit scene without any good reason to reduce exposure the camera underexposes the sensor in order to make a good SOOC JPEG. The photo below is middle of the day blue sky and sunshine, front to side lit. I had to set an EC of +1.7 to push the highlights up to the sensor's saturation threshold. The camera would have exposed 1 and 2/3 stops less. To paraphrase you, gathering less light and fewer photons in the darker areas than if I had shot it at the correct exposure to record the highlights and fully use the sensor. The problem you've identified is the default mode of operation for all of our cameras. I rejected that long ago.

I stopped paying attention to the SOOC JPEG and the camera's metering and I learned to set a precise exposure for the sensors in my cameras placing the brightest highlight at the sensor's saturation threshold -- the best possible exposure. Backlit or frontlit I do the same thing -- don't blow the highlights and use all of the sensor. Then I process the raw file. After years of doing that with multiple cameras I can tell you that my average exposure is +1 to +1.3 EC above the camera's metered exposure (I go -EC for backlight).

 

[Ysarex83 77]

Yes, but what's your alternative -- you reduced overall exposure for a good reason. Blown highlights can really ruin a photo. So you have to choose here -- a photo with blown highlights or a photo with lower IQ in the midtones and shadows than the photo with blown highlights. Me? I hate blown highlights and it's a rare day that I'll let them clip.

What's the camera going to do? The camera will chose screwup for almost all the photos you take if you let it. In the backlight case the camera chooses to nuke the highlights into oblivion. Here's extreme backlighting; blue sky, sunshine and shooting right into the light. In order to not blow the highlights I had to set a -1.7 EC. There are thin clouds in that blue sky just above the horizon and I captured them. The camera, applying an additional 1.7 stops of exposure would have nuked those clouds and blown the sky. The camera JPEG below is what came from the exposure I set that retained the highlights. It's there so you can see what I did to process the raw file. You'll have to imagine the camera's screwup. So I lifted the shadows -- I wanted that big tree on the right to be visible with detail and texture. Is it noisier than it would be in the camera's blown highlight version? Yes, of course, but my highlights are not blown and for me that's the right choice.

View attachment 297554

Now let's flip the lighting around. Backlighting is not the typical photo for most folks. People much more commonly photograph frontlit scenes and scenes with lower contrast.

What's the camera going to do? The very thing that you're concerned about here and it's going to do that for lots if not most of everyone's photos. Let the camera select the exposure which will for the most part produce a good SOOC JPEG and the camera will underexpose the sensor. The very problem you've identified only it's doing it without good reason. In the backlit scene there was a good reason to reduce the exposure; don't blow the highlights. Now in a frontlit scene without any good reason to reduce exposure the camera underexposes the sensor in order to make a good SOOC JPEG. The photo below is middle of the day blue sky and sunshine, front to side lit. I had to set an EC of +1.7 to push the highlights up to the sensor's saturation threshold. The camera would have exposed 1 and 2/3 stops less. To paraphrase you, gathering less light and fewer photons in the darker areas than if I had shot it at the correct exposure to record the highlights and fully use the sensor. The problem you've identified is the default mode of operation for all of our cameras. I rejected that long ago.

I stopped paying attention to the SOOC JPEG and the camera's metering and I learned to set a precise exposure for the sensors in my cameras placing the brightest highlight at the sensor's saturation threshold -- the best possible exposure. Backlit or frontlit I do the same thing -- don't blow the highlights and use all of the sensor. Then I process the raw file. After years of doing that with multiple cameras I can tell you that my average exposure is +1 to +1.3 EC above the camera's metered exposure (I go -EC for backlight).

View attachment 297555

Just a follow up here to how the camera handles setting an exposure when making a SOOC JPEG.

It's garden season and I don't get too much time with my cameras for all the gardening work, but we had some early rain yesterday morning that sent me off for a walk in the park. I only took one photo and it's a pretty bland landscape, but let's take a look. It's clouds in a blue sky with sunshine on at least half+ of the foreground, a good size shaded area right up front.

Like most cameras I can put the raw file back in the camera and recreate a JPEG and even make some changes. I did that and the change I made was to simulate the exposure the camera would have taken. Here's that camera JPEG. To simulate what the camera would have done I pulled the exposure back -1.3 stops. A decent camera JPEG -- sky is a little too light, I haven't adjusted any of the camera's picture controls. I originally took the photo with an EC of +1.3 above the camera's metered exposure.

The camera would have set an exposure 1 and a third stops less than the exposure I set. The exposure I set with an EC of +1.3 fully utilized the sensor without clipping any highlights. Here's the raw file histogram (12 bit ADC). The exposure I set is perfect for the sensor and fully utilized it's recording capacity without clipping any highlights. The camera would have set an exposure that left MORE THAN HALF of the sensor unused. That's staggering; let's repeat it, the camera's exposure to create a good SOOC JPEG of a sunlit landscape with clouds in a blue sky would have underutilized the sensor by more than 50%!

And here's the photo processed from the raw file.

I live two blocks from the park. My street dead ends into the park and becomes this thoroughfare.