Exposing to the right

[WayneF]

Many programs on the computer can and do show the RAW histogram (the cameras could too, they just don't).

Many? Not hardly. Name one sold on Amazon that does. Sure, it is not impossible for someone to write a program to analyze raw files (nerd stuff), but the common stuff sold to users do not bother. No point. Editors show expected output after your edits. The first and most necessary required part of that is RGB conversion.

See for yourself. Set your camera to some really ridiculous settings like +maximum contrast and +maximum saturation (so that the jpeg histo will be noticeably much different than the RAW), then snap a photo in RAW+jpeg mode.

Yes, certainly much different, but not because it is showing Raw. That is to say, it can only show RGB converted from Raw (the purpose of the raw editor, because that is only thing your monitor can show. Printers cannot show raw either.) True, the camera settings are not in Raw. Raw is Raw. Raw editors do not pick up the camera settings from Exif (excluding Nikon Raw editors, and excluding a fairly poor try at recovering white balance). The camera settings are not in Raw data, but there is advantage in having the camera show an approximately correct JPG to represent the raw image you hope to achieve. Auto WB does it for me (not claiming Auto WB is correct, but it is convenient and better then some obviously wrong choice). It does not matter to the Raw conversion, it will be set then.

Open the jpeg in photoshop and check out the histo, then open the RAW in different RAW editors and look at the starting histo. In any program I've ever used, they are quite different, suggesting that the RAW converter really is showing the RAW histogram. And there's no good reason why it wouldn't do that, since the histogram is just counting up numbers of dots with each color/lightness, and the pattern of them doesn't matter.

Of course they are different, but it suggests to me that the raw data simply does not have the camera settings in it. The Raw editor is showing RGB created from the Raw data. This is what Raw editors do, convert to RGB. Computer monitors cannot show Raw. Raw data is Bayer, RGGB, one color at each pixel, and RGB is interpolated frfom raw to be three RGB colors at each pixel.

When and if "both" were edited to be "correct", they would be more similar.

 

[apaflo]

Many programs on the computer can and do show the RAW histogram (the cameras could too, they just don't).
See for yourself. Set your camera to some really ridiculous settings like +maximum contrast and +maximum saturation (so that the jpeg histo will be noticeably much different than the RAW), then snap a photo in RAW+jpeg mode.

Open the jpeg in photoshop and check out the histo, then open the RAW in different RAW editors and look at the starting histo. In any program I've ever used, they are quite different, suggesting that the RAW converter really is showing the RAW histogram. And there's no good reason why it wouldn't do that, since the histogram is just counting up numbers of dots with each color/lightness, and the pattern of them doesn't matter.

What that suggests is only that most RAW converters are not able to use the camera's configuration for defaults. The histogram is not of the RAW sensor data at all, it is from the JPEG produced with the program's default configuration, which of course is different than an histogram of the camera produced JPEG. Without changing the RAW converter configuration, write the RGB image to a JPEG and use that same editor to compare it with the camera produced JPEG. Different, of course. And neither is a RAW histogram.

There are programs that do show a RAW histogram, and they are definitely labeled as such. UFRAW is a RAW converter that has both a RAW histogram and one for the current RGB conversion.

And if you want an accurate histogram in camera, just set all your settings to "neutral" or "zero" or whatever so that it makes minimum adjustments when converting to jpeg and thus minimal differences between the RAW and the LCD histogram. It still won't be absolutely perfect, but almost. The higher bit depth of the RAW doesn't matter for the LCD, because the LCD doesn't have that kind of resolution anyway.

The result isn't even close to accurate. It might be "useful" though, but that depends on the skill of the photographer at interpeting the data. There are very valid arguments for using an accurate White Balance, and there are other valid arguments for using only Daylight White Balance. In particular gamma (contrast) should be set relatively low. (Bit depth of the RAW, for an histogram, only affects scaling for the graph's axis and is not related at all to LCD resolution.)

Of course, if you ever shoot jpeg only, then don't set all your settings to zero, because you'll get horrible looking images. it's only if you know you'll be editing from RAW later ("custom shooting modes" are good for this to switch all the settings instantly)

ETTR is of no value when shooting JPEG only, and the histogram is exactly what it should be. It always very accurately shows data for the JPEG.

 

[WayneF]

Exiftool will not create or generate anything. It extracts data from tags.

But it doesn't take much to realize there necessarily has to be a full sized JPEG available while the RAW file is available on the camera... to allow the magnified view available on the LCD.

I agree both notions are possibly true, and would be reasonable. Or it is also conceivably true both instances could convert and create what they need for the job at hand. I don't know details. And agreed that it would be a bit heroic for ExifTool. But if just assuming, it would seem expedient for Nikon, no other processing load at that display zoom time.

I do still wonder how the 30MB of full size JPG can fit into the 50MB NEF file?


And now I'm wondering how I came up with a 30MB JPG file? Obviously it is, but the D800 manual says Fine JPG averages 16MB. I don't shoot much JPG, I would use DX for those uses anyway. Photoshop Image Size shows this one as size 103 MB, which it obviously should be. So it would seem my Fine got about 3:1 compression, and the manual assumes about 6:1. I assumed my image should be a smaller JPG than normal, it is a shot of my computer screen, and half of it is white. However 100% shows the pixel grid, so I guess it has extreme fine detail, a poor choice for this.

Still, this case is 30MB JPG and a 50MB sum, so I wonder how that can fit in there? Obviously, it must not be Fine.

 

[The_Traveler]

Somewhere back in the prior 78 posts at least twice it was said that there is only one correct exposure.
There is an assumption enclosed in that statement that is incorrect.
There is no guarantee that any single exposure that will provide the desired sufficiently good detail in the highlights and the shadows and will expose the centers of interest the way you want.
Mother Nature doesn't care.

 

[WayneF]

I didn't say Adobe (or whatever Raw editor) was showing me a histogram of the Raw data. You're making false assumptions. I said, "Later in the computer I can see histograms of my raw data before I process it to a final RGB photo..."; see illustration below.

Sorry, I just read the words you wrote. It is already processed, and it is always showing the RGB photo that it will output (usually after you do a few more things). But it never shows a histogram of raw data (which I think is your own complaint).

So I guess I do think the sensor raw capture is my "real data" and I do examine it carefully all the time to make sure I'm getting the best possible exposures. It would be nice if I could do that in the field but I'm happy with a process where I review my "real data" later and use that review to inform my work behind the camera. For me it's a more precise method than relying on the Red Green and Blue histograms displayed on the camera which we know have been interpreted by the camera processing software and do not directly represent my "real data."

Well, the RGB histogram is as good as it gets. It works out really well in practice. If unsatisfactory, go with the alternative I guess (which the only alternative is shooting JPG).

This thread began with a question about ETTR. To practice ETTR you need a way to examine the raw data as directly as possible to assess how you're exposing the sensor. Looking at data derived from the camera processed JPEG (camera manufacturer histograms of whatever flavor) is indirect and although it may be useful and possible to interpolate from that information it's off mark to suggest that it's anywhere near the "real data."

ETTR seems a mixed bag. The original idea was with techies, pushing everything as far right as it will go (which is possibly overexposure, but not clipping), so that at later processing, it can be pushed back down to where it should be, which also pushes any noise down too, offscale on the left. I think no one in this thread has mentioned that.

Then (those less techie, less concerned with noise) picked up the notion that all our pictures ought be exposed to the right, as a concept of "correct exposure". It is not of course, the histogram is NOT a light meter. It is true that many pictures do have white or bright content (which should be high in the histogram), so it does often help, but for example, the proverbial "black cat in a coal mine" picture obviously should not. The histogram has no clue where stuff should be, it only shows what result you got, good, bad, or ugly. We have to decide if it is where it should be, which frankly, seems better done in the JPG image on the rear LCD.

But if checking the histogram (I do too), at least look at the RGB histogram, and ignore the single gray luminosity histogram, which is useless regarding exposure.
The ONE THING the RGB histogram does show is if there is any clipping.

 

[table1349]

Would one of you Hijack Masters mind signing my copy of your training manual??:mrgreen:

 

[WayneF]

Would one of you Hijack Masters mind signing my copy of your training manual??:mrgreen:

No, sorry, I think you are an imposter, who has not even participated in this thread, except for two flamboyant tries to attract attention to yourself.

Seems to me this thread has stayed on track much better than most six page threads.

 

[Ysarex]

I didn't say Adobe (or whatever Raw editor) was showing me a histogram of the Raw data. You're making false assumptions. I said, "Later in the computer I can see histograms of my raw data before I process it to a final RGB photo..."; see illustration below.

Sorry, I just read the words you wrote. It is already processed, and it is always showing the RGB photo that it will output (usually after you do a few more things). But it never shows a histogram of raw data (which I think is your own complaint).

I think you're still failing to read the words I wrote and understand what I said. I don't know what "it" is in your second sentence that is always showing the RGB photo and never shows a histogram of raw data. That's between you and "it." I said I do examine histograms of my raw data and I showed you an illustration of raw data histograms. I'll show it to you again since it seems you missed it the first time. You might want to read the first line in that illustration where it says: RAW DATA HISTOGRAM.


View attachment 66301

You just previously told me, "No, Adobe (or whatever Raw editor) is not showing a histogram of the Raw data. Nothing does." Obviously you're wrong -- something does. You're looking at raw data histograms right now in the accompanying illustration.

I never did complain before about being unable to see raw data histograms since of course I look at them all the time. I understand I can't get them on the camera, but I'm OK with that since I do get them when I examine my raw files on the computer.

So I guess I do think the sensor raw capture is my "real data" and I do examine it carefully all the time to make sure I'm getting the best possible exposures. It would be nice if I could do that in the field but I'm happy with a process where I review my "real data" later and use that review to inform my work behind the camera. For me it's a more precise method than relying on the Red Green and Blue histograms displayed on the camera which we know have been interpreted by the camera processing software and do not directly represent my "real data."

Well, the RGB histogram is as good as it gets. It works out really well in practice. If unsatisfactory, go with the alternative I guess (which the only alternative is shooting JPG).

No, no and no. On the camera the RGB histogram is as good as it gets from the camera manufacturer. There is UniWB (which I don't like and don't use) but it is an option. Checking the RGB histogram works out really well for you in practice -- well that's your practice. I just finished showing you another alternative which is to examine the raw histograms later as a way to test and inform your practice exposing the sensor when you're behind the camera -- an alternative without shooting JPEG.

This thread began with a question about ETTR. To practice ETTR you need a way to examine the raw data as directly as possible to assess how you're exposing the sensor. Looking at data derived from the camera processed JPEG (camera manufacturer histograms of whatever flavor) is indirect and although it may be useful and possible to interpolate from that information it's off mark to suggest that it's anywhere near the "real data."

<snip>

But if checking the histogram (I do too), at least look at the RGB histogram, and ignore the single gray luminosity histogram, which is useless regarding exposure.
The ONE THING the RGB histogram does show is if there is any clipping.

Absolutely wrong. The RGB histogram shows if there is any clipping in the camera processed JPEG. That does not mean there is clipping in the raw file since we already know now that you were wrong to claim the RGB histogram shows us the "real data."

Joe

 

[table1349]

Would one of you Hijack Masters mind signing my copy of your training manual??:mrgreen:

No, sorry, I think you are an imposter, who has not even participated in this thread, except for two flamboyant tries to attract attention to yourself.

Seems to me this thread has stayed on track much better than most six page threads.

Perhaps you should re-familiarize with Post #1 and Post #53 by the OP and then look at #40 & #41. The OP seems to have gotten an answer to their question in their thread and have left the building by page 4.

 

[unpopular]

I think that there is a huge degree of misunderstanding about the histogram. It's important to remember that we didn't even have this option until digital came along - and still we managed to get satisfactory exposure somehow.

While it is true that the RGB histogram from the in-camera conversion is inaccurate, I have not found it to be so inaccurate that it's not useful, at least not in immediately telling me if I metered off the wrong region, and naturally, it's going to be conservative since conversion is a destructive process (you can't add detail to the jpg after all).

But overall, I do agree what has been said, understanding and trusting the meter is going to be much more fruitful than chimping around with the histogram.

One place I do use the histogram is to determine the quality of data recorded. As some of you know I expose for the hilights, and 95% of the time this works well. Where it does not is in low contrast scenes that should be rendered across a wide range of tones (i.e. subjects in shadow) where the bulk of the data is crammed up against the upper fifth of the tonal range, and practically nothing is recorded below the middle. In such instances it is better to expose to place Zone IX a little bit lower on the histogram and apply an S-Curve such that the shadows do not need to be stretched across tarnation.

 

[WayneF]

I think you're still failing to read the words I wrote and understand what I said. I don't know what "it" is in your second sentence that is always showing the RGB photo and never shows a histogram of raw data. That's between you and "it." I said I do examine histograms of my raw data and I showed you an illustration of raw data histograms. I'll show it to you again since it seems you missed it the first time. You might want to read the first line in that illustration where it says: RAW DATA HISTOGRAM.

Oops! You're right of course, I get it now. I was skimming and not paying enough attention. Sorry. That was an unexpected surprise, far from main stream, and I have never bothered to go those extra steps, because frankly, it seems pointless to me, unnecessary at best. Different strokes though, good luck with it.

Absolutely wrong. The RGB histogram shows if there is any clipping in the camera processed JPEG. That does not mean there is clipping in the raw file since we already know now that you were wrong to claim the RGB histogram shows us the "real data."

My entire point then was that the single gray luminosity histogram DOES NOT SHOW REAL DATA (or clipping for the most part). And the RGB histogram is the simple alternative, and in comparison, of these choices in the camera, yes, I do call that the real data. You've taken it to another level outside the camera, that, sorry, but I am not interested in.

Clipping is not a difficult situation to recognize. We do know about the clipping, it leaves data stacked up at 255. It is not always that bad, sometimes it is done intentionally (with care) to improve contrast. But as to correcting it, recognizing it in the camera is what seems important - it would be too late if later in your raw tools. We are going to have to make the same shifts later, so even though it does not affect the Raw file, just set the camera WB more correctly for the final goal (at least to Auto), and it will be a similar operation, and should not be much issue.

And if some situation is considered critical (probably involving Daylight red shifting), you can always just back off a bit, and put it back later in the Raw editor. If you have to always push the limit, there will be surprises.

 

[apaflo]

But overall, I do agree what has been said, understanding and trusting the meter is going to be much more fruitful than chimping around with the histogram.

The histogram helps show exactly what the meter is measuring. Meter's aren't wrong, but photographers commonly haven't enough clue what the meter has measured and therefore get unacceptable results because they trusted that the measurement accurately described something it didn't measure.

It's "chimping around with the histogram" that can make the meter useful.

One place I do use the histogram is to determine the quality of data recorded. As some of you know I expose for the hilights, and 95% of the time this works well. Where it does not is in low contrast scenes that should be rendered across a wide range of tones (i.e. subjects in shadow) where the bulk of the data is crammed up against the upper fifth of the tonal range, and practically nothing is recorded below the middle. In such instances it is better to expose to place Zone IX a little bit lower on the histogram and apply an S-Curve such that the shadows do not need to be stretched across tarnation.

Your lead into that is quite correct, but the example you give is not. Place Zone IX at just less than clipping, and in post processing place it a bit lower (into Zone IX when no texture is require or into VIII to get visible texture).

Or in other words, if something should be at about 245 in a JPEG image, expose it at 254 if you wish. Everything brighter than that will be clipped and lose all detail. But in post processing the brightness should be dropped to 245 or less, which will allow the detail to be visible. Areas that in the scene were brighter will still be clipped, whether at 255 or when reduced to 246. Note that basically all levels greater than 245 in a JPEG, whether printed or viewed on a computer monitor, will lack detail. They may not be totally into clipping, but close enough that it makes little difference.

Just originally exposing with Zone IX lower on the histogram merely loses dynamic range, actually causing the shadows to potentially need stretching.

 

[unpopular]

Well. Histogram reflects what has been recorded (and in most cases, converted)... but that's splitting hairs.

As for placement, I still maintain to place the outer edge of the tonal range (zone IX) to the outer limits of the histogram (245-250, in 8-bit scale) and just let them sit there, pulling shadows while leaving the hilight mostly alone. The reason for this is because you'll record deeper into the shadows, permitting better control over local contrast on the low-end.

But, keep in mind that my eye also tends to very soft contrast.

 

[Ysarex]

I think you're still failing to read the words I wrote and understand what I said. I don't know what "it" is in your second sentence that is always showing the RGB photo and never shows a histogram of raw data. That's between you and "it." I said I do examine histograms of my raw data and I showed you an illustration of raw data histograms. I'll show it to you again since it seems you missed it the first time. You might want to read the first line in that illustration where it says: RAW DATA HISTOGRAM.

Oops! You're right of course, I get it now. I was skimming and not paying enough attention. Sorry. That was an unexpected surprise, far from main stream, and I have never bothered to go those extra steps, because frankly, it seems pointless to me, unnecessary at best. Different strokes though, good luck with it.

OK, making some progress. Thanks for the good luck wishes.

Absolutely wrong. The RGB histogram shows if there is any clipping in the camera processed JPEG. That does not mean there is clipping in the raw file since we already know now that you were wrong to claim the RGB histogram shows us the "real data."

My entire point then was that the single gray luminosity histogram DOES NOT SHOW REAL DATA (or clipping for the most part). And the RGB histogram is the simple alternative, and in comparison, of these choices in the camera, yes, I do call that the real data. You've taken it to another level outside the camera, that, sorry, but I am not interested in.

Clipping is not a difficult situation to recognize. We do know about the clipping, it leaves data stacked up at 255. It is not always that bad, sometimes it is done intentionally (with care) to improve contrast. But as to correcting it, recognizing it in the camera is what seems important - it would be too late if later in your raw tools. We are going to have to make the same shifts later, so even though it does not affect the Raw file, just set the camera WB more correctly for the final goal (at least to Auto), and it will be a similar operation, and should not be much issue.

And if some situation is considered critical (probably involving Daylight red shifting), you can always just back off a bit, and put it back later in the Raw editor. If you have to always push the limit, there will be surprises.

So, I know what clipping is. And you just said, "The ONE THING the RGB histogram does show is if there is any clipping." That's not accurate. The RGB histogram will show clipping if it's present in the camera processed JPEG. That camera JPEG might be your "real data" but it's not mine. Clipping in the camera JPEG is not an adequate indication of clipping in the raw file especially if you're going to leave the camera set to auto WB as you advocate. In high contrast lighting conditions the camera JPEG processor will clip the highlights long before they're clipped in the raw file.

Joe

 

[sk66]

I don't really see the point in checking the raw file histograms... it's not something you can "work with." Maybe to get an understanding of how the camera histograms relate to the raw data so that you can more accurately interpret the camera histograms. But that's not much different than just working with the files in your conversion software and determining what's there (recoverable) that wasn't shown in the camera.