Request for processing a RAW file

spacediver · Dec 7, 2015

I've just rendered a photo, which I took of the phosphors of my CRT display.

Here is the rendered image that I made, which was taken at the vertical center of my display.

Here is the RAW file for a similar image, which was taken halfway between the centre and the edge of the display, so the phosphors are a tiny bit larger than in the center (it's an FW900 which has a variable dot pitch).

Workflow was as follows:

Measure chromaticity and luminance (XYZ) of my display primaries.

Measure each of the three camera RGB filter's response to each of these primaries using dcraw and Matlab.

Take RAW image of phosphors.

Use dcraw to convert RAW to a 16 bit linear tiff.

Using Matlab:

Subsample the bayer array into XYZ values, using custom built transformation matrices based on the previous measurements.

Transform XYZ values into linear sRGB values.

Normalize sRGB values based on areas in the image where I knew the primaries were at full luminance, and where I knew the black parts were.

Gamma correct the values.

The result is pleasing, but I'm curious about one thing.

Given that I subsampled the RAW image, I lost half the resolution of the sensor (each 2x2 RGGB chunk of sensels was converted into an XYZ triplet, which was then converted into an RGB triplet representing a single pixel). This approach maintains a lot of image accuracy at the expense of resolution.

However, I'm very curious to see how a full resolution image that has been demosaiced looks like, compared to my subsampled one. I've played around a bit with RawTherapee, but before I experiment more, I'd like to see what's possible by an experienced user.

For the purposes of this particular post, I'm less concerned about color accuracy, and more concerned about image detail.

Below is a crop of the fully rendered image. Incidentally, the width of this cropped image corresponds to precisely 1 mm across the phosphor layer. Image was taken with a Canon Rebel EOS 450D (reversed kit lens), mounted on a newport linear stage.

Here is what happens when I use the default settings in RawTherapee. Colors are off, which is understandable, but a lot of the subtle detail is blown out, and the dark patches aren't black enough.

407370 · Dec 8, 2015

I have read your post several times and I am still unsure what you are trying to achieve.
Do you require an accurate rendition of reality or an augmented rendition with adjustments for colour and detail?
The RAW image supplied is quite noisy (normal for such magnification) this could be how it actually looks but is more likely to be a result of camera processing. To bring out detail I would suggest Tone Mapping. That can be done through several different processes and you have control over what the final image looks like.
3 examples of the RAW file you supplied with different tone mapping processing:

The first image is what the RAW file actually looks like. The second and third images are reduced noise examples.

Hope that helps.

spacediver · Dec 8, 2015

Thanks very much for your post, it's extremely helpful

407370 said:
I have read your post several times and I am still unsure what you are trying to achieve.
Do you require an accurate rendition of reality or an augmented rendition with adjustments for colour and detail?

Going for an accurate rendition.

407370 said:
The RAW image supplied is quite noisy (normal for such magnification) this could be how it actually looks but is more likely to be a result of camera processing.

Hard to tell - could be noisy shadow detail, or an 8 bit reflection of reality. The phosphors are not perfectly smooth surfaces - they have a grainy texture and exhibit signs of wear. What appears to be noise there in your left example may be an artifact due to 8 bit rendering. Experimenting with a greater exposure may provide answers. Another possibility is that the black offset of the RAW file hasn't been accounted for. Or it might be demosaicing artifacts, though I'm not familiar enough with demosaicing algorithms to say how likely this is (my gut tells me it's not demosaicing related).

The first step of my processing of the linear 16 bit TIFF file was to remove the offset - this may have clipped some of the noise. Also, when I normalized the black level in the RGB domain, and then converted it into an 8 bit image, below zero information was also clipped. That may be why I didn't see much apparent noise in my rendering.

407370 said:
T.o bring out detail I would suggest Tone Mapping. That can be done through several different processes and you have control over what the final image looks like.
Hope that helps.

I'm new to the concept of tone mapping, but as far as I understand, the technique I used didn't utilize any tone mapping (unless you count the gamma correction), as I was able to extract XYZ values directly from the RAW file. Importantly, the dynamic range of my display is more or less able to be captured by my camera, so the luminance range of the actual "scene" and the luminance range of the extracted image are naturally mapped once I normalized everything.

I can understand, however, that without access to XYZ information (as is normally the case), tone mapping may be necessary to maintain scene fidelity. Your second image shows a lot of the detail that I was hoping to see, especially in the green strips, so it's great to know that this is possible!

edit: forgot to ask: why would high magnification increase noise?

john.margetts · Dec 8, 2015

I think the first thing to note is that you cannot display the RAW file - any file that is capable of display has been processed to some degree.

The file your computer displays before you process a RAW file is either 1) an embedded JPEG that your camera produced or 2) a default processing of the RAW file.

Sent from my A1-840 using Tapatalk

spacediver · Dec 8, 2015

john.margetts said:
I think the first thing to note is that you cannot display the RAW file - any file that is capable of display has been processed to some degree.

Well you can display an undemosaiced representation (i.e. a raw image of the color filter array) by mapping the values onto a monochrome colormap. You can visually assess the image that way.

But yes, the first of the three images that 407370 posted isn't what the RAW image looks like - I'm gathering it's a demosaiced image.

john.margetts · Dec 8, 2015

spacediver said:
john.margetts said:

I think the first thing to note is that you cannot display the RAW file - any file that is capable of display has been processed to some degree.

Click to expand...

Well you can display an undemosaiced representation (i.e. a raw image of the color filter array) by mapping the values onto a monochrome colormap. You can visually assess the image that way.

But yes, the first of the three images that 407370 posted isn't what the RAW image looks like - I'm gathering it's a demosaiced image.

You could do that, but the result would be mostly green, not full colour.

Sent from my A1-840 using Tapatalk

spacediver · Dec 8, 2015

john.margetts said:
You could do that, but the result would be mostly green, not full colour.

Yes, and for the purposes of visually assessing noise, or overall image structure, that's ok.

spacediver · Dec 8, 2015

I should also clarify - when I use dcraw to convert the RAW to a 16 bit TIFF - it really isn't doing anything processing wise. I use the switches -D -4 -T, and this preserves all the raw sensel data in its original undemosaiced state. The rendered image I produced is all custom processing.