Shooting in RAW

[BrickHouse]

You guys must be fun at parties.

 

[Overread]

Few things that might help

1) Most (all?) RAW photos have a JPEG embedded into them. This is the image you view on the LCD and also what you view in the computer as well. There are even programs out there which can extract the JPEG from the RAW. The JPEG is edited to whatever your cameras JPEG editing settings are set to (which is why most keep them to faithful/neutral so that its as near to the RAW result as possible when reviewing the LCD and histogram on the back of the camera).

2) Adobe Camera RAW (which runs when you open any RAW in any photoshop/lightroom program) doesn't save nor change any data in a RAW shot. All it does is save a separate file which lists all the adjustments you make to that file in lightroom this also includes all the area selective work you do as well.
When you save the RAW you pick a file format for the save (JPEG - TIFF - PSD - other) and then the program saves that output in that output format. Some data is lost in this transfer because the file has no need to keep all of the light data that isn't being presented in the edit.

 

[soufiej]

No, what cnet said is correct. It's not what you said. The fact that there are lossy raw files and lossless JPEG files is simply an objective fact. It's a fact in the same way that Abraham Lincoln was the president during the civil war is a fact.

What CNET is saying is that the camera can make the JPEG conversion, or you can save the raw file and make the conversion later, to your own liking. But the conversion will have to eventually be made, if you ever want to look at it.

For a raw file to be viewable, it has first to be converted into a ****lossy**** format. Even tiff, which is both lossless and viewable is lossless because it holds the additional information a raw file stores, in addition to the lossy viewable format.

You just seem to not understand what the difference between a digital positive and a digital negative are. A digital positive image can directly be viewed on a display. A digital negative (like raw) doesn't even have a color space defined. Raw images literally don't have color. They literally don't have a contrast level, as recorded there is no way of saying what level in a raw image is black, which is grey and which is white. They have data that can be converted to color spaces and grey scale, but as soon as you do that it isn't a raw file anymore.

Don't you remember? You said I never mentioned what a RAW file is.

"What CNET is saying is that the camera can make the JPEG conversion, or you can save the raw file and make the conversion later, to your own liking. But the conversion will have to eventually be made, if you ever want to look at it."

No, Cnet hasn't said anything about "the conversion will have to eventually be made". Yes, the RAW file can be downsampled to a corresponding Jpeg - or simply converted into one of several other formats. Cnet is not saying, though, the RAW file must be downsampled and compressed into a Jpeg package.

If I load a Jpeg file into my monitor, I have used a compressed, processed data package. It is "x" size. If I load the RAW version of that same image into my monitor, the viewable data has not been downsampled to Jpeg format. It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing.

RAW formatting is described as lossless in the same way the audio WAV file format is lossless. It is a full wave representation of the original source. Its accuracy exists in part in its sampling rate. Jpeg is lossy in the same way Mpeg is lossy. But they are both simply data packages, on's and off's, which cannot be viewed or printed until they have been turned into (converted to) an analog format. The process of "reconstruction" from a digital file to an analog output though is not lossy in the sense data is intentionally thrown away as it is in downsampling and compressing the data into a lower bit rate package.

There is a noise component which exists, embedded in the digital file system of either format which acts as a baseline reference for how the file will be reconstructed. You can loosely define the system as 8 bit, 12 bit, 16 bit or more by the number of bits of noise in the file. The actual digital noise is at a very low level (the least significant bits) which generally don't intrude on the analog file. If, in a RAW file, you consider that to be "lossy", then you're not comprehending how the digital data is being reconstructed into an analog format.

Make your "lossless" Jpeg file if you like by essentially using a higher bit rate in recording the data but you still will not have a data package as large as the corresponding RAW data package. That is because Jpeg is, ultimately, a lossy, processed and compressed format.

The rest is all just words you are confusing in your head.

 

[Ysarex]

.....If I load a Jpeg file into my monitor, I have used a compressed, processed data package. It is "x" size. If I load the RAW version of that same image into my monitor, the viewable data has not been downsampled to Jpeg format. It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing....

I think the crux of the problem is here: "It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing."

When you open a RAW file to edit you do not see "the data as the sensor recorded it." You see a converted RGB photo that is certainly lossy due to that conversion process. It does not have to be JPEG compressed which is a specific process, but you do not see the data as the sensor recorded it. The process to demosaic the RAW data is an interpolation process and as such does what the term interpolation implies -- it is lossy. There's all kinds of ways to be lossy. To see how lossy all you have to do is start comparing different RAW demosaicing algorithms. Furthermore, when you open a RAW file to edit you see a software interpretation of that interpolated data in terms of color and tone response. Open the same RAW file in two different processors and you'll see two different interpretations. An interpretation is also as the term implies lossy -- just not JPEG lossy -- there's all kinds of ways to be lossy.

Joe

 

[fjrabon]

[QUOTE="soufiej, post: 3455420, member: 173037]



If I load a Jpeg file into my monitor, I have used a compressed, processed data package. It is "x" size. If I load the RAW version of that same image into my monitor, the viewable data has not been downsampled to Jpeg format. It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing.

[/QUOTE]

Here is the problem, that single quote makes it clear: you can't load a raw file onto your monitor. You don't have to convert the raw file to a JPEG in particular, but you do have to convert it to a digital positive image, of which JPEG is one type (tiff is another). all of your other misunderstandings here stem from this fundamental misunderstanding. You can't view a raw file. You can't display a raw file on your monitor. You can't print a raw file. Raw files aren't digital images. You ***can*** display a JPEG image on a monitor (along with other types of digital positive images, like tiff, png, gif,etc). A JPEG image is a digital image

I'm not even going to wade down the MP3 path again because arguing over a poor analogy is pointless when the poor analogy stems from a basic misunderstanding of the thing we are talking about.

 

[Alexr25]

A RAW file after editing is essentially the same package size as the original RAW file

No, Cnet hasn't said anything about "the conversion will have to eventually be made". Yes, the RAW file can be downsampled to a corresponding Jpeg - or simply converted into one of several other formats. Cnet is not saying, though, the RAW file must be downsampled and compressed into a Jpeg package.

If I load a Jpeg file into my monitor, I have used a compressed, processed data package. It is "x" size. If I load the RAW version of that same image into my monitor, the viewable data has not been downsampled to Jpeg format. It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing.

A RAW file is never edited or changed in any way. The data contained in the RAW file is used to create a new file in a format that can be displayed by the computer. RAW file data cannot be displayed as an image on your monitor, the image you see as the file thumbnail is the file's embedded jpeg, to "edit" a RAW captured image you must first use the RAW data to generate a file in a format the the computer can edit.
You need to grasp this fact to make any sense of RAW processing.

RAW formatting is described as lossless in the same way the audio WAV file format is lossless. It is a full wave representation of the original source. Its accuracy exists in part in its sampling rate. Jpeg is lossy in the same way Mpeg is lossy. But they are both simply data packages, on's and off's, which cannot be viewed or printed until they have been turned into (converted to) an analog format. The process of "reconstruction" from a digital file to an analog output though is not lossy in the sense data is intentionally thrown away as it is in downsampling and compressing the data into a lower bit rate package.

File formats are never described as lossy or lossless but rather the compression algorithms that some file formats use can be lossy or lossless. Whether a compression is lossy or lossless is not defined by whether data gets thrown away but rather on whether the original file can be reconstructed from the compressed file. Since WAV files are uncompressed it makes no sense to talk of them as lossy or lossless, they just are. RAW files on the other hand do come in all flavours, you can have RAW files that are uncompressed, compressed using lossless compression or compressed using lossy compression. The data inside the file is still RAW and still needs to be converted to an image format file (tif, jpeg, psd etc) before it can be displayed and edited.

There is a noise component which exists, embedded in the digital file system of either format which acts as a baseline reference for how the file will be reconstructed. You can loosely define the system as 8 bit, 12 bit, 16 bit or more by the number of bits of noise in the file. The actual digital noise is at a very low level (the least significant bits) which generally don't intrude on the analog file. If, in a RAW file, you consider that to be "lossy", then you're not comprehending how the digital data is being reconstructed into an analog format.

Make your "lossless" Jpeg file if you like by essentially using a higher bit rate in recording the data but you still will not have a data package as large as the corresponding RAW data package. That is because Jpeg is, ultimately, a lossy, processed and compressed format.

Nothing to do with bit rate.
Jpeg is a file format that uses a lossy compression algorithm but the amount of compression can be set by the user. So if you set the compression level to zero compression you will get an uncompressed Jpeg file. Quite simple really once you stop thinking of files a lossy or lossless and start thinking of the compression algorithm used to create the file.

The rest is all just words you are confusing in your head.

What can I say about who's head is confused?

 

[soufiej]

[QUOTE="soufiej, post: 3455420, member: 173037]



If I load a Jpeg file into my monitor, I have used a compressed, processed data package. It is "x" size. If I load the RAW version of that same image into my monitor, the viewable data has not been downsampled to Jpeg format. It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing.

Here is the problem, that single quote makes it clear: you can't load a raw file onto your monitor. You don't have to convert the raw file to a JPEG in particular, but you do have to convert it to a digital positive image, of which JPEG is one type (tiff is another). all of your other misunderstandings here stem from this fundamental misunderstanding. You can't view a raw file. You can't display a raw file on your monitor. You can't print a raw file. Raw files aren't digital images. You ***can*** display a JPEG image on a monitor (along with other types of digital positive images, like tiff, png, gif,etc). A JPEG image is a digital image

I'm not even going to wade down the MP3 path again because arguing over a poor analogy is pointless when the poor analogy stems from a basic misunderstanding of the thing we are talking about.[/QUOTE]

*

HERE is the problem as I see it ... In an earlier post you said, "Basically the program recreates a new JPEG from scratch every time you edit it."

Now you say, "You don't have to convert the raw file to a JPEG in particular ... "

The former is what I objected to. Then you launched into the same ol' CYA issues which don't really matter when it comes to how we display a digital file because we can't see a digital file. Saying now the RAW file can exist as another form other than Jpeg is not what you had earlier claimed. End of discussion!

We are dancing around words and trying to prove a point which, IMO, largely doesn't need to be made. Most simply because how we express the words to describe the process can vary and still be either correct or incorrect depending on where in the use of a digital file we care to discuss the process.

The issue remains whether a Jpeg format is a lossy format. And it is, by definition. It is a compressed (therefore, "proceesed") format without a restoration algorithm which returns the file to its (mostly) lossless size and quality. Again, saying, "File formats are never described as lossy or lossless but rather the compression algorithms that some file formats use can be lossy or lossless", is just arguing words. Is a motor driven device meant to transport individuals from here to there a "car" or an "automobile"? Is it driven by a "motor" or an "engine"? Introducing more word games into this discussion is only going to muddy the waters even further.

We do describe file formats as being lossy or lossless as a convenient shorthand rather than describing the mathematical algorithms used to compress, compress/decompress or simply transfer existing data files from one storage location to another. Jpeg is always described as a lossy file format just as MP3 is always a lossy file format. You can increase the file size of a Jpeg compression but you can never create a Jpeg file with all the data of a RAW file. If you could, what would have been the point in creating the Jpeg file format? None! So why bother with inventing Jpeg if you have RAW already? How you display or print the file data package is not the real issue here. Jpeg is "x". Raw is "X". You can decrease the variance between the two but the Jpeg is always going to be the lower case (compressed) data package and the RAW file is always going to be the upper case (uncompressed) data package.

How a RAW data file is displayed is again dancing around a dead tree trunk in the dark of night. Data points in use are considered "least significant" and "most significant" with the vast majority of data points falling between those two extremes. Saying we cannot display a RAW file is stating the obvious and it is exactly what I have stated on several occasions. Humans do not see nor hear in a strictly binary fashion. Digital storage media and "use" is based upon a strictly binary process. Therefore, right, we cannot display a series of on's and off's on a display monitor. It must be converted to an analog format - which everyone arguing about how a RAW file is displayed simply seems to ignore. You all talk as though you are viewing 1's and 0's on your monitor. And you are NOT!

However, given the conversion to an analog format which is usable for viewing, printing or listening to music, digital storage began as a lossless format and several options remain for using the highest amount of available data points to construct and reconstruct the digital file. Lossy formats were invented using algorithms which relied on perception. Basically, if "this" exists, then "that" can be discarded without altering perception of the event. A lossy file format will always be an "if this/then that" file format used for compression of the file data package size. Because many folks detected problems with the "this then that" formats, newer formats which compress and decompress were created. Ways to increase the package size were designed for the lossy formats. None though ever construct/store/reconstruct/transfer and use the original source image to the "X" size of a lossless file. Again, if they act the same as a lossless file, why create a different algorithm? Other than the monetary value of a proprietary format, there would be none.

Arguing further about how much data is not viewed or not heard or not printed is beyond the point. Those data points which are used "mathematically" to form the file format exist in all formats and are deemed "least significant". You can describe them as "lossy" but that is again semantics and, basically, wrong. We cannot see an analog image by viewing the data from a digital file. Therefore, a portion of the digital file is used to instruct the converter as to which file format it is working with. Period. Subcodes! Period! We can all say the same thing in a dozen different ways. None of that changes the basic issue of the debate.

The original debate was whether Jpeg is a "lossy format", which it is. This debate was whether editing a "RAW FILE" must create a Jpeg file. Which it does not. We have also debated whether RAW file formatting was accepted as " ... the image as seen by the camera's sensor", which we do!

All the rest becomes BS words!!! And a vast waste of time to continue!

 

[photoguy99]

This is pretty epic. Well done, everyone.

 

[fjrabon]

If I load a Jpeg file into my monitor, I have used a compressed, processed data package. It is "x" size. If I load the RAW version of that same image into my monitor, the viewable data has not been downsampled to Jpeg format. It is the data as the sensor recorded it and it is "X" size package. At no point must a RAW file be "converted" to a Jpeg for viewing or printing.



Here is the problem, that single quote makes it clear: you can't load a raw file onto your monitor. You don't have to convert the raw file to a JPEG in particular, but you do have to convert it to a digital positive image, of which JPEG is one type (tiff is another). all of your other misunderstandings here stem from this fundamental misunderstanding. You can't view a raw file. You can't display a raw file on your monitor. You can't print a raw file. Raw files aren't digital images. You ***can*** display a JPEG image on a monitor (along with other types of digital positive images, like tiff, png, gif,etc). A JPEG image is a digital image

I'm not even going to wade down the MP3 path again because arguing over a poor analogy is pointless when the poor analogy stems from a basic misunderstanding of the thing we are talking about.

*

HERE is the problem as I see it ... In an earlier post you said, "Basically the program recreates a new JPEG from scratch every time you edit it."

Now you say, "You don't have to convert the raw file to a JPEG in particular ... "

The former is what I objected to. Then you launched into the same ol' CYA issues which don't really matter when it comes to how we display a digital file because we can't see a digital file. Saying now the RAW file can exist as another form other than Jpeg is not what you had earlier claimed. End of discussion!

We are dancing around words and trying to prove a point which, IMO, largely doesn't need to be made. Most simply because how we express the words to describe the process can vary and still be either correct or incorrect depending on where in the use of a digital file we care to discuss the process.

The issue remains whether a Jpeg format is a lossy format. And it is, by definition. It is a compressed (therefore, "proceesed") format without a restoration algorithm which returns the file to its (mostly) lossless size and quality. Again, saying, "File formats are never described as lossy or lossless but rather the compression algorithms that some file formats use can be lossy or lossless", is just arguing words. Is a motor driven device meant to transport individuals from here to there a "car" or an "automobile"? Is it driven by a "motor" or an "engine"? Introducing more word games into this discussion is only going to muddy the waters even further.

We do describe file formats as being lossy or lossless as a convenient shorthand rather than describing the mathematical algorithms used to compress, compress/decompress or simply transfer existing data files from one storage location to another. Jpeg is always described as a lossy file format just as MP3 is always a lossy file format. You can increase the file size of a Jpeg compression but you can never create a Jpeg file with all the data of a RAW file. If you could, what would have been the point in creating the Jpeg file format? None! So why bother with inventing Jpeg if you have RAW already? How you display or print the file data package is not the real issue here. Jpeg is "x". Raw is "X". You can decrease the variance between the two but the Jpeg is always going to be the lower case (compressed) data package and the RAW file is always going to be the upper case (uncompressed) data package.

How a RAW data file is displayed is again dancing around a dead tree trunk in the dark of night. Data points in use are considered "least significant" and "most significant" with the vast majority of data points falling between those two extremes. Saying we cannot display a RAW file is stating the obvious and it is exactly what I have stated on several occasions. Humans do not see nor hear in a strictly binary fashion. Digital storage media and "use" is based upon a strictly binary process. Therefore, right, we cannot display a series of on's and off's on a display monitor. It must be converted to an analog format - which everyone arguing about how a RAW file is displayed simply seems to ignore. You all talk as though you are viewing 1's and 0's on your monitor. And you are NOT!

However, given the conversion to an analog format which is usable for viewing, printing or listening to music, digital storage began as a lossless format and several options remain for using the highest amount of available data points to construct and reconstruct the digital file. Lossy formats were invented using algorithms which relied on perception. Basically, if "this" exists, then "that" can be discarded without altering perception of the event. A lossy file format will always be an "if this/then that" file format used for compression of the file data package size. Because many folks detected problems with the "this then that" formats, newer formats which compress and decompress were created. Ways to increase the package size were designed for the lossy formats. None though ever construct/store/reconstruct/transfer and use the original source image to the "X" size of a lossless file. Again, if they act the same as a lossless file, why create a different algorithm? Other than the monetary value of a proprietary format, there would be none.

Arguing further about how much data is not viewed or not heard or not printed is beyond the point. Those data points which are used "mathematically" to form the file format exist in all formats and are deemed "least significant". You can describe them as "lossy" but that is again semantics and, basically, wrong. We cannot see an analog image by viewing the data from a digital file. Therefore, a portion of the digital file is used to instruct the converter as to which file format it is working with. Period. Subcodes! Period! We can all say the same thing in a dozen different ways. None of that changes the basic issue of the debate.

The original debate was whether Jpeg is a "lossy format", which it is. This debate was whether editing a "RAW FILE" must create a Jpeg file. Which it does not. We have also debated whether RAW file formatting was accepted as " ... the image as seen by the camera's sensor", which we do!

All the rest becomes BS words!!! And a vast waste of time to continue!

Nobody here, least of all me, debated that JPEG is a lossy format. (well, except that there is a such thing as lossless JPEG, but as it's not a common standard, we can ignore that).

What we have been pointing out is that, as you are using the term, all formats that are displayable are lossy. Raw is not a displayable format. This isn't a word game.

What I, and others, have repeatedly pointed out to you, is that when you view an image, you are viewing a lossy format.

And yes, when you "edit a raw image" you are not editing the raw file. You are editing how your raw converter displays a lossy viewable image.

Even formats which are displayable and lossless, like some versions of tiff, create a lossy displayable image.

You keep sticking to some "analog to digital" conversion idea, which is what happens in digital audio, but not digital imagery. There is no analog to digital conversion for digital imaging, until you get to the stage of literally lighting up pixels. And at that stage it's simply turning the pixels that the digital files told the computer to do.

This very question belies a complete misunderstanding that you call a "word game" which is really just you not understanding how it works:

"why bother with inventing Jpeg if you have RAW already?"

Because raw is not a displayable format. JPEG is (along with a few other types of digital images).

This isn't about digital to analog conversion (I think your WAV v MPEG analogy here is confusing you, comparing WAV to MPEG is comparing apples to apples. They're the same sort of thing. You can play a WAV file on a computer in the ***same exact way*** you can play an MPEG. The only purpose of MPEG is to compress. This is not the case with raw v JPEG. Comparing raw to JPEG is comparing apples to organges. They don't even do the same thing. JPEG ***is not*** just a compressed version of a raw file. JPEG is viewable, raw is not. You have to convert a raw file to a viewable format. I hate to keep repeating this, but this is the fundamental issue that you seem to keep repeatedly missing. When you "view a raw file" what you are viewing is almost certainly a jpeg. When you take a raw file with your camera and view it with your camera, you are looking at a jpeg that your camera made with its internal raw converter. If you want to view a raw image, it ***has to be converted to a viewable format*** JPEG does not require this. JPEG is viewable on its own.

That is why a displayable format is necessary. Yes, jpeg usually runs a compression algorithm, but you can set the level of compression such that it is only removing information that can't be displayed anyway. But again, let's not get into minutiae, since it's not the issue you're misunderstanding. You're misunderstanding the more fundamental issue.

JPEG does two things in most cases:

1) converts the raw image into a viewable image. It assigns tones on the grey scale and color (and a few other things).
2) compresses.

You understand the 2nd part for the most part (though I think you're analogizing it with MP3 confuses the issue more than it clarifies). But you keep missing the first issue. Repeatedly.

You can't "only use raw"

Yes, I said "you don't have to convert to JPEG in particular..." but notice the important part is what you cut with the ellipses. I said that because there are other alternatives to JPEG. But they're lossy viewable formats. You ***HAVE TO CONVERT A RAW IMAGE*** The conversion can be to tiff, jpeg, bmp, gif, png, etc. But you can't just leave it in raw, if you want to view it.

 

[soufiej]

Nobody here, least of all me, debated that JPEG is a lossy format. (well, except that there is a such thing as lossless JPEG, but as it's not a common standard, we can ignore that).

OH! PLEASE!!!

Find another hobby.


Of course that is exactly what you have been saying and you can't even keep it straight when you deny you have said exactly that.

GIVE IT UP!!!

 

[fjrabon]

OH! PLEASE!!!

Find another hobby.


Of course that is exactly what you have been saying and you can't even keep it straight when you deny you have said exactly that.

GIVE IT UP!!!

link?

Outside of saying that there is a such thing as JPEG lossless (which is true) where did I say that? What I was responding to is how you kept talking about not needing to use JPEG. You kept saying that JPEG's only purpose is to compress data (which isn't true, its primary purpose is to create a viewable image, yes it also compresses, I never said otherwise). You kept saying that people shouldn't use JPEG because it's lossy, that they should instead use raw, because it was "higher fidelity." You kept saying that you could view and edit a raw image (which isn't true).

What I kept saying like 2983432 times is that you ****HAVE TO USE A LOSSY FORMAT TO VIEW AN IMAGE**** so raw is only important if you are going to edit color and exposure. I also said that if you open a raw file, you are viewing a lossy image, you are not viewing the raw file. I said that if you "edit a raw file" what you are doing is creating a conversion (almost always JPEG) that is lossy and changing the way that the raw converter gets the JPEG from the raw file.

To make it as simple as possible, JPEGs come from raw files. Always. You can't create a JPEG without a raw file. Any time you look at or edit an image, you are not looking at or editing a raw file.

The two are completely different things. The choice isn't whether or not you use raw or JPEG, the choice is when you convert from raw to JPEG. You can have the camera do it for you, shortly after it takes the picture, or you can have a program on your computer change your raw file into a jpeg. but you ****MUST CONVERT YOUR RAW FILE TO JPEG (or another viewable image type)**** raw files themselves are completely useless on their own.

 

[Mr. Innuendo]

You ever see the movie "Scanners"? That part right before the dude's head exploded?

Yeah... That.

 

[Vic Vinegar]

I just want to chime in with something that you might not realize until it's too late. If you're doing RAW, get lots of memory cards or get high capacity cards because 30-40 RAW images can easily take up a GB worth of storage.

 

[baturn]

I wonder what happened to the OP??

 

[Nettles]

He's out taking photos.