D7000 sensor benchmark results are in!

[emh]

Second, my original analogy is exactly the right one. DxO mark scores are measured results based on RAW images, just like the 0-60 or the quarter mile or the slalom. And much like those automotive measurements, the DxO mark is measured in a controlled, repeatable environment.

The power (hp) and torque (lb/ft) ratings, weight etc. are the specs provided by the manufacturer. In the camera world, these are the number of megapixels or the bits per pixel or the ISO range printed in the specifications page of your manual.

OK, so let's ret-con my original post to assume that the hp and tq curves were pulled off a dyno run (a measured, repeatable test to generate numeric values). Now, my main point is that seeing a power curve is interesting and can be useful in certain specific situations, but in no way is it going to tell you a car's lap time. The only way to get the full picture, IMO, is to run it around a circuit against its rivals and compare times (in addition to 0-60, 1/4 mi/trap, or a plethora of other subjective and objective tests auto reviewers use). Each part is not a sole decider, but rather a piece that plays a role in defining the whole. I think my personal frustration with this kind of rationale comes from hearing arguments like "car X is faster than car Y because it has more hp" which is akin to "camera X takes better pictures than camera Y because it has a higher DxO rank."

So like I said earlier, I think we mostly agree with the fact that DxO tests can provide (semi-)useful information, but is not the main factor in final image quality. I think where we differ is just how big of a part it does play. Well that, and analogies.

I agree that times around a track is more useful, but only if all cars are driven around the same track by the same driver. See my original post on this -- that's exactly what I said we would have in an ideal world -- same subject, same lighting, same photographer. And it would be the kind of photography we would care about. We don't have that in most cases.

In the absence of that ideal comparison, benchmarks provide important and useful information for making buying decisions. Heck of a lot better than comparing the number of megapixels or the advertised color depth or the maximum ISO range, which is what a lot of people do go by.

Like you said, we can agree to disagree on the importance of benchmarks. After all, Canon wouldn't spend all that money on marketing if it didn't brainwash at least some people But there can be no disagreement about what's a metaphor and what's an analogy. We are talking analogies here :greenpbl:

 

[j-dogg]

Awww my Digital Rebel XTi placed 67th

I still love him :hug:: he beat out the Leica M8 and the Canon 1Ds :thumbup:

 

[Derrel]

Okay, back to some comments about the ACTUAL SUBJECT OF THE ORIGINAL POST, the D7000 sensor benchmark results, and interpreting those results. With, if you can believe it, some actually RELEVANT introduction and analysis of the Thom Hogan line of B.S.-derailment that one particular Canon-user brought in to bolster his arguments that the benchmarks are irrelevant.

FIRST, a Thom Hogan news article from his site: 2010 Nikon News and Comments by Thom Hogan

Nikon IS a Hardware Company
Oct 29 (commentary)--I know many of you are dying to hear what I've got to say about all the new Nikon products that have appeared since I headed off into the African wild, so I'll give you a little tease:
bballD7000a.jpg
bballD7000b.jpg
The top image is a 100% view of a straight Capture NX2 conversion of a D7000 NEF taken in my test gym. ISO 3200, no noise reduction, default sharpening, all the usual parameters for my test shots. The bottom is the same file processed via ACR 6.3 Beta about as best as I could come up with using Adobe's sharpening and noise reduction controls plus a bit of other adjustment to make up for the deliberate slight underexpose I set in my tests (curiously, the camera wanted to set something close to my underexposure as its actual exposure, but that's a different story for a different day and is easily explained). Yes, there's some noise reduction smudging going on (the camera actually does slightly better in JPEGs than this).

Still, this is the best cropped sensor low light results I've seen to date (and yes, I've tested Sony and Canon cameras in this environment). Remember, we've got a lot of pixels here (16mp). For the 12mp cameras we've been seeing about this much of the scene at 100% view (image shown below is resized D7000):
bballd7000c.jpg

So you're wondering if the D7000 performs decently in low light? Yes, it does. But don't believe all the bull you see on some Internet forums: this is not D700 let alone D3s level of performance. It's quite good performance, though, enough to bring DX shooters into the low light. In particular, if I were shooting a basketball game with a D90 and D7000 side by side and displaying the resulting images at the same size, the D7000 wins hands down. It's got cleaner color, more detail, and less noise when I clean both up and print them. To refresh your memory, here's the D90 (this isn't quite an apples-to-apples comparison--remember these are frequency based lights, and there's some slightly misleading resizing going on here --but it's close enough to give you an idea of what to expect [note the loss of color in the ball, especially on the shadow side]):
bballd7000d.jpg
So, Nikon? Please leave software to others (see next story). But please continue making hardware.
-30-
end of Thom Hogan commentary.

Okay, DxO Haters...so...what can we extract from his testing of many different cameras over a period of several years in the gymnasium where he plays basketball AND tests cameras for indoor, low-light, and high-ISO performance? 1)First, the new D7000 is the best APS-C sensor Nikon, Sony, and Canon camera that he has tested in the same location using his now standard basketball rim/net/basketball test shot method. 2) While good, this new Nikon with APS-C sensor is not as good as a full-frame D700, and of course, not as good as the king of High ISO, the Nikon D3s body. and 3) compared with the Nikon D90, the D7000 wins, as he puts it, "hands down".

Okay...so, back to the DxO Mark overall sensor scores. Hogan says, "this is the best cropped sensor low light results I've seen to date." DxO Mark High ISO score for the D7000 is 1,167. The D90 has a DxO Mark High ISO score of 977. Hogan states that, "if I were shooting a basketball game with a D90 and D7000 side by side and displaying the resulting images at the same size, the D7000 wins hands down. It's got cleaner color, more detail, and less noise when I clean both up and print them." Okay, so DxO Mark's score at High ISO seems to relate DIRECTLY, as in side-by-side, same event, BETTER images from the D7000, the higher-rated camera. Hogan also states that although the D7000 is excellent for a small-sensor camera, that the full-frame D700 and D3s are better performers in low light: the DxO mark score for the D700's sensor at High ISO is a whopping 2,303. Which is over DOUBLE the score for the D7000.

So...it seems that the DxO mark scores for High ISO directly correlate to the end results that Hogan's own tests have shown. The D7000 is "hands down" better than a D90, shot side by side at ISO 3,200. But not nearly as good as a D700, or a D3s full-frame. Huh...EXACTLY what could be interpreted by looking at the DxO Mark scores. it seems that on one hand, Hogan says their single, cumulative score is worthless, but that SOME of their individual measurements are valid and useful, and in the case of High ISO tests under identical testing situations, the DxO Mark scores correlate with his empirical results.

I would postulate that the reason the D7000 beats the D90 "hands down" at ISO 3,200 in indoor basketball has more to do with the new camera's exceptionally low noise electronics and wider dynamic range potential of 13.9 EV, plus 14-bit capture in NEF mode, and higher MP count, even despite having a minutely smaller sensor area (1.53x on the D7000 vs 1.50x for the D90). As some of you know, most D-slr cameras have very good, wide Dynamic Range at their baseline ISO values of 100 or 200, but the Dynamic Range and color saturation drops off VERY badly at ISO 1600 and 3200. It's typical for a camera that might have a 12-stop Dynamic Range at ISO 100 to have somewhere in the range of 6.5 to 7.0 stops' worth of DR at ISO 3,200. As you might know, the noise goes up, and the color goes to hell, once the ISOs get into the 1,600 to 6,400 range with most all small-sensor d-slrs.

Looking here DxOMark - Compare sensors
the Nikon D700 full-frame camera's signal to noise ratio is CLEARLY evident, with a commanding, substantial and across the board superior performance to either crop-body camera. And that is what is to be expected from a full-frame sensor with pixel wells that are around TWICE as large as those found in the two crop-frame camera sensors. Imagine a Chevy 350 cubic inch V-8 engine producing over twice the horsepower of a 2.2 litre 4-cylinder engine! Whoda' thunk it, right! (snort) If a person is not smart enough to use the DxO Web Site and its multi-compare feature and its multiple measurements, graphs, and charts, it basically shows that that person is incapable of understanding scientific testing measurement concepts, and metrics. If you can balance your checkbook, or read an annual report, you're probably okay using DxO Mark results to get a feel for how various cameras perform in multiple imaging tests. If you can barely read the TV listings on-line, then stick to what friends tell you.