Dynamic Range of RAW digital sensor data

Gisle Hannemyr wrote:

I've checked my library and searched the web, but to no avail.
Do anyone know about a good source for this type of data?

Did you look at Roger Clarke's site? I seem to recall it is quite
detailed in this regard, and may include the specifics you're looking for.


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.

Gisle Hannemyr wrote:
Alan Browne writes:
Gisle Hannemyr wrote:

I've checked my library and searched the web, but to no avail.
Do anyone know about a good source for this type of data?

Did you look at Roger Clarke's site? I seem to recall it is quite
detailed in this regard, and may include the specifics you're
looking for.

Yes, for instance "Dynamic Range and Transfer Functions of Digital
Images and Comparison to Film":
http://www.clarkvision.com/imagedetail/dynamicrange2/
- and a number of related pages linked to from that page,

The title is of course promising, but unless I am missing something,
the data he reports for both scene and output intensity are
/data numbers/ (i.e. the numeric values of the pixels in his
image files) before and after RAW conversion, which AFAIK don't reveal
actual scene luminosity.
Roger uses linear data, I don't know which converter he is using to do
this but I do know he is looking at
linear data for each channel before it has been de-mosaiced

Scott

Gisle Hannemyr wrote:
[]
I can't see how this translates into real life dynamic range without
knowing what 100 DN means in terms of light, or - at least - the
native gamma of the sensor he pulls these data from.

Gisle,

The sensor is linear - gamma = 1

There may be some black level offset, though, so the best-fit equation is
likely to be y = Ax + B, where B is quite small. Of course, the lens
aperture, transmission, light spectrum etc. would all come into working
out light-levels from DN.

Cheers,
David

Gisle Hannemyr wrote:

Yes, for instance "Dynamic Range and Transfer Functions of Digital
Images and Comparison to Film":
http://www.clarkvision.com/imagedetail/dynamicrange2/

Mr. Clark is using the same totally incorrect dynamic range definition
that sensor manufactures use in their marketing. Ignoring the photon
shot noise totally. He claims:

-- "Further image analysis shows at least 10.6 stops are recorded
-- by the canon 1D Mark II camera (the full range of of detail in
-- this image, Other testing of the noise level versus intensity
-- shows the Canon 1D Mark II has 11.7 stops of dynamic range.

In order to have a photographically acquired image that truly holds 11.7
stops scene dynamic range one has to have full well capacity of 11
million electrons due to the photon shot noise alone. That in case of an
ideal sensor, some electrons more in order to overcome the noises of a
real sensor.

11.7 stops == 2^11.7 == 3327:1 in linear quantity. 3327^2 == 11068929
electrons or 11.07 million electrons full well.

And 10.6 stops == 2^10.6 == 1552:1 in linear quantity, 1551^2 ==
2408704 electrons or 2.4 million electrons full well.

These "results" are _enormously_ incorrect, because of the incorrect
definition of the dynamic range.

Also, the definition of dynamic range goes down to S/N ratio of 1:1 or
1/sqr(1). The image information at signal level of 1 electron is not
usable at all,, the S/N ratio of 1:1 just happens to be part of the
definition of dynamic range. In case N electrons are required for
acceptable S/N level then sqr(N) must be subtracted from the DR that is
expressed in stops. Assuming ideal sensor, more in case of real sensor.

BTW: For the material on the above linked page Mr. Clark was using
Polaroid SprintScan 4000 scanner, that has way lesser dynamic range than
any film.

Timo Autiokari

Gisle Hannemyr wrote:
[]
I just can't make sense of his DN numbers.

If we look at fig. 8a, in "Dynamic Range and Transfer Functions of
Digital Images and Comparison to Film":
[ http://www.clarkvision.com/imagedetail/dynamicrange2/ ]
he says that "the digital camera keeps going to the bottom
end of data below 70 DN", and then refers to his "black hole
in the scene measured at 19 DN,

Well, the data in the black hole is obviously consisting of noise only
(black current noise?) so the noise floor must lie above that. From
the scatter plot in fig. 8a, it looks like the his data for the
1DII stretches from 70 DN to 70000 DN on the logarithmic scene scale.
I.e. he have a scatter plot showing a 1000:1 linear ratio. This
is about equal to 10 EV (or stops). But in the text, he claims that
this data demonstrates a 11.7 stop dynamic range!

He is obviously interpreting the data different than me, but how
he interprets them is beyond me.

I find the fig. 8a graph somewhat confusing, particularly (a) having "the
human eye" response on there and (b) missing the major grid lines for
scene intensity. I want to be able to compare the digital camera to a
straight-line through the origin, and figure 8a doesn't easily allow this.

It does show that my "linear" response was incorrect, because of some
deliberate "smooth clipping" at the higher end to allow for a higher
dynamic range. It's unclear where this is happening - it must be in the
RAW to Image conversion. So it's a function of the software.

Two points of detail:

- He doesn't say the black-hole is at 19 DN, but that the noise measured
19 DN.

- He doesn't say that the data he shows here demonstrated 11.7 stops
range, but that "Other testing of the noise level versus intensity shows
the Canon 1D Mark II has 11.7 stops of dynamic range."

There is plenty of scope for explaining all this in a number of different
ways, as some will approach if from film photography, some from the
physics, some from digital signal processing, and others from a human
vision characterisation!

Cheers,
David

Timo Autiokari wrote:

Gisle Hannemyr wrote:

Yes, for instance "Dynamic Range and Transfer Functions of Digital
Images and Comparison to Film":
http://www.clarkvision.com/imagedetail/dynamicrange2/

Mr. Clark is using the same totally incorrect dynamic range definition
that sensor manufactures use in their marketing.

Clark is correct as per standard engineering practice. Any "marketing"
information derived from such is also correct.

Ignoring the photon shot noise totally.

Yes. He also ignored the phase of the Moon as well. And why not? It
is not relevant. The DR is a measure of how the system responds to its
input, from the minimum discernable signal up until compression
(however defined).

11.7 stops == 2^11.7 == 3327:1 in linear quantity. 3327^2 == 11068929
electrons or 11.07 million electrons full well.

Ignoring the incorrect mathematics, your progression from a
dimensionless number to a physical unit is most hilarious...

Gisle Hannemyr wrote:

But in the text, he claims that this data demonstrates a 11.7 stop dynamic range!

http://www.clarkvision.com/imagedetail/evaluation-1d2/ has a table that
shows 11.6 stops at ISO 100. The read-noise at this ISO is 13
electrons; the "minimum discernable signal" is subject to definition,
and it appears Clark has defined it to 1 stddev over that distribution:
13 + sqrt(13) == 16 electrons. The 1D2, like all cameras worth
owning, is virtually linear all the way until the pixel is full of
electrons, some 53000 in this case. So the DR is then simple enough:
log(compressed_signal/minimum_signal)/log(2) == log(53000/16)/log(2) ==
11.7 stops.

wrote:

The DR is a measure of how the system responds to its
input, from the minimum discernable signal up until
compression

The above is almost correct, should be "...up until clipping".

The way the DR is most often expressed by the manufactures of imaging
sensor (the same way that also Mr. Clark define it) is _not_ according
to your own above statement.

The full well capacity divided by the sensor noises is not at all the
same as "how the system responds to its input". The input is light.

This "DR" (the full well capacity divided by the sensor noises) has no
direct relation with the f/stop range of the scene that the camera is
able to capture. The only thing that can be derived from such "DR" value
is that the true dynamic range of the system is always _much_ lesser.

He also ignored the phase of the Moon as well. And why not?
It is not relevant.

It would be very beneficial for you to study the issue a little. Just
google "photon shot noise" gets you started very well.

Timo Autiokari

Timo Autiokari wrote:

wrote:

The DR is a measure of how the system responds to its
input, from the minimum discernable signal up until
compression

The above is almost correct, should be "...up until clipping".

Serves me right to assume you are capable of abstraction.

In most systems, there is a fair amount of gain compression prior to
hitting the power supply rails ("clipping"). Optical sensors are
something of an exception to this rule: they are staunchly linear
until the pixel saturates. Not having designed any myself, I'll
speculate that there probably isn't enough input signal even at pixel
saturation to drive the following electronics into a significant amount
of compression, so practically speaking the clip-point is the
compression point of interest here.

The way the DR is most often expressed by the manufactures of imaging
sensor (the same way that also Mr. Clark define it) is _not_ according
to your own above statement.

With exactly one exception -- specifically, you -- I haven't seen any
use of the word "dynamic range" elsewhere that is substantially
different.

This "DR" (the full well capacity divided by the sensor noises) has no
direct relation with the f/stop range of the scene that the camera is
able to capture.

DR(camera) = DR(scene) - noise_figure

Can't be more direct that that. The "noise figure" (go ahead, google
that up too) of the Canon 1D2 at ISO 100 appears to be about 2 stops
(assuming a quantum efficiency of ~1/4 and an MDS of 1 stddev over
read-noise).

The only thing that can be derived from such "DR" value
is that the true dynamic range of the system is always _much_ lesser.

Oh dear, ~14 stops to 11.7 stops.

He also ignored the phase of the Moon as well. And why not?
It is not relevant.

It would be very beneficial for you to study the issue a little. Just
google "photon shot noise" gets you started very well.

You desperately need a course in basic signal processing -- audio, RF,
doesn't matter much: the concepts apply across the board. This stuff
isn't even particularly hard. Certainly not has hard as explaining how
you can start with a pure number and arrive at electrons...

Timo Autiokari wrote:

Gisle Hannemyr wrote:

Yes, for instance "Dynamic Range and Transfer Functions of Digital
Images and Comparison to Film":
http://www.clarkvision.com/imagedetail/dynamicrange2/

Mr. Clark is using the same totally incorrect dynamic range definition
that sensor manufactures use in their marketing. Ignoring the photon
shot noise totally. He claims:

Timo,
You are way off base here. It is more than just marketing
by sensor manufacturers, it is the standard used in
engineering. (I would have responded sooner but I was in Africa
the last couple of weeks.) I'll give the following link:
http://www.clarkvision.com/imagedeta...rmance.summary
then go to the bottom of the page to the references and
download the Kodak sensor data sheets (those marked KAF).
You'll see the same methods and definitions used that I use.
On the same above web page, I've plotted the Kodak data along
with my data and those of others who have studies sensors.
We have a collectively consistent picture.

-- "Further image analysis shows at least 10.6 stops are recorded
-- by the canon 1D Mark II camera (the full range of of detail in
-- this image, Other testing of the noise level versus intensity
-- shows the Canon 1D Mark II has 11.7 stops of dynamic range.

In order to have a photographically acquired image that truly holds 11.7
stops scene dynamic range one has to have full well capacity of 11
million electrons due to the photon shot noise alone. That in case of an
ideal sensor, some electrons more in order to overcome the noises of a
real sensor.

11.7 stops == 2^11.7 == 3327:1 in linear quantity. 3327^2 == 11068929
electrons or 11.07 million electrons full well.

And 10.6 stops == 2^10.6 == 1552:1 in linear quantity, 1551^2 ==
2408704 electrons or 2.4 million electrons full well.

You are confusing precision in intensity measurements with range
of measurements. Let's try another analogy: estimating length with
your eye. For a small length, like a mm, you might have an error
of a fraction of a mm (let's say 0.25 mm error at length 0.5 mm).
At 1 meter, you may have an error of a few cm. At 100 meters, you
might have an error of a few meters. Over 100 meters, what
dynamic range in length measurements do you have? It is on the
order of: 100 meters / 0.5 mm = 100 /0.0005 = 20,000.
The range of measurement is 20,000 even though the precision
is not high.

You don't need that 0.25 mm accuracy to know that 100 meters
is big. Same with photons. The sensor can measure 50,000
photons (electrons) with an error of sqrt(50,000) =223, but you
know it is still a big number regardless of the error bar.
Then at small numbers of photons, e.g. 4, the noise is sqrt(4)=2.
The range of such a measurement is 50,000/4 = 12,500, even though
the signal-to-noise ratio never exceeds 223.

So, don't confuse signal-to-noise ratio with dynamic range.


These "results" are _enormously_ incorrect, because of the incorrect
definition of the dynamic range.

wrong. Your definition of dynamic range is incorrect. I use
electronics industry standard definitions.

Also, the definition of dynamic range goes down to S/N ratio of 1:1 or
1/sqr(1). The image information at signal level of 1 electron is not
usable at all,, the S/N ratio of 1:1 just happens to be part of the
definition of dynamic range. In case N electrons are required for
acceptable S/N level then sqr(N) must be subtracted from the DR that is
expressed in stops. Assuming ideal sensor, more in case of real sensor.

We've been over this before, yet you have never shown any
data that proves your point and you ignore data that proves
you are wrong. See Figure 5 and the paragraph above Figure 5 at:
http://www.clarkvision.com/photoinfo...ht.photography
where it is shown and discussed how less than a fraction of the
noise still shows image detail. For example, Figure 5
set 5, patch A has 1.2 photons is clearly discernible (electrons)/pixel
on average, with noise of 3.9 electrons, or over 3 times smaller
than the noise. It is the same with high ISO film: the grain
is quite large and a lot of image components are at a S/N
less than 1. You have yet to respond with a definition of
acceptable S/N ratio that will include film as a usable
medium (and note it was acceptable for decades).

BTW: For the material on the above linked page Mr. Clark was using
Polaroid SprintScan 4000 scanner, that has way lesser dynamic range than
any film.

Wrong. (Ironic, as your own definition of dynamic range says film
effectively has no dynamic range!) To the contrary, the sprintscan
has pulled data out of images that I nor professional labs could
not print. It has more than adequate dynamic range for the
task.

Roger