[LONG] Theoretical estimates for film-equivalent digital sens

This or similar topics appear quite often, but most treatments avoid
starting "from first principles". In particular, the issues of
photon Poisson noise are often mixed up with electron Poisson noise,
thus erring close to an order of magnitude. Additionally, most people
assume RGB sensors; I expect that non-RGB can give "better" color
noise parameters than (high photon loss) RGB. [While I can easily
detect such errors in calculations of others, I'm in no way a
specialist, my estimates may be flawed as well... Comments welcome.]

Initial versions of this document were discussed with Roger N Clark;
thanks for a lot of comments which lead to major rework of
calculations; however, in order of magnitude the conclusions are the
same as in the beginning of the exchange... [I do not claim his
endorsement of what I write here - though I will be honored if he does
;-]

I start with conclusions, follow with assumptions (and references
supporting them), then conclude by calculations, and consideration of
possible lenses.

CONCLUSIONS:
~~~~~~~~~~~

Theoretical minimal size of a color sensor of sensitivity 1600ISO,
(which is equivalent to Velvia 50 36x24mm in resolution and noise)
is 13mm x 8.7mm. Similar B&W sensor can be 12x8mm. Likewise,
theoretical maximum sensitivity of 3/4'' 8MP color sensor is
1227 ISO.

[All intermediate numbers are given with quite high precision; of
course, due to approximations in assumptions, very few significant
digits are trustworthy.]

These numbers assume QE=1, and non-RGB sensor (to trade non-critical
chrominance noise vs. critical luminance noise). For example, in a
2x2 matrix one can have 2 cells with "white" (visible-transparent)
filter, 1 cell with yellow (passes R+G) filter, another with cyan
(passes G+B) filter.

ASSUMPTIONS:
~~~~~~~~~~~

a) Photopic curve can be well approximated by Gaussian curve
V(lambda) = 1.019 * exp( -285.4*(lambda-0.559)^2 )
see
http://home.tiscali.se/pausch/comp/radfaq.html

b) Solar irradiation spectrum on the sea level can be well approximated
by const/lambda in the visible spectrum (at least for the purpose
of integration of photopic curve). See

http://www.jgsee.kmutt.ac.th/exell/Solar/Intensity.html
http://www.clas.ufl.edu/users/emarti...irradiance.htm

In the second one lower horizontal axis is obviously in nm, and the
upper one complete junk. Sigh...)

c) Sensitivity of the sensor is noise-bound. Thus sensitivity of
a cell of a sensor should be measured via certain noise level
at image of 18% gray at normal exposure for this sensitivity.

d) The values of noise given by Velvia 50 film and Canon 1D Mark II
at 800ISO setting at image of 18% gray are "acceptable". These
two are comparable, see
http://clarkvision.com/imagedetail/d...gnal.to.noise/
Averaging 15 and 28 correspondingly, one gets 21.5 as the "acceptable"
value of S/N in the image of 18% gray.

e) Noise of the sensor is limited by the electron noise (Poisson noise
due to discrete values of charge); other sources of noise are
negligeable (with exposition well below 40sec). See
http://www.astrosurf.com/buil/d70v10d/eval.htm

f) The AE software in digital cameras is normalizing the signal so
that the image of 100% reflective gray saturates the sensor.
[from private communication of Roger Clark; used in "d"]

g) Normal exposure for 100ISO film exposes 18% gray at 0.08 lux-sec.
See
http://www.photo.net/bboard/q-and-a-...?msg_id=004kMM

h) The color "equivalent resolution" numbers in
http://clarkvision.com/imagedetail/f...digital.1.html
may be decrease by 25% to take into account recent (as of
2005) improvements in demosaicing algorithms. E.g., see
http://www.dpreview.com/reviews/koni...200/page12.asp
Taking largest numbers (Velvia 50 again, and Tech Pan), this gives
16MP B&W sensor, and 12MP color sensor.

i) Eye is much less sensitive to the chrominance noise than to
luminance noise. Thus it makes sense to trade chrominance
noise if this improves luminance noise (up to some limits).

In particular, sensors with higher-transparency filter mask give
much lower luminance noise; the increased chrominance noise (due
to "large" elements in the to-RGB-translation matrix) does not
"spoil" the picture too much.

j) To estimate Poisson noise is very simple: to get S/N ratio K, one
needs to receive K^2 particles (electrons, or, assuming QE=1,
photons).

METAASSUMPTION
~~~~~~~~~~~~~~

In any decent photographic system the most important component
of performance/price ratio is the lenses. Since the price of the
lens scales as 4th or 5th power of its linear size, decreasing
the size of the sensor (while keeping S/N ratio) may lead to
very significant improvements of performance/price.

Details in the last section...

[This ignores completely the issue of the price of accumulated
"legacy" lenses, so is not fully applicable to professionals.]

Since sensor is purely electronic, so (more or less) subject to
Moore law, the theoretical numbers (which are currently an order
of magnitude off) have a chance to be actually relevant in not
so distant time. ;-)

PHOTON FLOW OF NORMAL EXPOSURE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, we need to recalculate 0.08 lux-sec exposure into the the
photon flow.

Assume const/lambda energy spectral density (assumption b),
integration of photonic curve gives const*0.192344740294
filtered flow. With constant spectral density at
1 photon/(sec*mkm*m^2), const = h * c, so the eye-corrected energy
flow is 3.82082403851941e-20 W/m^2 = 2.60962281830876e-17 lux.

Thus 0.08 lux-sec corresponds to (constant) spectral density
3065.57711860622 photon/(mkm*mkm^2). This is the total photon flow
of the image of 18% gray normally exposed for 100ISO film.

B&W SENSOR
~~~~~~~~~~
One can imagine (among others) 3 different theoretical types of B&W
sensor: one giving "physiologically correct" response of the
photopic curve, one accepting all photons in the "extended visible"
range of the spectrum 380 to 780nm, and an intermediate one, one
accepting all photons in the "normal visible" range of the spectrum
400 to 700nm. See
http://en.wikipedia.org/wiki/Visible_light

To cover the first case, one needs to multiply the value obtained in
the previous section by the integral of the photopic curve,
0.106910937 mkm; for the other, one needs to multiply by the width
of the window, 0.4 mkm, and 0.3 mkm. Resulting values are
327.7437, 1226.23, and 919.673 photon/mkm^2 as the flow of 18% gray
normally exposed for 100ISO film.

However, since photopic curve should not produce any particularly
spectacular artistic effect, it makes sense to have the sensor
of maximal possible sensitivity, and achieve the photopic response
(if needed) by application of a suitable on-the-lens filter. So we
ignore the first value, and use the other two. For example, the
smaller value gives photon Poisson noise S/N ratio of 21.5 with a
square cell of 0.70896 mkm. The larger value of the window,
0.4 mkm, results in a square cell of 0.613977 mkm. These are
smallest possible sizes of the cell which can provide the required
S/N ratio at exposure suitable for 100ISO film.

To have 1600ISO sensor, these numbers should be quartupled; 16MP
3:2 ratio sensor based on the 0.4mkm spectral window results in
12x8mm sensor.

OPTIMIZING THE COLOR MASK
~~~~~~~~~~~~~~~~~~~~~~~~~

For color sensor, theoretical estimates are complicated by the
following issue: different collections of spectral curves for the
filter mask can result in identical sensor signal after suitable
post-processing. (This ignores noise, and de-mosaicing artefacts.)
Indeed, taking a linear combination of the R,G,B cells is equivalent
to substituting the transparency curves for mask filters by the
corresponding linear combination. (This assumes the linear
combination curve fits between 0 and 1.)

As we saw in B&W SENSOR section, a more transparent filter results
in higher S/N at the cell; if the filter is close to transparent,
cell's signal is close to luminance, thus higher transparency
results in improvement of luminance noise.

To estimate color reproduction, take spectral sensitivity curves
of the different types of sensors cells. Ideally, 3 linear
combinations of these curves should match the spectral sensitivity
curves of cones in human eyes. Assuming 3 different types of sensor
cells, this shows that spectral curves of cells should be linear
combinations of spectral sensitivity curves of cones. In
principle, any 3 independent linear combinations can be used for
sensors curves; recalculation to RGB requires just application of
a suitable matrix. However, large matrix coefficients will result
in higher chrominance noise. (Recall that we assume that [due to
high transparency] the luminance is quite close to signals
of the sensors, thus matrix coefficents corresponding to luminance
can't be large; thus all that large matrix coefficients can do is
to give contribution to CHROMINANCE noise.)

Without knowing exactly how eye reacts to chrominance and luminance
noise it is impossible to optimize the sensor structure; however,
one particular sensor structure is "logical" enough to be close to
optimal: take 2 filters in a 2x2 filter matrix to be as transparent
as possible while remaining a linear combination of cone curves.
This particular spectral curve is natural to call the W=R+G+B curve.
Take two other filters to be as far as possible from W (and
from each other) while keeping high transparency; in particular,
keep the most powerful (in terms of photon count) G channel, and
remove one of R and B channels; this may result, for example, in
the following filter matrix

W Y W Y W Y W Y
C W C W C W C W
W Y W Y W Y W Y
C W C W C W C W

here C=G+B, Y=R+G. Since the post-processing matrix R=W-C, B=W-G,
G=C+Y-W does not have large matrix coefficients, the increase in
chrominance noise is not significant.

Above, W means the combination of the cone sensitivity curves with
maximal integral among (physically possible) combinations with
"maximal transparency" being 1. While we cannot conclude that this
results in the optimal mask, recall the following elementary fact:
to estimate the maximal *value* f(x) one can make quite large errors
in the *argument* x, and still get good approximation for f(xMAX).
Thus choosing the matrix above gives a pessimistic estimate, AND one
should expect that it is not very far of the correct one.

TRANSPARENCY OF THE COLOR MASK
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Actually, what is R, G, B in colorimetry are in turn linear
combinations of responses of cones. Use cones sensitivity curves
from
http://www.rwc.uc.edu/koehler/biophys/6d.html

Now use RR, GG, and BB to denote *these* curves, not "usual" R, G, B
of colorimetry. Since I could not find these data in table form,
the values below are not maximal possible, but just first
opportunities which come to mind.

Using 0.9RR+0.35GG, one gets a quite flat curve; one may assume that
in range 0.42--0.65nm the sensitivity is above 0.9. with one at
700nm going down to 0.6, and 400nm going down to 0.8. So the
filter "compatible" with cone sensitivity curves can easily achive
0.9 transparency in the range 400--700nm, which would give photon
count 827.705822 photon/mkm^2 in the W (R+G+B) type cell. Taking
GG and 0.9RR+0.35BB curves as other types of sensors, one gets
average transparency about 0.8 and 0.85. Taking average
transparency of the filter over a 2x2 WCWY matrix cell 0.85, one
gets photon count averaged over different kinds of color-sensitive
cells as 781.722165 photon/mkm^2.

As above, we assume that this average photon count is the count
giving contribution into luminance noise.

FINAL ESTIMATES
~~~~~~~~~~~~~~~

With above average photon count at a cell, to get S/N ratio 21.5
one needs a square cell of 0.768975 mkm. Recall that this is the
the smallest possible cell which can provide the required S/N ratio
at exposure suitable for 100ISO film.

Quadrupling to get sensitivity 1600ISO, and taking 12MP equivalent
of 36x24mm Velvia 50, one gets the 13 x 8.7 mm sensor.

HOW GOOD CAN 36x34mm SENSOR GO?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In other direction, 36x24 mm color sensor at sensitivity 1600ISO can
(theoretically) be equivalent (or better) than 10 x 6.6 cm Velvia 50
film; that is 1/2 frame of 4x5 in film. In yet other words, take
36x24mm sensor with resolution and noise better than 4x5 Velvia 50
film; it has theoretical maximum of sensibility at 800ISO.
Likewise, to achieve resolution and noise of 8x10in Velvia 50 film,
the maximal sensibility of 36x24mm sensor is 200ISO.

THE QUESTION OF LENSES
~~~~~~~~~~~~~~~~~~~~~~

Of course, preceeding section completely ignores the issue of
lenses; on the other hand, a cheap prosumer zoom lens with
28--200mm equivalent paired with a digital sensor easily gives
resolution of 3.3mkm per single line (with usable image diameter
about 11mm, see
http://www.dpreview.com/reviews/koni...200/page12.asp
); so we know it is practically possible to create a lens which
saturates the theoretical resolution of 1600ISO sensor (but probably
not 800ISO and 200ISO sensor!). It is natural to expect that a
non-zoom lens could saturate resolution of 800ISO sensor.

This gives theoretical resolution limit of a "practical" lens +
800ISO digital 36x24mm sensor: it is equivalent to best 4x5in 50ISO
film (with non-zoom lens). With zoom lense, one can achieve quality
of 2.5x4in 50ISO film; sensor is at 1600ISO, lense is 28-200mm zoom.

Some more estimates of how practical is "practical": the zoom
mentioned above is bundled with $600 street price camera which
weights about 580g. Assume the lens takes 1/2 of the price, and
1/4 of the weight. Rescaling from 11mm diagonal image size to the
36x24mm image size will increase price to $70K--$280K (assuming that
price is proportional to 4th-5th power of the size [these numbers
were applicable 20 years ago, I do not know what holds today]), and
will increase the weight to 9kg.

On the other hand, the 4:3 aspect ratio sensor of the same area as
the mentioned above 13 x 8.6 mm sensor (1600ISO sensor equivalent
in quality to Velvia 50 at 36x24mm) is 12.2 x 9.17mm, diagonal is
15.26mm. It is 0.9'' sensor (in the current - silly - notation).

Rescaling the mentioned above lens to this size gives lens price
$1100--$1500, and weight about 750g; both quite "reasonable".
Recall that this 28--200 equivalent zoom lens will saturates resolution
of an equivalent of Velvia 50 36x24mm film.

A very nice write up, I will admit I have not gone through all of it
yet in detail. One thing to consider is that CCD have a read out noise
of around 10 electrons, whereas this noise level will not greatly
effect the signal to noise when looking at 400 detected photons with an
noise level of 20 electrons it will start to dominate in darker parts
of the scene. For instance by the time you are down 5 stops from full
white the readout noise will be larger then the photon noise, by a
small amount.

The idea of using non-RGB filters is sound and a number of CCD sensors
have used filters more like C, Y and M. Why RGB is used on digital
cameras I am not sure.

Scott

[A complimentary Cc of this posting was sent to
Scott W
], who wrote in article .com:
A very nice write up, I will admit I have not gone through all of it
yet in detail. One thing to consider is that CCD have a read out noise
of around 10 electrons, whereas this noise level will not greatly
effect the signal to noise when looking at 400 detected photons with an
noise level of 20 electrons it will start to dominate in darker parts
of the scene. For instance by the time you are down 5 stops from full
white the readout noise will be larger then the photon noise, by a
small amount.

This is a very valid remark. However, note that these were
*theoretical* estimates; after translation into this language your
remark becomes:

Readout noise should be decreased too; otherwise shadows noise is
going to be well above Poisson noise.

Thanks,
Ilya

Scott W wrote:
The idea of using non-RGB filters is sound and a number of CCD sensors
have used filters more like C, Y and M. Why RGB is used on digital
cameras I am not sure.

My guess is that to do otherwise would increase the chroma noise too
much. Chroma noise in digital cameras at high ISO is already
intrusive, and anything that increases it may be unwelcome, even if
sensitivity improved. Without direct experimental data it's hard to
say.

The other issue is how well non-RGB filters could be made to
approximate the colour matching functions of typical display systems.
Red and green are quite well matched by sensors of a typical camera,
but the blue is quite a way off because its spectral sensitivity is
too broad.[1] It would be a matter of measuring some physically
realizable filters and seeing what colour matching functions resulted.

Andrew.

[1] The Reproduction of Colour, 6th Edition, Robert Hunt, p556.

Ilya Zakharevich wrote:

PHOTON FLOW OF NORMAL EXPOSURE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, we need to recalculate 0.08 lux-sec exposure into the the
photon flow.

Assume const/lambda energy spectral density (assumption b),
integration of photonic curve gives const*0.192344740294
filtered flow. With constant spectral density at
1 photon/(sec*mkm*m^2), const = h * c, so the eye-corrected energy
flow is 3.82082403851941e-20 W/m^2 = 2.60962281830876e-17 lux.
What unit is 'mkm', wavenumber?
-- Hans

[A complimentary Cc of this posting was sent to

], who wrote in article :
My guess is that to do otherwise would increase the chroma noise too
much. Chroma noise in digital cameras at high ISO is already
intrusive, and anything that increases it may be unwelcome, even if
sensitivity improved. Without direct experimental data it's hard to
say.

When I look on the digital images of a gray surface (those of "compare
two cameras" kind), it looks like my perception of noise is not
related to chrominance noise at all. At least a camera with higher
measured individual-channel R/G/B noise can produce much lower visible
noise if its noise reduction algorithm favors luminance noise (as
confirmed by luminance noise graph). Of course, it is in no way
scientific conclusion, but I may have seen about ten such
comparisons...

The other issue is how well non-RGB filters could be made to
approximate the colour matching functions of typical display
systems.

AFAIU, this has nothing to do with display (output) system, but only
with input system (cones). As far as the filters match cones, you can
postprocess colors into *any* display system (if the initial color is
in the gamut of the display system).

And if you do not match the cone sensitivity, colors which look the
same will get different when stored. After this no amount of
post-processing will be able to fix this.

Hope this helps,
Ilya

[A complimentary Cc of this posting was sent to
HvdV
], who wrote in article :
filtered flow. With constant spectral density at
1 photon/(sec*mkm*m^2), const = h * c, so the eye-corrected energy
flow is 3.82082403851941e-20 W/m^2 = 2.60962281830876e-17 lux.
What unit is 'mkm', wavenumber?

Yes (?); Wavelength. IIRC, wavenumber is 1/wavelength (or some such;
2pi comes to mind...).

[BTW, because of non-linearity of wavelength vs wavenumber, spectral
density which constant per wavelength becomes very non-constant when
measured per wavenumber.]

Yours,
Ilya

lid writes:

[1] The Reproduction of Colour, 6th Edition, Robert Hunt, p556.

I didn't realize that the 6th edition was out. Does it say how it
differs from the previous edition (e.g. in a preface?).

I do have the 3rd, 4th, and 5th editions already, but they don't cover
digital imaging much.

Dave

Hi Ilya,


Yes (?); Wavelength. IIRC, wavenumber is 1/wavelength (or some such;
2pi comes to mind...).
Yes, wavenumber is 2 * pi / lambda, units m^-1

[BTW, because of non-linearity of wavelength vs wavenumber, spectral
density which constant per wavelength becomes very non-constant when
measured per wavenumber.]
Yes, but the choice is arbitrary. Since wavenumber which is proportional to
photon energy, an interesting quantity for many applications, spectroscopy
people tend towards wavenumber whereas optical people like wavelength since
resolving power scales with that.

BTW, can you substantiate your interesting assumption:
----
In any decent photographic system the most important component
of performance/price ratio is the lenses. Since the price of the
lens scales as 4th or 5th power of its linear size, decreasing
the size of the sensor (while keeping S/N ratio) may lead to
very significant improvements of performance/price.
---
with some examples?
The tradeoff of lens aperture and expense vs sensor size determines
ultimately the size and shape of the digital camera. After the 'fashion
factor' of course.

-- hans

[A complimentary Cc of this posting was sent to

], who wrote in article :
Scott W wrote:
The idea of using non-RGB filters is sound and a number of CCD sensors
have used filters more like C, Y and M. Why RGB is used on digital
cameras I am not sure.

My guess is that to do otherwise would increase the chroma noise too
much. Chroma noise in digital cameras at high ISO is already
intrusive, and anything that increases it may be unwelcome, even if
sensitivity improved. Without direct experimental data it's hard to
say.

Judge for yourself: visit

http://ilyaz.org/photo/random-noise

Yours,
Ilya