Image Compression Benchmark

I am planning to publish an Image compression benchmark, comparing
various standard and research algorithms. But moving forward, I need
some help from the photographers here (apologies for this relatively
long post).

Imagine someone working on next generation Jpeg replacement but using
512x512 pixel images, with 8-bits-per-channel while developing/
evaluating algorithms. I think it will sound a bit absurd to raw
shooters here but that is almost the case. The compression
researchers use a standard set of images and those images are now all
VERY old.

Check this out, http://en.wikipedia.org/wiki/Standard_test_image

Some researchers have now started using their own set of images to
solve this but the need for common images is very critical. Else it
becomes impossible to compare results across research done by
different people. (For example, if A says his algorithm achieves a
loss of X at bit-rate Y on his image, and B tells similar figures for
the image he used, then for someone reading both research-papers those
numbers don't mean anything. Unless both A and B used 'same' image.)

I am trying to fill this gap by publishing a set of images which come
from recent cameras and also cover all the stuff which matters to
compression people.

But for these images to be useful for a benchmark, they should
'probably' also cover what matters to YOU all, the photographers :-)

I have put up for review the Jpeg'ed version of images I have picked
till now,
http://www.sachingarg.com/temp
http://www.sachingarg.com/temp/thumbs.jpg
http://www.sachingarg.com/temp/histograms.jpg

(The full set of 16-bit uncompressed images goes to more than a GB in
size, so only Jpegs available for now. The final set will be freely
available for download. All images were made from raw images from a
wide range of cameras with minimal post-processing.)

It will be great to have your opinion on these images. Do you think
they cover everything which you as a photographer would like to see in
a benchmark comparing compression algorithms? What all is missing?

Any other thoughts on stuff you would like to see (or not see) in the
benchmark?

Sachin Garg [India]
www.sachingarg.com | www.c10n.info

ps, a more technical and compression focussed discussion on this is on
at comp.compression since a few weeks. You can check that out too,
comments welcome everywhere :-)

http://groups.google.com/group/comp....923e0a460a5926

Sachin Garg wrote:

I have put up for review the Jpeg'ed version of images I have picked
till now,
http://www.sachingarg.com/temp/thumbs.jpg

I applaud your effort. Probably comparisons should use 8-10 Mp images
so yours are large enough. You need to have more pictures of people.
The whitewater image, which was the only one I looked at full size,
has an out-of-focus background. Setting hyperfocal distance (short of
infinity) would yield a more useful test.

Currently we have JPEG 2000 and Microsoft whatever it's called
as possible replacements for old JPEG. Good luck!

On Dec 8, 10:03 pm, Bill Tuthill wrote:
Sachin Garg wrote:

I have put up for review the Jpeg'ed version of images I have picked
till now,
http://www.sachingarg.com/temp/thumbs.jpg

I applaud your effort. Probably comparisons should use 8-10 Mp images
so yours are large enough.

Most of my images as in 6 MP range (few are bigger, few MUCH bigger (a
39MP and a 24 MP)), I will try to find bigger alternates with
interesting content.

You need to have more pictures of people.

There are legal issues with images with people in them. It will be
great if I can find 'raw' shots of some celebrity faces but they will
need to be available in a free enough license (people should be able
to copy, use, republish them for research purposes).

The whitewater image, which was the only one I looked at full size,
has an out-of-focus background. Setting hyperfocal distance (short of
infinity) would yield a more useful test.

Thanks, didn't noticed that.

Currently we have JPEG 2000 and Microsoft whatever it's called
as possible replacements for old JPEG. Good luck!

There is another AIC (advanced image coding) open-call in jpeg, but I
don't know much details about it.

Sachin Garg [India]
www.sachingarg.com | www.c10n.info