CoolPix 950, FAT16 problem with large card

I just encountered a problem with a Nikon CoolPix 950 which caused
inconvenience and might have resulted in a loss of images from the CF
card. As a similar problem might exist with other camera models,
other people might want to be aware of the issue.

I just bought the 950 used. It came with the original 8MB Compact
Flash card... much too small to be useful. I stopped by an
electronics store and bought the smallest-capacity CF card they had in
stock... 1 GB.

Popped it in the camera, let the camera format it, and started taking
photos. All seemed to be well. After a few days, I pulled the card,
popped it into a CF/PCMCIA adapter, and inserted the adapter into my
laptop's card slot.

Voila... nothing happened. The icon didn't appear. No files available.

I dug around a bit, and found that the Linux kernel's support module
for the FAT/VFAT filesystem objected to the filesystem on the card.
The partition table was valid, there was a single FAT16 partition on
the card, but the filesystem on that partition was unmountable. The
kernel reported that the filesystem had almost a quarter-of-a-million
clusters, which is more than the limit allowed for a FAT16 filesystem.

I tried formatting the partition using the linux "mkfs.vfat" program.
The filesystem formatted OK, the filesystem-test program "fsck.vfat"
said that it was OK... but when I put the card back into the camera,
the camera insisted that the card was unformatted. When I allowed the
camera to reformat the card again, the same problem resulted.

So... Linux wouldn't read the camera's filesystem, and the camera
wouldn't read the Linux-constructed filesystem.

This problem didn't exist for the 8MB card which came with the 950,
nor with a 128 MB card which came with a CoolPix 800 I'd bought
earlier.

Further testing showed why... the Linux utility was (silently)
deciding to build a FAT32 filesystem on the 1-gig card due to the
large number of sectors, and the CoolPix 950 can't read FAT32.

So, I used "mkfs.vfat -f 16" on the Linux system, forcing the system
to format the card with a FAT16 filesystem. The Linux utility did so,
simply increasing the size of the clusters so that the number of
clusters didn't exceed 2^16.

Problem solved, I believe. The CoolPix 950 happily accepted the card
with this FAT16 partition without requiring a reformat, built the dcim
directory hierarchy, and has allowed me to take a bunch of pictures.
Linux can mount the filesystem and read the files, and the Linux
filesystem-check program declares that the filesystem is clean (no
internal inconsistencies).

I suspect that the original formatting of the card, by the CoolPix
950, was a case of data corruption waiting to happen. The camera
appears to have built an "illegal" filesystem... one which had cluster
numbers that couldn't be stored in 16 bits. I'd guess that the
camera's firmware wasn't written to handle such large cards, and
doesn't fail gracefully when faced by them.

If I'd stored enough images on the card, there's a fair chance that
some of the images (old or new) would become corrupt or unreadable...
or the card would have "run out of space" when it was only about 25%
full.

Fortunately, the camera firmware seems able to handle FAT16
filesystems with larger cluster sizes. I'll have to try filling it up
with big TIFF images and make sure that everything still works OK when
the card has lots of data on it.

--
Dave Platt AE6EO
Friends of Jade Warrior home page: http://www.radagast.org/jade-warrior
I do _not_ wish to receive unsolicited commercial email, and I will
boycott any company which has the gall to send me such ads!

On Sun, 13 Jan 2008 21:58:46 -0800, (Dave Platt)
wrote in :

I just encountered a problem with a Nikon CoolPix 950 which caused
inconvenience and might have resulted in a loss of images from the CF
card. As a similar problem might exist with other camera models,
other people might want to be aware of the issue.

I just bought the 950 used. It came with the original 8MB Compact
Flash card... much too small to be useful. I stopped by an
electronics store and bought the smallest-capacity CF card they had in
stock... 1 GB.

Popped it in the camera, let the camera format it, and started taking
photos. All seemed to be well. After a few days, I pulled the card,
popped it into a CF/PCMCIA adapter, and inserted the adapter into my
laptop's card slot.

Voila... nothing happened. The icon didn't appear. No files available.

I dug around a bit, and found that the Linux kernel's support module
for the FAT/VFAT filesystem objected to the filesystem on the card.
The partition table was valid, there was a single FAT16 partition on
the card, but the filesystem on that partition was unmountable. The
kernel reported that the filesystem had almost a quarter-of-a-million
clusters, which is more than the limit allowed for a FAT16 filesystem.


http://en.wikipedia.org/wiki/File_Allocation_Table#Final_FAT16

Finally in November 1987, Compaq DOS 3.31 introduced what is today
called the FAT16 format, with the expansion of the 16-bit disk sector
index to 32 bits. The result was initially called the DOS 3.31 Large
File System. Although the on-disk changes were apparently minor, the
entire DOS disk code had to be converted to use 32-bit sector
numbers, a task complicated by the fact that it was written in 16-bit
assembly language.

In 1988 the improvement became more generally available through
MS-DOS 4.0 and OS/2 1.1. The limit on partition size was now dictated
by the 8-bit signed count of sectors-per-cluster, which had a maximum
power-of-two value of 64. With the usual hard disk sector size of 512
bytes, this gives 32 KB clusters, thereby fixing the "definitive"
limit for the FAT16 partition size at 2 gibibytes. On magneto-optical
media, which can have 1 or 2 KiB sectors, the limit is proportionally
greater.

Much later, Windows NT increased the maximum cluster size to 64 KB by
considering the sectors-per-cluster count as unsigned. However, the
resulting format was not compatible with any other FAT implementation
of the time, and it generated massive internal fragmentation. Windows
98 also supported reading and writing this variant, but its disk
utilities didn't work with it.

The number of root directory entries available is set at formatting
time, and is stored in a 16 bit signed field setting an absolute
limit of 32767 entries (32736, a multiple of 32, in practice). For
historical reasons, FAT12 and FAT16 media generally use 512 root
directory entries on non-floppy media, and other sizes may be
incompatible with some software or devices (entries being file and/or
folder names in the old 8.3 format).[4] Some third party tools like
mkdosfs allow the user to set this parameter.[5]


The apparent problem is thus your Linux system, not the camera or the
card.

--
Best regards,
John Navas
Panasonic DMC-FZ8 (and several others)

In article ,
John Navas wrote:

http://en.wikipedia.org/wiki/File_Allocation_Table#Final_FAT16

#snip#

The apparent problem is thus your Linux system, not the camera or the
card.

I believe you're incorrect, John.

FAT16 has an ultimate limit of 2 GB, which is set by the maximum
number of clusters, multiplied by the maximum cluster size. As the
entry you cited pointed out,

The limit on partition size was now dictated by the 8-bit signed count
of sectors-per-cluster, which had a maximum power-of-two value of 64.
With the usual hard disk sector size of 512 bytes, this gives 32 KB
clusters, thereby fixing the "definitive" limit for the FAT16
partition size at 2 gibibytes.

2 gibibytes is, indeed, 65536 sectors times 32768 bytes per sector...
and according to the Wikipedia article, the limit in FAT16 is actually
only 65517 clusters.

However, you can only reach the 2-gib limit if you use 32k-byte
sectors. If you use fewer sectors per cluster, the 65517-cluster
limit still applies and the maximum size of the filesystem is reduced.

The problem with the Coolpix 950 seems to be that its filesystem
formatter does *not* use 32k-byte clusters. It formats the filesystem
using fewer sectors per cluster... enough fewer that the required
number of clusters exceeds 65517. It shouldn'ta done that, I think.
The cluster size used by the camera allows support of a 128MB flash,
and I think it would allow a 256MB flash, but it won't go beyond that.

While the "Final FAT16" format does increase the number of sectors
allowed on the disk to a 32-bit number, it didn't increase the cluster
index fields beyond 16 bits. That was left for the later FAT32
filesystem formats.

--
Dave Platt AE6EO
Friends of Jade Warrior home page: http://www.radagast.org/jade-warrior
I do _not_ wish to receive unsolicited commercial email, and I will
boycott any company which has the gall to send me such ads!

On Sun, 13 Jan 2008 23:34:34 -0800, (Dave Platt) wrote:

The problem with the Coolpix 950 seems to be that its filesystem
formatter does *not* use 32k-byte clusters. It formats the filesystem
using fewer sectors per cluster...

I've been taking digital pictures for more than eight years. I've never
formatted a CF card. They come formatted.

Don www.donwiss.com (e-mail link at home page bottom).

On Sun, 13 Jan 2008 21:58:46 -0800, Dave Platt wrote:

I just encountered a problem with a Nikon CoolPix 950 which caused
inconvenience and might have resulted in a loss of images from the CF
card. As a similar problem might exist with other camera models,
other people might want to be aware of the issue.

I just bought the 950 used. It came with the original 8MB Compact
Flash card... much too small to be useful. I stopped by an
electronics store and bought the smallest-capacity CF card they had in
stock... 1 GB.

Popped it in the camera, let the camera format it, and started taking
photos. All seemed to be well. After a few days, I pulled the card,
popped it into a CF/PCMCIA adapter, and inserted the adapter into my
laptop's card slot.

Voila... nothing happened. The icon didn't appear. No files available.

I dug around a bit, and found that the Linux kernel's support module
for the FAT/VFAT filesystem objected to the filesystem on the card.
The partition table was valid, there was a single FAT16 partition on
the card, but the filesystem on that partition was unmountable. The
kernel reported that the filesystem had almost a quarter-of-a-million
clusters, which is more than the limit allowed for a FAT16 filesystem.

I tried formatting the partition using the linux "mkfs.vfat" program.
The filesystem formatted OK, the filesystem-test program "fsck.vfat"
said that it was OK... but when I put the card back into the camera,
the camera insisted that the card was unformatted. When I allowed the
camera to reformat the card again, the same problem resulted.

So... Linux wouldn't read the camera's filesystem, and the camera
wouldn't read the Linux-constructed filesystem.

This problem didn't exist for the 8MB card which came with the 950,
nor with a 128 MB card which came with a CoolPix 800 I'd bought
earlier.

Further testing showed why... the Linux utility was (silently)
deciding to build a FAT32 filesystem on the 1-gig card due to the
large number of sectors, and the CoolPix 950 can't read FAT32.

So, I used "mkfs.vfat -f 16" on the Linux system, forcing the system
to format the card with a FAT16 filesystem. The Linux utility did so,
simply increasing the size of the clusters so that the number of
clusters didn't exceed 2^16.

Problem solved, I believe. The CoolPix 950 happily accepted the card
with this FAT16 partition without requiring a reformat, built the dcim
directory hierarchy, and has allowed me to take a bunch of pictures.
Linux can mount the filesystem and read the files, and the Linux
filesystem-check program declares that the filesystem is clean (no
internal inconsistencies).

I suspect that the original formatting of the card, by the CoolPix
950, was a case of data corruption waiting to happen. The camera
appears to have built an "illegal" filesystem... one which had cluster
numbers that couldn't be stored in 16 bits. I'd guess that the
camera's firmware wasn't written to handle such large cards, and
doesn't fail gracefully when faced by them.

If I'd stored enough images on the card, there's a fair chance that
some of the images (old or new) would become corrupt or unreadable...
or the card would have "run out of space" when it was only about 25%
full.

Fortunately, the camera firmware seems able to handle FAT16
filesystems with larger cluster sizes. I'll have to try filling it up
with big TIFF images and make sure that everything still works OK when
the card has lots of data on it.

Rather strange. I've used quite a variety of CF and SD cards from 8mb to
2gb on several Linux systems and never had an issue. What distro are you
running?

On Sun, 13 Jan 2008 23:34:34 -0800, Dave Platt wrote:

In article ,
John Navas wrote:

http://en.wikipedia.org/wiki/File_Allocation_Table#Final_FAT16

#snip#

The apparent problem is thus your Linux system, not the camera or the
card.

I believe you're incorrect, John.

FAT16 has an ultimate limit of 2 GB, which is set by the maximum
number of clusters, multiplied by the maximum cluster size. As the
entry you cited pointed out,

The limit on partition size was now dictated by the 8-bit signed count
of sectors-per-cluster, which had a maximum power-of-two value of 64.
With the usual hard disk sector size of 512 bytes, this gives 32 KB
clusters, thereby fixing the "definitive" limit for the FAT16
partition size at 2 gibibytes.

2 gibibytes is, indeed, 65536 sectors times 32768 bytes per sector...
and according to the Wikipedia article, the limit in FAT16 is actually
only 65517 clusters.

As I understand it, FAT16 has a maximum theoretical limit of 4gb though
virtually all implementations have a practical limit of 2gb.



However, you can only reach the 2-gib limit if you use 32k-byte
sectors. If you use fewer sectors per cluster, the 65517-cluster
limit still applies and the maximum size of the filesystem is reduced.

The problem with the Coolpix 950 seems to be that its filesystem
formatter does *not* use 32k-byte clusters. It formats the filesystem
using fewer sectors per cluster... enough fewer that the required
number of clusters exceeds 65517. It shouldn'ta done that, I think.
The cluster size used by the camera allows support of a 128MB flash,
and I think it would allow a 256MB flash, but it won't go beyond that.

While the "Final FAT16" format does increase the number of sectors
allowed on the disk to a 32-bit number, it didn't increase the cluster
index fields beyond 16 bits. That was left for the later FAT32
filesystem formats.

Note: 2**32 = 4gb. Most implementations seem to want to avoid the full use
of unsigned integers.

In article ,
ray wrote:

Rather strange. I've used quite a variety of CF and SD cards from 8mb to
2gb on several Linux systems and never had an issue. What distro are you
running?

Debian (testing/unstable), with a 2.6.23 kernel.

My guess at this point is that the CoolPix 950 firmware has a hard-coded
4k-byte cluster size coded in its FAT16 formatting logic. This was
probably seen as adequate to handle all of the CF cards then on the
market, back when the camera was released, and the camera was simply
not tested with cards larger than around 128 MB.

Formatting the card as FAT16, with a suitably-large cluster size (16k
or 32k bytes) seems to entirely resolve the problem.

--
Dave Platt AE6EO
Friends of Jade Warrior home page: http://www.radagast.org/jade-warrior
I do _not_ wish to receive unsolicited commercial email, and I will
boycott any company which has the gall to send me such ads!

"Dave Platt" wrote in message
...
I just encountered a problem with a Nikon CoolPix 950 which caused
inconvenience and might have resulted in a loss of images from the CF
card. As a similar problem might exist with other camera models,
other people might want to be aware of the issue.

I just bought the 950 used. It came with the original 8MB Compact
Flash card... much too small to be useful. I stopped by an
electronics store and bought the smallest-capacity CF card they had in
stock... 1 GB.

Popped it in the camera, let the camera format it, and started taking
photos. All seemed to be well. After a few days, I pulled the card,
popped it into a CF/PCMCIA adapter, and inserted the adapter into my
laptop's card slot.

Voila... nothing happened. The icon didn't appear. No files available.

I dug around a bit, and found that the Linux kernel's support module
for the FAT/VFAT filesystem objected to the filesystem on the card.
The partition table was valid, there was a single FAT16 partition on
the card, but the filesystem on that partition was unmountable. The
kernel reported that the filesystem had almost a quarter-of-a-million
clusters, which is more than the limit allowed for a FAT16 filesystem.

I tried formatting the partition using the linux "mkfs.vfat" program.
The filesystem formatted OK, the filesystem-test program "fsck.vfat"
said that it was OK... but when I put the card back into the camera,
the camera insisted that the card was unformatted. When I allowed the
camera to reformat the card again, the same problem resulted.

So... Linux wouldn't read the camera's filesystem, and the camera
wouldn't read the Linux-constructed filesystem.

This problem didn't exist for the 8MB card which came with the 950,
nor with a 128 MB card which came with a CoolPix 800 I'd bought
earlier.

Further testing showed why... the Linux utility was (silently)
deciding to build a FAT32 filesystem on the 1-gig card due to the
large number of sectors, and the CoolPix 950 can't read FAT32.

So, I used "mkfs.vfat -f 16" on the Linux system, forcing the system
to format the card with a FAT16 filesystem. The Linux utility did so,
simply increasing the size of the clusters so that the number of
clusters didn't exceed 2^16.

Problem solved, I believe. The CoolPix 950 happily accepted the card
with this FAT16 partition without requiring a reformat, built the dcim
directory hierarchy, and has allowed me to take a bunch of pictures.
Linux can mount the filesystem and read the files, and the Linux
filesystem-check program declares that the filesystem is clean (no
internal inconsistencies).

I suspect that the original formatting of the card, by the CoolPix
950, was a case of data corruption waiting to happen. The camera
appears to have built an "illegal" filesystem... one which had cluster
numbers that couldn't be stored in 16 bits. I'd guess that the
camera's firmware wasn't written to handle such large cards, and
doesn't fail gracefully when faced by them.

If I'd stored enough images on the card, there's a fair chance that
some of the images (old or new) would become corrupt or unreadable...
or the card would have "run out of space" when it was only about 25%
full.

Fortunately, the camera firmware seems able to handle FAT16
filesystems with larger cluster sizes. I'll have to try filling it up
with big TIFF images and make sure that everything still works OK when
the card has lots of data on it.

--
Dave Platt AE6EO
Friends of Jade Warrior home page: http://www.radagast.org/jade-warrior
I do _not_ wish to receive unsolicited commercial email, and I will
boycott any company which has the gall to send me such ads!
Most cameras are quite picky about the file system. Nearly all of them will
refuse to
recognize a card which was formatted by the computer.
Jim

On Sun, 13 Jan 2008 23:34:34 -0800, (Dave Platt)
wrote in :

In article ,
John Navas wrote:

http://en.wikipedia.org/wiki/File_Allocation_Table#Final_FAT16

#snip#

The apparent problem is thus your Linux system, not the camera or the
card.

I believe you're incorrect, John.

FAT16 has an ultimate limit of 2 GB, which is set by the maximum
number of clusters, multiplied by the maximum cluster size. As the
entry you cited pointed out,

The limit on partition size was now dictated by the 8-bit signed count
of sectors-per-cluster, which had a maximum power-of-two value of 64.
With the usual hard disk sector size of 512 bytes, this gives 32 KB
clusters, thereby fixing the "definitive" limit for the FAT16
partition size at 2 gibibytes.

2 gibibytes is, indeed, 65536 sectors times 32768 bytes per sector...
and according to the Wikipedia article, the limit in FAT16 is actually
only 65517 clusters.

2 GB is actually 65 K *clusters* times a *cluster* size of 32 KB.
Each cluster is 64 sectors.

However, you can only reach the 2-gib limit if you use 32k-byte
sectors.

32 KB *clusters*. Sector size is still 512 bytes.

The problem with the Coolpix 950 seems to be that its filesystem
formatter does *not* use 32k-byte clusters. It formats the filesystem
using fewer sectors per cluster... enough fewer that the required
number of clusters exceeds 65517. It shouldn'ta done that, I think.
The cluster size used by the camera allows support of a 128MB flash,
and I think it would allow a 256MB flash, but it won't go beyond that.

The apparent problem is that your Linux disto was misinterpreting the
FAT16 partition format.

--
Best regards,
John Navas
Panasonic DMC-FZ8 (and several others)

On Mon, 14 Jan 2008 11:06:41 -0800, Dave Platt wrote:
In article ,
ray wrote:

Rather strange. I've used quite a variety of CF and SD cards from 8mb to
2gb on several Linux systems and never had an issue. What distro are you
running?

Debian (testing/unstable), with a 2.6.23 kernel.

My guess at this point is that the CoolPix 950 firmware has a hard-coded
4k-byte cluster size coded in its FAT16 formatting logic.

All sorts of "hard coding" logic (sic) can bite you in the @$$ as
technology moves forward while your piece of crap^h^h^h^h equipment
(camera) stands still. :-)

I have an old Kodak DC-280 which is only capable of displaying the
(approx.) number of images available on the CF card as "nnn" (i.e., a
max count of 999.) With the camera set at its max ("High")resolution a
1 GB card will play nice in the camera, with a image count that will
only display as' "---" in the nnn-count filed. With an empty 2GB card
installed, the camera claims only 12 images are left. And, indeed, it
would only take 12 pictures and then displayed "Card Memory Full".

I believe the counting problem is the typical binary (or decimal) field
overflow (without detection) and the rest of the camera's logic
continues on with illogical/erroneous values.

In a slightly-not-as-old Kodak DX3600, the 1 GB card claims 1580 images
are available. With the 2 GB card, it claims 3158 images are available.
From date-of-purchase to this day I am sure I have not taken 3158
pictures with this camera. Maybe not even 1580...

Oh, those were the days! 2 MB Pixel technology.

Jonesy
--
Marvin L Jones | jonz | W3DHJ | linux
38.24N 104.55W | @ config.com | Jonesy | OS/2
*** Killfiling google posts: http://jonz.net/ng.htm