How should I permanently store digital photographs?

Ken Weitzel wrote:

Don, I too have a large collection of old records - even
a couple of old shellac 78's. And old 45's. I've copied
them all to CD's, and am very pleased with the results.
If you can hear/don't like the compression aspects
of mp3, you can even do it uncompressed.

If you'd like to consider taking the task on yourself,
feel free to mail me; I'll be more than happy to share
what I've learned.

I've saved a copy of your post, and I'll probably email
you after the holidays. I've considered running the output
of my turntable+amp to my computer soundcard line input,
then using scratch/pop elimination software to create a WAV
file, then burning the WAV to CD. I haven't tried it, but
assume that's all a well-proven path. What stops me is
the time it would take. That's what I meant by it not
being "feasible" for me. I suppose if I did one album a week,
in few years, I'd have my collection done. Just in time
to transfer it all over to ... gamma ray wafers?

Cheers, and happy holidays.

"Farga Palenga Jengis" writes:

timeOday wrote:

5.25" disks were never in widespread usage
compared to CDs today.

That's just flat out incorrect. For many years,
5.25" mini-floppy disks were the main means
of distribution of software. It's all you could
purchase on the commercial market when 8"
floppies went away and before 3.5" micro-
minis became popular. Wordstar, Liesure Suit
Larry and GW BASIC were distributed on
5.25" mini-floppies. So were 1-2-3 and every
other app from that era you can name.

If you mean quantity as opposed to market
share, well, that's true but meaningless. If you
had data or applications during that period, it
was on 5.25" mini-floppies, tape or a Winchester
HD.

I don't think it's meaningless. I think how many of them with data are
out there has a big influence on how well supported they are in their
future. I had CP/M 400K 5.25" floppies, and DOS 360K, and DOS 1.2M.
Still do, actually, but the last time I tried the particular data I
wanted was unreadable -- not surprising for floppies, a *very* fragile
medium.

It's similar to the fact that you can still buy turntables to play
vinyl LPs, but it's much harder, perhaps impossible, to buy a Betamax
VCR (though the beta name is still used on unrelated professional
equipment).

If I were trying to preserve the US Constitution
or the Leaning tower of Pisa or gigabytes of data
from the Mars rovers, I agree that is a whole
different ballgame. Those artifacts are worth
millions of dollars.

Money isn't the only measure of worth. To me,
photos of my family are priceless, and I would like
to hand my children images of our family that they
can count on to remain usable. To a working press
photographer, his back catalog is precious, even
though on the market, it's not "worth millions."

But I can't afford the kind of archiving that is applied to things
like the Constitution, no matter how precious my photos are.

I think the transition period for CDs (maybe it
has already begun) will be longer than for 3.5" disks,
which was much longer than for 5.25" disks, and
so on, because computers have become so ubiquitous,
and the need for improvement is decreasing.

That statement that is belied by the history of
technology which predicts an opposite course.
Remember the Patent Office director who
foresaw no further technological developments since
everything had already been invented? Look at the
history of the automobile. Convergence into an
aerodynamically similar shape was predicted just ten
years ago. What happened instead was a fracturing into
hundreds of model types, each with smaller market share.

Actually, all cars *do* look alike today, compared to 20 years ago.
There's been a huge amount of convergence.

And I think that patent office director is an urban legend; I can't
find it on Snopes or as an attributed quote anywhere else right now,
though.

You say CD is safe. A few years ago, industry analysts
were predicting that the 100MB Zip drive would be the
removable standard. Ubiquity guarantees nothing.
Even standardization guarantees nothing. What will the
format be for dual-layer DVDs? I'd be willing to bet
that when it's decided, it will become irrelevant because
a new optical storage method has come into use.

Industry analysts are wrong all the time, most especially when
predicting the future. Observing that CDs *are* the ubiquitous
removable medium is quite different from predicting that something
*will be*.

Dual-layer DVD drives and media are on the shelves now, so I think the
standard has been decided, and it ain't irrelevant yet. (It may be
irrelevant because of low write speed, though; 2.7x, when 8x and 16x
are becoming common for single-layer.)

Here is what I do. I have two computers, and one
makes nightly backups to the other.

In what format? With what operating system and
backup application? Will they both remain usable
when XP goes away in a couple of years? MS backup
systems are notorious for lacking backward compatibility.
Use Ghost? Tried to restore from a five-year-old image?
Backup is a short-term safety practice, not an archiving
solution.

XP won't go away in a couple of years, any more than 98 is "gone away"
now. I haven't even upgraded any of my computers *to* XP; perhaps
never will.

Periodically, I copy my data to CDs.

Ever had a CD delaminate on you? Do you store
them in a temp-hum controlled area and periodically
refresh them by copying? And in what format? What's
your plan to translate and pull forward existing data
when protocols change? Your plan must go far beyond
simple backup of existing file formats.

No, I've never had a CD delaminate, either a pressed CD or a CD-R.
And I've never been shown one by a friend yet, either. It's damned
rare.

While you're right in theory about file formats, I don't expect to
lose software support for TIFF or JPEG files in my lifetime.

When I visit my parents, I take a copy of my backup
CDs and leave them there. I expect to switch to DVD
soon.

Using what format? Single layer straight data copy? How
long do you expect that hardware to remain active in
the market? Tried to find a computer with a micro-floppy
drive lately? Tried to use Zip disks at work? I'd bet that
in ten years, your backup CDs are unreadable from
age or lack of hardware/software to read them. Same
for DVD. If you don't have a refresh/pull-through plan,
your kids will inherit nothing from you but coasters.

I'd bet that in 10 years my backup CDs and DVDs can be read in every
computer in the house without trouble. And all the data will still be
on the (then-current) hard drives, too.

Tech boards are full of requests from people trying
to read old manuals and docs created with obsolete
word processing programs (Wang) that can't be read
today. If they find someone with a copy of the software,
it's on a medium nobody has a drive for. If you don't
translate and pull forward periodically, you're left
behind and the data becomes unusable.

Yes, and that's largely a problem because of poor file-format choices,
like proprietary word process or page layout programs. There's
nothing in that space vaguely comparable to JPEG or TIFF.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

"Farga Palenga Jengis" writes:

Mike F wrote:

Oh no !!! 8" floppies are gone ??? I still have
several boxes of them and the Double Sided Double
Density (1,2 megs !!) drives for them (maybe I'll start
a museum ??)

Heh. That's the way to keep any technology available
-- store the media, the hardware, spare parts and
manuals. Just recently, I purged all my 8" and 5.25"
media and gear, and dumped almost all of my 3.5"
stuff. I kept a few software items and some blank
micro-floppies, but I fear it's a bit like putting lettuce
into the fridge that you don't want to toss after making
a salad. It's too good to discard -- I'll wait 'till it rots.

I've already dumped the Sparq (!), Zip and Peerless
gear. I have one Jaz drive left, and use it with fingers
crossed since I only have one.

I still have functional 5.25" and 3.5" floppy drives (all my computers
have 3.5" floppies, you need them for emergency boot and recovery),
and two functional zip drives. No Sparq, Peerless, or Jazz, I never
did own any. I'm starting to think of cleaning out my 5.25" software
collection and using the drawers to hold CDs, but haven't done it yet
-- it'll be a LOT of work.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

"Don Lathrop" writes:

Yes, but Bill suggested that CD technology could
easily be placed into DVD equipment, and that those
who suggest that's a technology drag were
blind to the fact that the R&D was already done.
My point was that CD technology is already obsolete,
and the difficulty is retaining backward compatibility.

So you assert, but you give no evidence. Nobody has produced a DVD
drive that doesn't read CDs yet. Why not, if the compatibility costs
them a lot? There'd probably be a market.

As to the recopying of existing data onto new media,
yes, everyone does that, but that's not my point, either.
My point was that recopying, backing up and storage
are only part of an archiving solution. You must also
be prepared to translate or migrate formats that go
obsolete, and to index your data to make such a chore
possible, and obtain batch software to accomplish it.

You mention vinyl. I have LOTS of vinyl, and it is
simply not feasible for me to tape it -- who uses tape
anymore? It's difficult to find good metal tape anywhere.
It's not feasible to record it onto CD, either. So I'm
stuck with the vinyl. Some of my LPs aren't available
on CD or the (inferior) MP3. So I keep a turntable
and a stock of styli.

What do you mean it's not feasible to record it onto CD? It's
actually fairly easy. And MP3 isn't particularly inferior, especially
compared to vinyl -- if you pick the right bit rate.

Suppose you just ignore your collection of older images
for a few years while tech marches on, and then go
back to work on your archival Canon RAW images,
only to find that the new RAW standard isn't compatible
with your archived images. Suppose this is in 2015,
not an unreasonable scenario.

No problem, I still have the old software too. But support for the
old formats won't be dropped particularly quickly; it's a sunk cost,
and dropping it will cause screams.

It's not enough to just have the data on a medium that
is readable by your hardware. It has to be compatible
with your software as well.

Sure, but that's not a problem, and won't be in the forseeable future,
unless you pick stupidly. TIFF and JPEG are really pretty safe.

If you have ever gone to the trouble of rerecording
analog Hi-8 video to MiniDV (I have), you know
what a major pain that is, and how time consuming.

That's a realtime copy, and it's a pain. Very different from
converting a batch of files on the computer.

I have also converted 16mm film to VHS, then to MiniDV.

The loss of quality from analog to digital is great. Factor
in the time involved, the expense, and many people will
simply not do this chore, and lose their data.

Funny, I see it as a big *increase* in quality. And, most especially,
stability.

Will you not just keep going, regardless of the direction
that mass storage moves? Even when it *finally* moves
away from mechanical devices?

Because, as I've stated, it's not just having the data available
that is the archivist's problem. It's being able also to use the
data. Can you handle old Lotus PIC files today, from archived
spreadsheets? How about PCX files? There are dozens of
graphics formats that I have in my existing archived data that
I doubt I could even view today, much less manipulate.

Why? Old proprietary formats?

I can read PCX files with several pieces of software I have on the
system.

If you care enough, and your data is sufficiently worthwhile,
then it's easy.

No, it's far from easy. It's very difficult. In order to truly archive
all your data, you must routinely examine the entire index for
formats that are marginal, translate them (if you can) into current
standard forms, and re-archive them in multiple locations. This
takes time, knowledge and skill. In library science, this is called
"reading the shelf." Fail to do that, and you risk losing data
accessibility for a portion of your collection due to obsolescence.

Yes, a digital archive requires some attention.

Again, I'm not necessarily talking about the average home user
with a few shoebox JPGs, but working pros and technicians
dealing with thousands of images and the associated text, sound
and log files.

I'm a home user with, oh, something like 20,000 images on the hard
disk, indexed, labeled, etc. It's easy, so far. When some of the
formats start to age out, there will be some lumps in the workload;
but I don't think that TIFF or JPEG will start to age out during my
life. If they do, I'll cope.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

"Don Lathrop" writes:

Yes, but Bill suggested that CD technology could
easily be placed into DVD equipment, and that those
who suggest that's a technology drag were
blind to the fact that the R&D was already done.
My point was that CD technology is already obsolete,
and the difficulty is retaining backward compatibility.

So you assert, but you give no evidence. Nobody has produced a DVD
drive that doesn't read CDs yet. Why not, if the compatibility costs
them a lot? There'd probably be a market.

As to the recopying of existing data onto new media,
yes, everyone does that, but that's not my point, either.
My point was that recopying, backing up and storage
are only part of an archiving solution. You must also
be prepared to translate or migrate formats that go
obsolete, and to index your data to make such a chore
possible, and obtain batch software to accomplish it.

You mention vinyl. I have LOTS of vinyl, and it is
simply not feasible for me to tape it -- who uses tape
anymore? It's difficult to find good metal tape anywhere.
It's not feasible to record it onto CD, either. So I'm
stuck with the vinyl. Some of my LPs aren't available
on CD or the (inferior) MP3. So I keep a turntable
and a stock of styli.

What do you mean it's not feasible to record it onto CD? It's
actually fairly easy. And MP3 isn't particularly inferior, especially
compared to vinyl -- if you pick the right bit rate.

Suppose you just ignore your collection of older images
for a few years while tech marches on, and then go
back to work on your archival Canon RAW images,
only to find that the new RAW standard isn't compatible
with your archived images. Suppose this is in 2015,
not an unreasonable scenario.

No problem, I still have the old software too. But support for the
old formats won't be dropped particularly quickly; it's a sunk cost,
and dropping it will cause screams.

It's not enough to just have the data on a medium that
is readable by your hardware. It has to be compatible
with your software as well.

Sure, but that's not a problem, and won't be in the forseeable future,
unless you pick stupidly. TIFF and JPEG are really pretty safe.

If you have ever gone to the trouble of rerecording
analog Hi-8 video to MiniDV (I have), you know
what a major pain that is, and how time consuming.

That's a realtime copy, and it's a pain. Very different from
converting a batch of files on the computer.

I have also converted 16mm film to VHS, then to MiniDV.

The loss of quality from analog to digital is great. Factor
in the time involved, the expense, and many people will
simply not do this chore, and lose their data.

Funny, I see it as a big *increase* in quality. And, most especially,
stability.

Will you not just keep going, regardless of the direction
that mass storage moves? Even when it *finally* moves
away from mechanical devices?

Because, as I've stated, it's not just having the data available
that is the archivist's problem. It's being able also to use the
data. Can you handle old Lotus PIC files today, from archived
spreadsheets? How about PCX files? There are dozens of
graphics formats that I have in my existing archived data that
I doubt I could even view today, much less manipulate.

Why? Old proprietary formats?

I can read PCX files with several pieces of software I have on the
system.

If you care enough, and your data is sufficiently worthwhile,
then it's easy.

No, it's far from easy. It's very difficult. In order to truly archive
all your data, you must routinely examine the entire index for
formats that are marginal, translate them (if you can) into current
standard forms, and re-archive them in multiple locations. This
takes time, knowledge and skill. In library science, this is called
"reading the shelf." Fail to do that, and you risk losing data
accessibility for a portion of your collection due to obsolescence.

Yes, a digital archive requires some attention.

Again, I'm not necessarily talking about the average home user
with a few shoebox JPGs, but working pros and technicians
dealing with thousands of images and the associated text, sound
and log files.

I'm a home user with, oh, something like 20,000 images on the hard
disk, indexed, labeled, etc. It's easy, so far. When some of the
formats start to age out, there will be some lumps in the workload;
but I don't think that TIFF or JPEG will start to age out during my
life. If they do, I'll cope.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

"Don Lathrop" writes:

Ken Weitzel wrote:

Don, I too have a large collection of old records - even
a couple of old shellac 78's. And old 45's. I've copied
them all to CD's, and am very pleased with the results.
If you can hear/don't like the compression aspects
of mp3, you can even do it uncompressed.

If you'd like to consider taking the task on yourself,
feel free to mail me; I'll be more than happy to share
what I've learned.

I've saved a copy of your post, and I'll probably email
you after the holidays. I've considered running the output
of my turntable+amp to my computer soundcard line input,
then using scratch/pop elimination software to create a WAV
file, then burning the WAV to CD. I haven't tried it, but
assume that's all a well-proven path. What stops me is
the time it would take. That's what I meant by it not
being "feasible" for me. I suppose if I did one album a week,
in few years, I'd have my collection done. Just in time
to transfer it all over to ... gamma ray wafers?

Same amount of time it took me to tape them in the first place, but
happening all at once. And doesn't require my full-time attention.

And the later transfers are much easier -- fewer individual pieces to
handle.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

"Don Lathrop" writes:

Ken Weitzel wrote:

Don, I too have a large collection of old records - even
a couple of old shellac 78's. And old 45's. I've copied
them all to CD's, and am very pleased with the results.
If you can hear/don't like the compression aspects
of mp3, you can even do it uncompressed.

If you'd like to consider taking the task on yourself,
feel free to mail me; I'll be more than happy to share
what I've learned.

I've saved a copy of your post, and I'll probably email
you after the holidays. I've considered running the output
of my turntable+amp to my computer soundcard line input,
then using scratch/pop elimination software to create a WAV
file, then burning the WAV to CD. I haven't tried it, but
assume that's all a well-proven path. What stops me is
the time it would take. That's what I meant by it not
being "feasible" for me. I suppose if I did one album a week,
in few years, I'd have my collection done. Just in time
to transfer it all over to ... gamma ray wafers?

Same amount of time it took me to tape them in the first place, but
happening all at once. And doesn't require my full-time attention.

And the later transfers are much easier -- fewer individual pieces to
handle.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

writes:

Chemically deactivated inorganic compound will without doubt outlast a
CD-R, such example being B&W prints and films processed with
polysulfide toner.

B&W prints depend on the gelatine layer, and the paper -- both
organic. And tasty to various insects, etc.
If a CD-R was a thermal printed ATM receipt that
will fade after a while, a polysulfide processed print is a note
written on paper with pencil.

Meanwhile, the guy that has shot film, on his cheap little Point &
Shoot
camera, and who has carefully stored the prints and negs in archival
plastic
album pages, ends up keeping his whole library of his life's
pictures! Is
that ironic, or what?

I stored my Quicktake pics to two different media.
1. Ricoh CD-R disc
2. 640MB MO disk.
MO is better time proven than CD-R technology and the lack of fierce
price war should keep the quality high(MOs are very expensive, ~$40 for
five 640MB disks). They're often used in medical, banking, government
data which should ensure the availability of playback device into the
future.

3. I'm going to make a third set of backups for the pics I really like
and this will be a physical print out from a photo lab with Fuji
Frontier silver-halide machine.

Again, what do you mean "silver-halide"? Chromagenic color materials
are not very permanent.

I've wondered what you could do with a two-dimensional "bar-code" type
structure (there are several in use) and an inkjet printer with
carbon-pigment inks on all-cotton archival paper for archiving an
image. What kind of bit density can you read off that kind of paper
reliably with a scanner? I *think* that a letter-size piece of paper
might well store a high-res jpeg of an image in that format. Then the
hardware you're dependent on is a scanner, plus documentation on the
bar-code and the jpeg formats. If the software still exists in
working form when you try to decode the page, so much the better.
This is an extreme approach, but for *unattended* storage might work
out better than lots of other approaches.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

writes:

Chemically deactivated inorganic compound will without doubt outlast a
CD-R, such example being B&W prints and films processed with
polysulfide toner.

B&W prints depend on the gelatine layer, and the paper -- both
organic. And tasty to various insects, etc.
If a CD-R was a thermal printed ATM receipt that
will fade after a while, a polysulfide processed print is a note
written on paper with pencil.

Meanwhile, the guy that has shot film, on his cheap little Point &
Shoot
camera, and who has carefully stored the prints and negs in archival
plastic
album pages, ends up keeping his whole library of his life's
pictures! Is
that ironic, or what?

I stored my Quicktake pics to two different media.
1. Ricoh CD-R disc
2. 640MB MO disk.
MO is better time proven than CD-R technology and the lack of fierce
price war should keep the quality high(MOs are very expensive, ~$40 for
five 640MB disks). They're often used in medical, banking, government
data which should ensure the availability of playback device into the
future.

3. I'm going to make a third set of backups for the pics I really like
and this will be a physical print out from a photo lab with Fuji
Frontier silver-halide machine.

Again, what do you mean "silver-halide"? Chromagenic color materials
are not very permanent.

I've wondered what you could do with a two-dimensional "bar-code" type
structure (there are several in use) and an inkjet printer with
carbon-pigment inks on all-cotton archival paper for archiving an
image. What kind of bit density can you read off that kind of paper
reliably with a scanner? I *think* that a letter-size piece of paper
might well store a high-res jpeg of an image in that format. Then the
hardware you're dependent on is a scanner, plus documentation on the
bar-code and the jpeg formats. If the software still exists in
working form when you try to decode the page, so much the better.
This is an extreme approach, but for *unattended* storage might work
out better than lots of other approaches.
--
David Dyer-Bennet, , http://www.dd-b.net/dd-b/
RKBA: http://noguns-nomoney.com/ http://www.dd-b.net/carry/
Pics: http://dd-b.lighthunters.net/ http://www.dd-b.net/dd-b/SnapshotAlbum/
Dragaera/Steven Brust: http://dragaera.info/

In article ,
says...
As DVDs go from single to dual-layer capability,
engineers would dearly love to drop the requirement
to be able to read CD data.

No they wouldn't.

Seriously, optical media is optical media. It's all read the same way -
with lasers. Legacy support is REALLY easy.

So I think it is YOU who doesn't know what they're talking about.
--
http://www.pbase.com/bcbaird/