long-term storage for digital photos, videos etc

In the old days, I used film. I had my first SLR in 1983 (Ricoh KR5).
Around 2002, I started getting a CD of the pictures along with the
prints. The JPEG files are around 500 kB. When I have time, I plan to
scan the old negatives in higher resolution. Presumably this will
result in TIFF files, which I should then convert to JPEGs for viewing
on a webserver etc. What resolution is recommended for ISO 200 35-mm
film? How large would the resulting TIFF be? What size JPEG should
result from this (such that viewing at 20×30 cm, say, shows no obvious
lack of quality)?

In 2007, I bought a Pentax K10D and in 2015 a Ricoh GR. I tend to take
more pictures with the digital cameras, for a variety of reasons, and
appreciably more with the GR, since I almost always have it with me.
With the K10D, the procedure was similar to film: get prints made and
get the photos on CD as well. However, there are three differences.
First, I get both the JPEGs (produced by the camera) and the RAW files
on the disk. Second, these days it is usually a DVD. Third, the JPEG
files are bigger, around 3 MB. (This is fine as far as quality goes for
my purposes.) The JPEGs from the GR about 6 MB.

Up until now, I have copied all the JPEG files to web pages (I run my
own server). Even if I continue at the present rate (on average a photo
per day) for the rest of my life, disk space shouldn't be a problem;
we're looking at around 100 GB. Apart from the disk (RAID, but still),
I like to have robust backup copies. However, several questions arise:

o Are CDs/DVDs the best option for long-term storage? Or is there
something better today? Most important is permanence, but with
several hundred photos per batch, one DVD is not big enough if the
RAW files are included as well. Of course, I could process
smaller batches, or split big batches among several DVDs, but with
other storage media being reasonably inexpensive now, the question
is whether it would be good to move to something else.

o Some people just buy a new SD card when the one in use is full,
and use the old one as a backup. This might not be the cheapest
option, but is not too expensive. How robust is this medium for
long-term storage?

o What about other storage media, such as USB sticks? Are all of
these essentially the same under the hood, essentially SSDs,
though with varying speeds? (In the old days, at least, a
limitation of SSDs was how often they could be rewritten, but I
don't think that this is a problem today, especially in this case
where it is rewritten at most a few times before becoming static
as long-term storage.)

o A problem with SD cards in some case, such as on my wife's camera,
a Pentax Optio, is that there are several directories, each with
a few files. This is not easy to use, especially if restoring
files from backup. I suppose that I could mount it as a disk,
though, create a new directory, and copy all the files into that.

o My wife has a video camera which writes directly to (small) CDs.
What is the best way to secure long-term copies of such media?

o These days, SD cards are large enough for videos, and we have
various cameras with SD cards which can take videos (the Pentax
Optio, the KR5, the GR, iPhone, iPad). We haven't done much with
videos yet, in the past because it was too much trouble because
storage was small and expensive, these days because it is not
clear what the best path forward is. What do people suggest here?
I don't think that putting the videos on a hard disk for viewing
with a web browser would be a good idea---probably too much disk
space, and might take too long to load. Perhaps better would be
some sort of USB device which one could attach locally when one
wants to see the videos.

Relatively quickly, I'd like to come up with a good strategy, and stick
too it.

In article , Phillip Helbig (undress to
reply) wrote:

In the old days, I used film. I had my first SLR in 1983 (Ricoh KR5).
Around 2002, I started getting a CD of the pictures along with the
prints. The JPEG files are around 500 kB. When I have time, I plan to
scan the old negatives in higher resolution. Presumably this will
result in TIFF files, which I should then convert to JPEGs for viewing
on a webserver etc. What resolution is recommended for ISO 200 35-mm
film?

the highest your scanner supports.

the key is to get a high quality film scanner, not a flatbed with some
goofy attachment.

How large would the resulting TIFF be?

big, but who cares.

file size does not matter. disk space is cheap, far cheaper than the
time it takes to scan.

What size JPEG should
result from this (such that viewing at 20×30 cm, say, shows no obvious
lack of quality)?

depends on various factors, but in general, about 10-20% of the tiff.


In 2007, I bought a Pentax K10D and in 2015 a Ricoh GR. I tend to take
more pictures with the digital cameras, for a variety of reasons, and
appreciably more with the GR, since I almost always have it with me.
With the K10D, the procedure was similar to film: get prints made and
get the photos on CD as well.

those were low resolution and completely worthless.

However, there are three differences.
First, I get both the JPEGs (produced by the camera) and the RAW files
on the disk. Second, these days it is usually a DVD. Third, the JPEG
files are bigger, around 3 MB. (This is fine as far as quality goes for
my purposes.) The JPEGs from the GR about 6 MB.

don't waste your time with dvds and the jpgs are bigger because they're
*much* higher resolution than the worthless cd you got from the camera
store.

Up until now, I have copied all the JPEG files to web pages (I run my
own server). Even if I continue at the present rate (on average a photo
per day) for the rest of my life, disk space shouldn't be a problem;
we're looking at around 100 GB. Apart from the disk (RAID, but still),
I like to have robust backup copies. However, several questions arise:

raid is not a backup.

o Are CDs/DVDs the best option for long-term storage? Or is there
something better today?

no, nor have they ever been. not only are they not reliable, they're a
huge pain in the ass. some of them are probably no longer readable.

Most important is permanence, but with
several hundred photos per batch, one DVD is not big enough if the
RAW files are included as well. Of course, I could process
smaller batches, or split big batches among several DVDs, but with
other storage media being reasonably inexpensive now, the question
is whether it would be good to move to something else.

absolutely.

use multiple hard drives, with at least one off site. consider using a
cloud service too.

o Some people just buy a new SD card when the one in use is full,
and use the old one as a backup. This might not be the cheapest
option, but is not too expensive. How robust is this medium for
long-term storage?

that's incredibly stupid and incredibly expensive.

o What about other storage media, such as USB sticks? Are all of
these essentially the same under the hood, essentially SSDs,
though with varying speeds?

they're not as sophisticated as ssds and usually much slower,
especially the cheap ones.

(In the old days, at least, a
limitation of SSDs was how often they could be rewritten, but I
don't think that this is a problem today, especially in this case
where it is rewritten at most a few times before becoming static
as long-term storage.)

there is a limit, but it's high enough that it's not a problem.

o A problem with SD cards in some case, such as on my wife's camera,
a Pentax Optio, is that there are several directories, each with
a few files. This is not easy to use, especially if restoring
files from backup. I suppose that I could mount it as a disk,
though, create a new directory, and copy all the files into that.

how is that a problem? copy everything. done.

o My wife has a video camera which writes directly to (small) CDs.
What is the best way to secure long-term copies of such media?

copy whatever you have to a hard drive, then replace the camera.

modern smartphones shoots *much* higher quality video.

o These days, SD cards are large enough for videos, and we have
various cameras with SD cards which can take videos (the Pentax
Optio, the KR5, the GR, iPhone, iPad). We haven't done much with
videos yet, in the past because it was too much trouble because
storage was small and expensive, these days because it is not
clear what the best path forward is. What do people suggest here?
I don't think that putting the videos on a hard disk for viewing
with a web browser would be a good idea---probably too much disk
space, and might take too long to load. Perhaps better would be
some sort of USB device which one could attach locally when one
wants to see the videos.

hard drives are the best solution and faster than an sd card.

Per Phillip Helbig (undress to reply):
o Are CDs/DVDs the best option for long-term storage?
My experience is that CDs and DVDs degrade with time and temperature.
Something about the ones you burn at home using a different process (dyes vs
physical pitting?) from the pre-recorded ones.... but that's above my pay
grade... Bottom line for me: I would not trust CDs or DVDs.

o Some people just buy a new SD card when the one in use is full,
and use the old one as a backup. This might not be the cheapest
option, but is not too expensive. How robust is this medium for
long-term storage?

I feel the most comfortable with database-based backups to hard drives.

My data lives on a NAS box and I have a second NAS box mirrored to it "Just
In Case".... the longterm goal being to find a home offsite for that second
box and back up to it nightly - my own cloud solution, so-to-speak.

File copy backups have the problem that if a file quietly disappears or
becomes corrupted, you will never know.... OTOH, a database-based backup like
Macrium Reflect (which I use) will (hopefully) catch problem files...although
accidental manual deletions probably would slip through the cracks.

In addition to the NAS mirror, I have five 1-TB drives that I rotate between
my PC, my car's glove compartment, my garden shed, and the neighbor's garden
shed on a monthly basis.

On day one of each month, I take the oldest drive, format it, and have
Macrium to a full data backup to it. Then, on each succeeding day, I have
Macrium do an incremental backup to it - so I have my data as of the
beginning of the month plus any changes as of any given day of the month.

If I had less data, I would supplement that scheme with one of the
cloud-based solutions like Carbonite..... In fact, I have what I call "Data"
(which fits easily on a 1 TB drive) and what I call "Media" which occupies
about 8-9 TB on the NAS boxes. "Media" in my world is basically my ripped
movie collection and is more-or-less expendable because I could always re-rip
from the DVDs....

So: "Data" is where it's at for me and, now that you have me thinking about
it, maybe 1 TB is within feasibility for something like Carbonite over a FIOS
connection.
--
Pete Cresswell

In article ,
(PeteCresswell) wrote:


File copy backups have the problem that if a file quietly disappears or
becomes corrupted, you will never know....

modern file systems can detect and correct bitrot, without any
additional software.

OTOH, a database-based backup like
Macrium Reflect (which I use) will (hopefully) catch problem files...although
accidental manual deletions probably would slip through the cracks.

only if it checksums every copy, which would be an option because it's
very slow.

In addition to the NAS mirror, I have five 1-TB drives that I rotate between
my PC, my car's glove compartment, my garden shed, and the neighbor's garden
shed on a monthly basis.

that's good, unless you live where it gets very cold, very hot or both.

the drives should also be encrypted so that if they're lost or stolen,
the contents cannot be read.

On day one of each month, I take the oldest drive, format it, and have
Macrium to a full data backup to it. Then, on each succeeding day, I have
Macrium do an incremental backup to it - so I have my data as of the
beginning of the month plus any changes as of any given day of the month.

no need to reformat it each time.

On 1/4/2018 11:05 AM, (PeteCresswell) wrote:
Per Phillip Helbig (undress to reply):
o Are CDs/DVDs the best option for long-term storage?
My experience is that CDs and DVDs degrade with time and temperature.
Something about the ones you burn at home using a different process (dyes vs
physical pitting?) from the pre-recorded ones.... but that's above my pay
grade... Bottom line for me: I would not trust CDs or DVDs.

o Some people just buy a new SD card when the one in use is full,
and use the old one as a backup. This might not be the cheapest
option, but is not too expensive. How robust is this medium for
long-term storage?

I feel the most comfortable with database-based backups to hard drives.

My data lives on a NAS box and I have a second NAS box mirrored to it "Just
In Case".... the longterm goal being to find a home offsite for that second
box and back up to it nightly - my own cloud solution, so-to-speak.

File copy backups have the problem that if a file quietly disappears or
becomes corrupted, you will never know.... OTOH, a database-based backup like
Macrium Reflect (which I use) will (hopefully) catch problem files...although
accidental manual deletions probably would slip through the cracks.

In addition to the NAS mirror, I have five 1-TB drives that I rotate between
my PC, my car's glove compartment, my garden shed, and the neighbor's garden
shed on a monthly basis.

On day one of each month, I take the oldest drive, format it, and have
Macrium to a full data backup to it. Then, on each succeeding day, I have
Macrium do an incremental backup to it - so I have my data as of the
beginning of the month plus any changes as of any given day of the month.

If I had less data, I would supplement that scheme with one of the
cloud-based solutions like Carbonite..... In fact, I have what I call "Data"
(which fits easily on a 1 TB drive) and what I call "Media" which occupies
about 8-9 TB on the NAS boxes. "Media" in my world is basically my ripped
movie collection and is more-or-less expendable because I could always re-rip
from the DVDs....

So: "Data" is where it's at for me and, now that you have me thinking about
it, maybe 1 TB is within feasibility for something like Carbonite over a FIOS
connection.


When I decide on a method, or procedure, I use the following analysis:
Does it work;
is it sustainable;
what is my plan B (method of getting out,) if something goes wrong.

Of course only you can decide how valuable the data is to you.

Note: my none answer, and why.

--
PeterN

"Phillip Helbig (undress to reply)" wrote

| o Are CDs/DVDs the best option for long-term storage? Or is there
| something better today?

Why not have redundant storage? I use DVDs and
CDs for virtually all data. I also copy it to an old hard
disk. I also put much of it on a USB stick. And I try
to keep it all on my computer.
For basic, periodic backup I use DVDs, but I
like to also use other methods.

Some people claim CDs/DVDs don't last. I've
never had one go bad. I have homemade CDs going
back to at least 2004-ish that are still fine. I
recently checked my Visual Studio 6 CD, from '99.
That's also still fine.
We had a discussion about this recently in a
Windows group. It turned out that the person
who most distrusted CDs/DVDs hadn't made one
since the early 2000s. He got a few "coasters"
and decided CDs were undependable, never
sticking around to see the technology mature.

On 1/4/2018 5:40 AM, Phillip Helbig (undress to reply) wrote:
In the old days, I used film. I had my first SLR in 1983 (Ricoh KR5).
Around 2002, I started getting a CD of the pictures along with the
prints. The JPEG files are around 500 kB. When I have time, I plan to
scan the old negatives in higher resolution. Presumably this will
result in TIFF files, which I should then convert to JPEGs for viewing
on a webserver etc. What resolution is recommended for ISO 200 35-mm
film? How large would the resulting TIFF be? What size JPEG should
result from this (such that viewing at 20×30 cm, say, shows no obvious
lack of quality)?

In 2007, I bought a Pentax K10D and in 2015 a Ricoh GR. I tend to take
more pictures with the digital cameras, for a variety of reasons, and
appreciably more with the GR, since I almost always have it with me.
With the K10D, the procedure was similar to film: get prints made and
get the photos on CD as well. However, there are three differences.
First, I get both the JPEGs (produced by the camera) and the RAW files
on the disk. Second, these days it is usually a DVD. Third, the JPEG
files are bigger, around 3 MB. (This is fine as far as quality goes for
my purposes.) The JPEGs from the GR about 6 MB.

Up until now, I have copied all the JPEG files to web pages (I run my
own server). Even if I continue at the present rate (on average a photo
per day) for the rest of my life, disk space shouldn't be a problem;
we're looking at around 100 GB. Apart from the disk (RAID, but still),
I like to have robust backup copies. However, several questions arise:

o Are CDs/DVDs the best option for long-term storage? Or is there
something better today? Most important is permanence, but with
several hundred photos per batch, one DVD is not big enough if the
RAW files are included as well. Of course, I could process
smaller batches, or split big batches among several DVDs, but with
other storage media being reasonably inexpensive now, the question
is whether it would be good to move to something else.

o Some people just buy a new SD card when the one in use is full,
and use the old one as a backup. This might not be the cheapest
option, but is not too expensive. How robust is this medium for
long-term storage?

o What about other storage media, such as USB sticks? Are all of
these essentially the same under the hood, essentially SSDs,
though with varying speeds? (In the old days, at least, a
limitation of SSDs was how often they could be rewritten, but I
don't think that this is a problem today, especially in this case
where it is rewritten at most a few times before becoming static
as long-term storage.)

o A problem with SD cards in some case, such as on my wife's camera,
a Pentax Optio, is that there are several directories, each with
a few files. This is not easy to use, especially if restoring
files from backup. I suppose that I could mount it as a disk,
though, create a new directory, and copy all the files into that.

o My wife has a video camera which writes directly to (small) CDs.
What is the best way to secure long-term copies of such media?

o These days, SD cards are large enough for videos, and we have
various cameras with SD cards which can take videos (the Pentax
Optio, the KR5, the GR, iPhone, iPad). We haven't done much with
videos yet, in the past because it was too much trouble because
storage was small and expensive, these days because it is not
clear what the best path forward is. What do people suggest here?
I don't think that putting the videos on a hard disk for viewing
with a web browser would be a good idea---probably too much disk
space, and might take too long to load. Perhaps better would be
some sort of USB device which one could attach locally when one
wants to see the videos.

Relatively quickly, I'd like to come up with a good strategy, and stick
too it.




Most scanners will give you a choice of jpg or tif.

For size constraints , no harm in scanning to jpg, just as long as you
do not edit it (or opt for losless).

I scan my 35mm stuff at 3200 dpi


BTW: Make many backups. External hard drives, DVD etc.

In article , Mayayana
wrote:

| o Are CDs/DVDs the best option for long-term storage? Or is there
| something better today?

Why not have redundant storage?

yep, just not with the mediums you suggest.

I use DVDs and
CDs for virtually all data. I also copy it to an old hard
disk. I also put much of it on a USB stick. And I try
to keep it all on my computer.

all bad choices.

cd/dvds are not archival and a pain in the ass. they also don't hold
much.
old hard drives are more likely to fail.
usb sticks are not cost effective nor are they big enough to back up a
hard drive.

buy multiple hard drives from different sellers so you get different
batches. clone among them and keep at least one off site.



Some people claim CDs/DVDs don't last.

they're correct. they don't.

I've never had one go bad.

buy a lottery ticket.

I have homemade CDs going
back to at least 2004-ish that are still fine. I
recently checked my Visual Studio 6 CD, from '99.
That's also still fine.

that's a commercial cd.

In article , philo
wrote:

Most scanners will give you a choice of jpg or tif.

scanner software does that.

the scanner just spits out bits.

For size constraints , no harm in scanning to jpg, just as long as you
do not edit it (or opt for losless).

the harm is that you don't have a lossless copy, regardless if you edit
it.

always scan to the highest quality possible. you can always downsize
later.

upsizing a low quality scan requires rescanning at a higher quality,
which is a huge pain in the ass.

tl;dr - do it right the first time.

On 1/4/18 6:40 AM, Phillip Helbig (undress to reply) wrote:
Are CDs/DVDs the best option for long-term storage? Or is there
something better today? Most important is permanence, but with
several hundred photos per batch, one DVD is not big enough if the
RAW files are included as well.

For optical media, you might consider M-DISC's.

They're claimed to have a life of hundreds of years, because
they don't use "dyes" in the recording process, but rather
some inorganic compound.

If you needed more space than a DVD offers, get Bluray which
offers 25gb and 50gb capacities. Also available in M-DISC.

Of course, any burner (Bluray or otherwise) should have
M-DISC support if you want to do this.