Backing Up Is Hard To Do

Actually it isn’t, but it is challenging to figure a reliable way to back up computer files that isn’t so messy that you don’t actually get round to doing it.

I’ve been working with computers for close to 40 years and throughout that time I’ve been through more backup regimes than I care to remember.  The aim of all this backing up was to try and protect my digital stuff against both personal (and colleagues) incompetence and the inevitable hardware failures.  I’ve backed up files onto tapes, cassettes, floppy discs, ZIP drives, CD-Rs, DVD-Rs, USB sticks, memory cards, external hard drives, disk arrays and clouds – and all of these have at some point let me down.  In some cases files have got corrupted, in others (ZIP, I’m looking at you) the entire device has died and I’ve had cloud storage services go bust.  I can also recall a lengthy exercise when I was involved with copying scientific data files from one obsolete disc storage system, to another disc storage system which rapidly became obsolete quite quickly too.

And the incompetence thing, yes I have accidentally deleted an entire directory of stuff. Damn the power of the UNIX command line.

Old DVDs

This moment of reflection has been triggered by my unearthing a huge pile of recordable CDs and DVDs that were for a long time my default storage system for my collection of photographs.  This was an entirely reasonable approach at time when cameras generated 4 or 8 MB files and you could fit an entire trip onto a couple of 4GB DVDs.  But the camera files, especially the RAW ones got bigger and someone invented digital video so it’s quite possible to generate 100s of GB for files on a single project.  I don’t think I’ve had a terabyte project yet, but it’ll happen one day.  This turns into an awful big pile of DVDs. And then there’s DVD rot too.  All in all I think it’s safe to say that the day of DVDs as my storage device of choice has passed.  I have (with some relief, and not inconsiderable guilt) consigned my collection of discs to land-fill.

So what next.  At this point I can only see two credible ways of the storing the multi-terabyte collection of files I’ve got to worry about.  Hard discs or clouds. Trouble is that I don’t really trust either.  Hard discs do die.  If you’re lucky you get a little bit of warning and you can frantically copy the latest versions of files onto something else.  If you’re unlucky it all just stops working!  Clouds (yes, I know it’s really just someone else’s disc storage) can be fickle too.  The cloud providers can go bust or suffer major system failures – and at that point all you can do is hope that things get recovered.  The other challenge with clouds is the level of connectively to the remote systems – my external hard discs are connected by USB 3 (or similar or faster) but the clouds are connected via wifi and internet connections – and even on the best wifi connection a 100GB fileset is going to take a while to move around.

My current solution, is to use a mix of discs and cloud.   I’ve got two work-flows, one for photographic work and another for everything else.

For photographs and videos (where everything gets saved on increasingly small memory cards) I copy these off onto as hard disc as soon as possible (usually at the end of each day).  At the time of a trip or assignment these files get copied onto another hard disc and (if it’s been a particular good trip) onto another hard disc which get stored somewhere else other than the room where I do most of my photo processing.  Having multiple workplaces (or even multiple houses) is a good way to do this – I’ve often kept a spare set of discs in the back of the car.  It’s probably not the safest single storage place, but you’d have to be seriously unlucky to have your car stolen and your house burnt down at the same time.  These backup include pretty much all the files I’ve shot (even the rubbish ones). 

Once I’ve started processing the files and pick out the better images - it's amazing that the 200GB of files from a trip can get reduced to 15 keepers!  At this stage I’ll make use of cloud storage for the images, and the keepers will wind up being stored both on various hard discs, and probably on various cloud services too, mostly in iCloud and Dropbox.

No sign of clouds - but these cables lead there eventually.

For non-photographic files (word docs and powerpoint decks) my usual working practice is to work directly (at least when I’m connected) on files synchronised with Dropbox or iCloud so that I’ve got (synchronisation permitting) a copy both locally in the cloud.  But even with this approach I do still tend to make a periodic extra copies of my working files onto another hard disc somewhere – this is probably as much to do with my (demonstrated) capacity to delete the entire contents of a file (just before saving) as it is with with my mistrust of things in the cloud.   

And if I’m getting really twitchy about a particular file, I’ll probably email it to myself too! 

And sometimes I’ll print a copy too – paper lasts a long time, right?