Is holographic storage the way forward?

Last week I got up in a discussion with someone at NBC Universal about archiving. “We reckon the solution is holographic storage,” they said. They then went on to say that such systems have been in development by companies such as InPhase Technologies… for around 7 years now, and the PhD’s who have come up with the idea reckon it’s good for around 50 years.

Well, I’ve heard holographic storage mentioned a few times, but I remain skeptical that this is the right way to go for now. The obvious problem is that it’s unproven. I take issue with the prediction that it’s good for 50 years when it’s only been in development for 7. I have had DTF2 tapes that have developed faults within a 6-month period, and countless disk drives that have died within a shorter period. It’s for the same reason that we don’t yet use LTO4 technology here at Surreal Road yet- it looks good on paper, but not yet as proven as LTO3…

Even so, let’s assume that they live up to the hype. What you essentially have is an investment in a particular product. If the company that manufactures the readers/writers or the company that manufactures the media (or both) goes bust, you’re left with something that then becomes useless. And as far as I’m aware, holographic storage technology is not particularly lucrative right now. So that adds a huge risk to the investment. On the back of this is that the technology isn’t exactly widespread. You couldn’t for example, archive to a bunch of holographic disks and then send them off to someone to restore them at a later date.

Aside from all this even, there is a larger issue lurking under the surface: no-one is particularly sure what data to archive anyway (in the film/video world at least). Right now, it seems that the digital cinema master is the best bet, as it is the format least likely to change right now. But what of non-D-Cinema productions? For instance, if your final output is DVCAM, should you archive the DVCAM avi or Quicktime files?

Personally I always convert everything to still sequences and then save off the audio separately. This minimizes the impact of any data corruption, allows quick access to specific portions of the production (if you only need to restore a specific shot later on for instance). It goes without saying that I also aim to create two copies of everything and keep one off-site if possible. Using an image format such as DPX means that it should be readable by at least some software in 10 years time. I have also anticipated the need to do spot-checks on the data integrity every year, and am ready to transcode everything completely or copy to a new media at some unspecified point in the future. Back in the ’90s I was archiving to CD-R (and slightly later to DVD-R), until it got to a stage (around 4-5 years later) when the discs were starting to become unreadable (despite being kept in ideal conditions). At this point I transferred everything to a new format (at that time I was actually using a nifty little system to back up raw data to DV tapes via firewire), and have repeated this a couple of times since. Needless to say, I still have data hanging around that is 15 years old. I now rely on Internet-based storage almost exclusively for everything except large files (but that’s a discussion for another article).

Another problem is metadata. There’s no agreed specification for many types of metadata (at least, not yet). By this I of course mean things like the title of a project, the respective rights to the images and so on. This isn’t a huge problem, you can pretty much get away with saving any relevant detail in a text file or Excel spreadsheet for instance (although notice how frequently Microsoft change the Excel and Word document formats- will they still be good 10 years from now?), but it is something that should be standardised. There is also the issue of other metadata, such as project files and software settings. Final Cut Pro XML is absolutely the right way to go in this regard- provided that you are using FCP of course. And even then, the project data is only really useful if you backup all the source data along with it, and let’s face it, that can often seem like a waste of time.

Ultimately, holographic storage may provide a decent long-term archive medium. But without a robust, long-term data strategy to support it, what is the real benefit?

Leave a Reply