A call for open formats

The following is taken from a post to the Telecine Internet Group:

Wanting around at IBC this year, one thing stuck me more than anything else. There are now more proprietary capture formats than ever before.

This isn’t anything new, after all video has a long and unsavoury history of competing formats, much to the chagrin of everyone who backed HD-DVD for instance. But with digital formats becoming dominant, this has reached fever pitch. And I would argue, it’s completely unnecessary at best, and at worst it’s completely detrimental to the industry.

RED gives no impression that their business model is anything other than packaging for a proprietary format. But they give you tools to work with it, that are for the most part pretty good, but also available for free. You can gain access to the SDK, but only if you are willing to sign an NDA. incredibly, this is the most accessible of all the formats. Silicon imaging want to charge you $1000+ just to decode footage shot on their cameras. And the new champion of digital camera formats, ArriRaw, is completely unsupported for the most part. I spoke to someone about the long awaited SDK, only to be told that it is actually available, but only to select Arri partners. Whatever the hell that means. And it goes on and on with the likes of Sony, Panasonic ad naseum.

Granted, this is nothing new. But what I don’t understand is why we as professionals dealing with the ramifications of all of this continue to do so with smiles on our faces. Everyone is excited at the Arri stand this year. The footage looks great. That is more important than the ability to post the footage, as perhaps it should be. But given the footage from the camera is so good, why limit the ability to properly work with it? Why shouldn’t I be able to take my ArriRaw files into any post-house, regardless of the grading system or infrastructure used. Surely this would be best for Arri et al?

And worst of all, why do we, as the hapless victims of this situation, continue to allow it to happen? Why do we continue to evangelise a technology that is ultimately detrimental to our day to day lives? The visual effects industry managed to find a common ground with OpenEXR, I can only hope we might one day do the same.

Posted: September 11th, 2011
Categories: Opinion
Tags: , , , , , ,
Comment from Peter - 9/11/2011 at 12:42 pm

Adobe has made an effort in this regard years ago with their Cinema DNG standard. For stills, regular DNG has become a de-facto standard that is supported by almost all RAW converters out there, with some camera manufacturers even supporting it as an aquisition format (Pentax for example). A closed-source lossless converter for almost all proprietary still RAW formats is available for free as well as a few open source projects, and the file format specification is openly available for anyone to use (though there is no SDK that ensures consistent results of RAW conversion and exchange of settings/looks regardless of the host software, like Red offers via their SDK). Manufacturers still insist on recording in their own formats and offering their own SDKs (Nikon and Phase One are prominent examples) and even encrypt some data (like white balance), so that makers of conversion software need to reverse-engineer the file format. I doubt that practice will be different in the film industry, no matter how much we want them to give us data in an open format. But a converter and universal support for an open standard like Cinema DNG would solve a lot of those issues. We are basically in a position now where the stills industry was several years ago.

One downside of the regular DNG (and most likely Cinema DNG, too) is that the way the sensor data is stored still depends on the sensor type. That means special sensors like Sigma’s Foveon type require special code in the reading application, i.e. it is not guaranteed that every type of DNG will be readable by future software, but the alternative would be debayering/conversion during the conversion process (which Adobe’s DNG Converter offers as an option), which means you no longer haver the original non-debayered RAW data from the sensor of course.

I was also shocked to learn that the Alexa only supports ProRes as a direct-to-edit format (ProRes is obviously proprietary and can only be read through Apple’s QuickTime architecture with their codec installed, i.e. there is no way to use it on Linux-based systems like Flame without a Mac acting as a gateway for example, and even on Windows there is no way to encode it, only non-optimized decoding). Avid’s DnxHD is an equally capable, but an open standard (SMPTE I think) and supported across platforms via open-source tools like ffmpeg and would have been a better choice IMO, especially now that everybody has seen how little Apple cares about the small market of highend professionals with the whole Final Cut Pro X incident.

You used to be able to request the Arriraw SDK from Arri’s website by filling out a form, don’t know what happened to that.

Comment from Jack - 9/17/2011 at 2:09 pm

Supposedly you can apply to be an “Arri Partner” by sending them an email. I’ve now done so, so will see what happens next.

Add your comment











*/ ?>
Comment from Peter - 9/11/2011 at 12:42 pm

Adobe has made an effort in this regard years ago with their Cinema DNG standard. For stills, regular DNG has become a de-facto standard that is supported by almost all RAW converters out there, with some camera manufacturers even supporting it as an aquisition format (Pentax for example). A closed-source lossless converter for almost all proprietary still RAW formats is available for free as well as a few open source projects, and the file format specification is openly available for anyone to use (though there is no SDK that ensures consistent results of RAW conversion and exchange of settings/looks regardless of the host software, like Red offers via their SDK). Manufacturers still insist on recording in their own formats and offering their own SDKs (Nikon and Phase One are prominent examples) and even encrypt some data (like white balance), so that makers of conversion software need to reverse-engineer the file format. I doubt that practice will be different in the film industry, no matter how much we want them to give us data in an open format. But a converter and universal support for an open standard like Cinema DNG would solve a lot of those issues. We are basically in a position now where the stills industry was several years ago.

One downside of the regular DNG (and most likely Cinema DNG, too) is that the way the sensor data is stored still depends on the sensor type. That means special sensors like Sigma’s Foveon type require special code in the reading application, i.e. it is not guaranteed that every type of DNG will be readable by future software, but the alternative would be debayering/conversion during the conversion process (which Adobe’s DNG Converter offers as an option), which means you no longer haver the original non-debayered RAW data from the sensor of course.

I was also shocked to learn that the Alexa only supports ProRes as a direct-to-edit format (ProRes is obviously proprietary and can only be read through Apple’s QuickTime architecture with their codec installed, i.e. there is no way to use it on Linux-based systems like Flame without a Mac acting as a gateway for example, and even on Windows there is no way to encode it, only non-optimized decoding). Avid’s DnxHD is an equally capable, but an open standard (SMPTE I think) and supported across platforms via open-source tools like ffmpeg and would have been a better choice IMO, especially now that everybody has seen how little Apple cares about the small market of highend professionals with the whole Final Cut Pro X incident.

You used to be able to request the Arriraw SDK from Arri’s website by filling out a form, don’t know what happened to that.

Comment from Jack - 9/17/2011 at 2:09 pm

Supposedly you can apply to be an “Arri Partner” by sending them an email. I’ve now done so, so will see what happens next.

Leave a Reply

Your email address will not be published. Required fields are marked *