Synaesthesia is the new software product by Surreal Road. It’s been in development for around four years now, and is almost at a point where it is production-ready.

But what is it?

Having worked on all sorts of film and TV productions in different capacities (of greatly varying budgets), it often amazed me how “disconnected” every role seems. This is especially true in areas like post-production, where people employed to enhance or otherwise change particular shots would do so without any knowledge of the history of that shot. It might be possible to find out the camera and lighting setup used for a particular shot in some cases, but what about the intent behind that setup? What was the cinematographer aiming for, and how can I better enhance that, as opposed to the more usual practices of (at best) attempting to reverse-engineer a shot in order to understand it, or (at worst) changing things in a more haphazard manner until something looks good.

This was a problem I’ve encountered on almost every production, and in part it’s unavoidable. The reality is that just as the writers are often left outside the gates of production, so too are the production crew long-gone when it comes to post. This also becomes a practical and logistical problem. Where is a particular reel of film? What was the time and date of a particular shot? On a very organised production, it is likely that the editor would be armed with most of this information, but in all other cases, there is simply no one around to ask.

I look at software created for the visual effects industry, and it is staggering: the functionality and capabilities of VFX software is advanced to the point where it’s possible to use these tools to quickly create shots that are indistinguishable from reality. But when it comes to the actual production process, we’re in a technological drought. Even popular writing software, such as Final Draft, is only slightly more useable than TextEdit, even with years of industry experience and development put into it. What was I supposed to use in my capacity as data manager on various things to stay on top of everything? Excel?

The solution of course, is that (those computer-savvy enough) people tend to cobble together some sort of database (usually in the ubiquitous FileMaker Pro) which serves the immediate needs of the production. Much of the time this works out rather well, the production ends up with a bespoke system that covers most of the bases, something “good enough”. But what about those people who haven’t the time or the resources to create something from scratch, or those people who just want to hit the ground running? Well, you are who Synaesthesia was designed for.

At its core, Synaesthesia is about keeping track of things about a production, from start to finish. Here’s a typical scenario:

1. You have a production. You add notes, storyboards, descriptions of characters, of sets, all to get a sense of what it’s about.
2. At some point you have a screenplay. You import that and it links all the scenes with sets and characters you’ve previously created, and adds anything that’s missing.
3. You refine the script, importing new versions as you go along, further fleshing out what you want to shoot and so on.
4. You create a database of people and equipment  you’re going to need, and assign them to different parts of the production.
5. You start shooting. You log what’s shot as it happens, along with notes, things like whether the take was good or not, what was recorded and making last-minute script updates.
6. You import data directly from digital footage (such as RED camera footage), in order to accurately log timecodes, and shooting parameters.
7. You start editing, having access to all your previous notes for each clip of footage that was shot. You can import sequences from an editing system and have Synaesthesia tell you which shot is used where. You can make changes to the edit from within Synaesthesia, and save those back to your editing system.
8. You can designate certain shots as needing effects work, and update those shots as new effects versions are completed.
9. Finally you can archive all the reels of footage, noting their locations, in case they’re ever needed again.

That’s quite a broad overview, and it assumes you’re going to use Synaesthesia from start to finish. But perhaps the best part of it is that you don’t have to. Maybe you’re only concerned with pre-production, and just want a place to keep storyboards, concept art, and screenplay versions organised? Maybe you just want to log continuity during a shoot? Or maybe you just want to tweak a couple of edits? Well then, Synaesthesia can help you.

It’s probably also helpful to mention what Synaesthesia (at least, in its current form) isn’t for:

• It’s not for budgeting or scheduling
• It’s not a replacement for software such as Final Draft
• It’s not a replacement for systems such as Final Cut Studio
• It’s not a server-based system, (it’s not possible for multiple people to make changes to the data at the same time).

A more detailed list of features is available  here. As I’ve said, Synaesthesia isn’t quite finished yet. It’s capabilities are still being worked out. But there are several key principles that we’ll always try to adhere to:

• It will be simple to use
• It will integrate with software you already use
• It will give you the information you need

But more than anything else, I want it to be for whatever you (the user) need. With that in mind, we will be inviting people to try out pre-release versions in order to tell us what you like, what you don’t, and what’s missing. You can sign up for an invitation here.

Where part 1 of this series… was mostly theory, anticipation and about establishing best practices for RedCode data, this part is all reality: what happens in practice  when you try and work with it.

After all, the vast majority of discussions of Red camera workflow are necessarily theoretical, after all very few people are using it for anything other than test footage. The editing stage is often the first stumbling block, as actually getting any of the footage into an editing suite to play around with can be challenging. There’s also a much more problematic issue that deserves attention: ensuring that the finished edit can actually be used for an online somewhere.

There is a popular workflow with Red footage which is to edit the R3D files directly in FCP. The benefit of this is that it is very simple: you just load the QuickTime proxies into a project and snip away as normal. You can even do your online in FCP if you wish, putting Final Cut’s colour correction and effects tools to good use. You can then export to a wide variety of formats, including some uncompressed* variants.

*Note: I’ve not done any tests on this workflow to see if working in this manner produces a “true” uncompressed result (compared to converting the RedCode data directly to an uncompressed format first, and editing with that). It is possible that somewhere along the path, probably at the Red proxy file, some amount of compression is introduced.

However, I’m not a big fan of workflows like this, because they make the process proprietary and reliant on specific software configurations. It’s entirely feasible that at some point RedCode will be treated in the same manner as formats such as DV, universally recognised, and with predictable results regardless of your editing system. Furthermore, we didn’t want to limit ourselves to having to use FCP for cutting (although ultimately that was exactly what we used). And finally, I was concerned that the cases of clips with duplicate  names would give us grief when un-picking the edit in preparation of the online.

The method we used in the end was to convert everything using the Redline command-line processor (from Red), by creating a batch script using our Redemption… database. We set everything to output 1920×1080 DVCPROHD movie files (which would retain the original timecodes) but prefixing our own reel numbers to each file. There were a couple of bugs in the Redline code (which are being addressed) that prevented the iso value of each file being read automatically so we had to plug those numbers in ourselves. It was also apparent that it was necessary to apply a gamma-curve during the conversion process to prevent everything looking flat: helpful for the online process perhaps, but not so much for the edit.

We then had a folder of movie files that could (in theory at least) be loaded into any editing system. We went with Final Cut, and an additional step was that we had to change the reel number FCP had assigned (based on the filename I assume) to our reel number.

That done, the editor took over and started to cut, as if working with any other footage. Meanwhile, we could start looking into getting the material prepped for the 4k online.

As a side note, one of the other things we were able to do easily due to the digital nature of all this was have our proprietary database automatically match the shoot notes to the Red data logs, so we were also able to produce a reference log for each clip.

The third and final part will look at using Autodesk’s Lustre to online the 4k files.

We’ve been using a bespoke Filemaker Pro database to handle all the Red Camera data that we have to deal with. It’s been really useful in getting data out of the RedCode (R3D) files and into a system we can analyse it properly. We’ve also used it in conjunction with Red’s “redline” command-line processor in order to prep both the offline and online phases of post-production.

Today I’m pleased to announce that we’re releasing the software and the Filemaker Pro source files for free, under a GNU GPL license.

You can find out more about it at http://redemption.surrealroad.com

Well, the red camera shoot, “Padded Cell” (Entitled Productions, Dir. Andrew Martin, DOP Simon Dennis), is in the can– it was a very interesting experience.

There were a few oddities about the camera itself- there is no back focus plane indicator anywhere on the camera body, so our focus puller had to manually measure where it was and mark it with a piece of tape. The record and power buttons are also very easy to press accidentally when setting up.

The camera internally numbers the clips by reel number and take. The take number is incremented every time you press record, and the reel number is incremented every time you change the flash card. However, there are a couple of things with this method that make it unreliable: first of all, the reel numbering is not consistent; there were several occasions where the number either didn’t change automatically, and even when set manually it seemed to just use whichever number it liked. The other thing is that starting and stopping the camera doesn’t necessarily imply a complete take; for instance on a few occasions we recorded a slate just before a set up, so you really need to go through and properly log everything afterwards. What all of this means is that you can potentially end up with two or more clips that have the same name (I’m looking at a few like that right now), and changing the name just in finder or windows explorer feels like a very bad idea, not least of all because the metadata is also written into the RedCode file itself, and it doesn’t appear that you can change that at all. So what I did straight away was just ignore the reel and clip numbers from the camera and number them sequentially myself. It’s also another reason why it’s critical to slate footage in some way whenever possible, or at the very least, try to log shot material soon after the shoot while it’s still fresh in your mind. We slated some of the shots in the traditional way, but I also kept a log of everything as I went along, including the reel number I’d assigned and the corresponding RedCode filename.

The only serious problem we had with the camera was right towards the end of the shoot, when we were getting errors whenever the record button was pressed, at which point it would instantly stop recording. Initially we suspected it was to do with the flash card we were using, but it turned out the camera was just overheating- 5 minutes later it was up and running again. The flash cards themselves worked well, they’re very robust as you’d suspect- all the precautions that other people have mentioned, like anti-static bags and so on, seemed completely unnecessary; in fact we didn’t lose a single bit of data from the flash cards the whole shoot.

What we did as far as saving the data was basically having a laptop with a USB flash card reader (again, contrary to what other people were saying, any old reader works fine), I would create a folder per reel (the “right” reel number as opposed to the one that the red camera said it was, and move the contents of the flash card to the folder. I moved it rather than copying it, because that way I could guarantee everything had gone across. The way Macs move files is to copy first and delete anyway, so in the event of a transfer error (there was one during the whole shoot, due to someone accidentally cutting off the power to the disk drive we were saving to…) nothing was actually lost and we could pick up where left off.

Finally each of these folders was converted to a DMG mac disk image file for various reasons, the nice thing about doing that was that you end up with a very portable read-only file per reel, which checks itself whenever you open it– and these become the original camera reels. Then it’s a matter of backing them up to tape and copying them to whoever needs them (the other nice thing abour Red is that it’s all very compact, we had just over 250GB of data for the whole shoot).

I’ve created an Automator workflow that does all of the transfer side of things automatically, but only works on Mac OS 10.5, and requires quite a powerful machine.  Grab it here… and submit feedback here…

The next part will look at preparing the rushes for the edit.

The delayed “Production 2.0″ event… took place in Soho, London last night. There was nothing to get that excited about, certainly not much on the nature of digital production… I haven’t covered previously.

The organisers presented a workflow using a Panavision Genesis that basically allowed rushes to be viewed immediately after shooting on a laptop, projector, or even an iPod. Most of this is thanks to the Codex Digital Recorder, a disk-based uncompressed video recorder that can transcode on-the-fly to a variety of different formats. All good stuff.

Also present were the Hat Factory to provide on-set VFX and editing capability, though that was very much a case of –insert VFX facility here– rather than them presenting anything in the way of innovation. Also present were transmissions bods Sohonet, though their exact role in all of this was very unclear. I would guess that if it’s your aim to bounce data around the world, that’s where they can help. I certainly wouldn’t consider them an integral part of the system though.

It was interesting to actually see it all come together in the flesh as it were. There were no apparent hiccups anywhere along the line, it seemed to work fairly smoothly (although we were practically in laboratory conditions), and I have no doubt that Codex Digital can in fact deliver on what they’re offering (although I am still waiting for the promised email to say that the footage from the event is available to view online).

Also of interest was the discussion about the workflow for the Wachowski (siblings?) forthcoming film, “Speedracer”. They made use of up to 7 Codex Digital Recorders, and their workflow was to send data off to the four corners of the Earth after each shoot, where it was colour-corrected, composited, and edited overnight as needed, and then sent back. At the dailies session the next day, the results were auto-conformed, and the production was able to watch a segment of the finished film rather than individual takes in isolation. Absolutely incredible, but I can almost feel the pain and heartbreak that the overnight crew must have gone through to make it happen.

The entire event left me wondering what the actual, tangible benefit of all of this really is. The only conclusion offered by the seminar was that it allows things to happen faster. “And it’s easy”, but as the fellow sat next to me pointed out, “of course it’s easy if you’re the designer of the system”. Because, at the end of the day, is browsing through a set of folders and files to find a shot as easy as spooling through a tape for “most” people? It remains to be seen.

UPDATE: The edited highlights can be viewed online now…