Having worked with DITs (the horrible acronym for Digital Intermediate Technician, the person responsible for the delivery of digital media from the set to post) over many years, and more recently, doing the job of a DIT on a couple of productions, I’ve come to one very simple conclusion: the process could be better.

That isn’t to say that most DITs do a terrible job; on the contrary, many are extremely passionate and conscientious. Rather it’s that they approach the role from a cinematography standpoint, and almost never from an IT background. Historically, most DITs were barely-trained people provided by the rental company, who knew how to operate a particular digital camera, and how to copy files on a Mac.

Problem 1: data transfer

GUI-based copying (whether through Windows or Mac), has several flaws:

• It batches copy operations together into one lump. If a single file fails, you have to restart the batch
• It doesn’t verify files
• It’s slow
• It’s prone to human error
• It doesn’t handle errors very well
• If it does encounter an error, it will stop the entire process
• Having multiple copy operations tends to cause problems

and on and on.

Solution 1: rsync

Rsync is a universal file copying application that runs from a command-line. It’s even built-in to the Mac OS. It’s extremely robust, and recommended in many situations. It’s startlingly easy to use, and there are even GUIs available for it.

Open terminal, and type:

rsync -avhP /Volumes/A001_C001/ /Volumes/RushesStorage/Day01/

And watch it go. You can cancel with control-c and run the same command to resume it. I defy you to find anything that will copy files faster. The only thing to pay attention to is the trailing slash at the end of the paths. More information on rsync.

Problem 2: rented storage media

Every production I’ve worked on has rented storage media (SD cards, CompactFlash Cards, LaCie drives and so on), and needless to say, every one of them has had at least one take that had to be aborted due to an error with said media. Disk drives are sensitive things at the best of times, and this is one of the reasons used disk drives lose value.

Flash storage, which is most popular with digital cameras for its high performance is especially problematic. The problem is, it can also be prohibitively expensive.

Solution 2: buy storage media when possible

The real solution here is proper planning. There is rarely a good argument for renting storage media. Cost is certainly a factor, but question what the cost is of having to reshoot due to drive failure? Also, consider planning to sell the media after a shoot, recouping much of the cost (and probably putting it on par with the cost of renting the media in the first place).

Problem 3: daily storage system

Low to mid-budget productions all use external (USB or Firewire) disk drives for immediate transfer of rushes by the DIT. On a given day, the camera(s) will produce anywhere from a few GB to 1 (or more) TB of data, which gets dumped onto these drives for an undetermined amount of time.

There’s nothing inherently wrong with this, but we can do better.

Solution 3: network attached storage

Rather than using a stack of G-RAIDs, consider using one or more network attached storage (NAS) systems. There are so many benefits to this, it’s incredible everyone isn’t doing it:

• Data is better protected, instantly
• All the data is available without having to swap through drives
• Data is accessible to multiple people, simultaneously
• You can run other services (like a media server)
• Data can be encrypted easily
• It’s lighter
• It’s almost the same cost*

I’ve used the following kit on the last two productions, giving me all the benefits above:

• QNAP 5-bay NAS (TS-569 Pro II) ~ $1000
• 5 x 3 TB hard disk drives ~ $650
• Generic gigabit ethernet hub

This will give you 9 TB of RAID 6 (meaning you can lose 2 disks). Why sacrifice 6 TB of storage space? From Wikipedia:

RAID 6 provides protection against data loss during an array rebuild, when a second drive is lost, a bad block read is encountered, or when a human operator accidentally removes and replaces the wrong disk drive when attempting to replace a failed drive

The QNAP is considered reasonably high-end (it even has HDMI out, imagine that!) by NAS standards, and so if price is a concern, there are many cheaper options available. I personally recommend the QNAP because it has decent Mac support, and is very user-friendly.

* Compare this to the equivalent in G-Technology Disks: 5 x 2 TB G-RAID ~$1450. If you opt not to use RAID storage, you then of course get more storage for your money with the NAS, and the difference in price becomes negligible.

And if you want to be really paranoid about backing up that data (as well you should), then you can just get an HDD dock, some extra internal hard drives, and copy data onto those as well.

Problem 4: inefficient workflow

Every DIT I’ve ever worked with does things the same way:

1. Receive camera media
2. Plug into laptop
3. Copy to external media
4. Return camera media
5. Make second copy to other disk drive
6. Wait for copying to complete
7. Watch media directly from external drive
8. Give one of the disk drives to director for rushes

Solution 4: more efficient workflow

With a NAS setup, here’s a better approach:

1. Receive camera media
2. Plug into NAS
3. Tell NAS to start copying to itself and/or other connected drives (yes, you can still use rsync in most cases)
4. Return camera media
5. Everyone who wants to watch needs only connect via ethernet

This now gives rise to a new problem:

New problem: DIT has too little to do

Solution: Do something useful and beneficial, like logging all that data as it’s being shot.

Synaesthesia, our film production & logging software has been updated. The latest version fixes a number of issues with shooting mode that were discovered on a recent multi-camera shoot, where Synaesthesia’s shooting mode was used to retrospectively log their clips (more on that at a later date).

The update adds the ability to apply a “template” to file paths in exported documents (accessible from the preferences window). So you can now for example, have Synaesthesia automatically change all file paths from  /Volumes/Data/file.mov to n:\file.mov if you need to send the file to a Windows user.

Finally, we’re announcing a permanent price reduction of Synaesthesia. Effective immediately, the new price will be $199 per user license (anyone who purchased a license in the last 30 days will be credited the difference).

Get the new version here.

Synaesthesia is the new software product by Surreal Road. It’s been in development for around four years now, and is almost at a point where it is production-ready.

But what is it?

Having worked on all sorts of film and TV productions in different capacities (of greatly varying budgets), it often amazed me how “disconnected” every role seems. This is especially true in areas like post-production, where people employed to enhance or otherwise change particular shots would do so without any knowledge of the history of that shot. It might be possible to find out the camera and lighting setup used for a particular shot in some cases, but what about the intent behind that setup? What was the cinematographer aiming for, and how can I better enhance that, as opposed to the more usual practices of (at best) attempting to reverse-engineer a shot in order to understand it, or (at worst) changing things in a more haphazard manner until something looks good.

This was a problem I’ve encountered on almost every production, and in part it’s unavoidable. The reality is that just as the writers are often left outside the gates of production, so too are the production crew long-gone when it comes to post. This also becomes a practical and logistical problem. Where is a particular reel of film? What was the time and date of a particular shot? On a very organised production, it is likely that the editor would be armed with most of this information, but in all other cases, there is simply no one around to ask.

I look at software created for the visual effects industry, and it is staggering: the functionality and capabilities of VFX software is advanced to the point where it’s possible to use these tools to quickly create shots that are indistinguishable from reality. But when it comes to the actual production process, we’re in a technological drought. Even popular writing software, such as Final Draft, is only slightly more useable than TextEdit, even with years of industry experience and development put into it. What was I supposed to use in my capacity as data manager on various things to stay on top of everything? Excel?

The solution of course, is that (those computer-savvy enough) people tend to cobble together some sort of database (usually in the ubiquitous FileMaker Pro) which serves the immediate needs of the production. Much of the time this works out rather well, the production ends up with a bespoke system that covers most of the bases, something “good enough”. But what about those people who haven’t the time or the resources to create something from scratch, or those people who just want to hit the ground running? Well, you are who Synaesthesia was designed for.

At its core, Synaesthesia is about keeping track of things about a production, from start to finish. Here’s a typical scenario:

1. You have a production. You add notes, storyboards, descriptions of characters, of sets, all to get a sense of what it’s about.
2. At some point you have a screenplay. You import that and it links all the scenes with sets and characters you’ve previously created, and adds anything that’s missing.
3. You refine the script, importing new versions as you go along, further fleshing out what you want to shoot and so on.
4. You create a database of people and equipment  you’re going to need, and assign them to different parts of the production.
5. You start shooting. You log what’s shot as it happens, along with notes, things like whether the take was good or not, what was recorded and making last-minute script updates.
6. You import data directly from digital footage (such as RED camera footage), in order to accurately log timecodes, and shooting parameters.
7. You start editing, having access to all your previous notes for each clip of footage that was shot. You can import sequences from an editing system and have Synaesthesia tell you which shot is used where. You can make changes to the edit from within Synaesthesia, and save those back to your editing system.
8. You can designate certain shots as needing effects work, and update those shots as new effects versions are completed.
9. Finally you can archive all the reels of footage, noting their locations, in case they’re ever needed again.

That’s quite a broad overview, and it assumes you’re going to use Synaesthesia from start to finish. But perhaps the best part of it is that you don’t have to. Maybe you’re only concerned with pre-production, and just want a place to keep storyboards, concept art, and screenplay versions organised? Maybe you just want to log continuity during a shoot? Or maybe you just want to tweak a couple of edits? Well then, Synaesthesia can help you.

It’s probably also helpful to mention what Synaesthesia (at least, in its current form) isn’t for:

• It’s not for budgeting or scheduling
• It’s not a replacement for software such as Final Draft
• It’s not a replacement for systems such as Final Cut Studio
• It’s not a server-based system, (it’s not possible for multiple people to make changes to the data at the same time).

A more detailed list of features is available  here. As I’ve said, Synaesthesia isn’t quite finished yet. It’s capabilities are still being worked out. But there are several key principles that we’ll always try to adhere to:

• It will be simple to use
• It will integrate with software you already use
• It will give you the information you need

But more than anything else, I want it to be for whatever you (the user) need. With that in mind, we will be inviting people to try out pre-release versions in order to tell us what you like, what you don’t, and what’s missing. You can sign up for an invitation here.

We’ve been using a bespoke Filemaker Pro database to handle all the Red Camera data that we have to deal with. It’s been really useful in getting data out of the RedCode (R3D) files and into a system we can analyse it properly. We’ve also used it in conjunction with Red’s “redline” command-line processor in order to prep both the offline and online phases of post-production.

Today I’m pleased to announce that we’re releasing the software and the Filemaker Pro source files for free, under a GNU GPL license.

You can find out more about it at http://redemption.surrealroad.com

Well, the red camera shoot, “Padded Cell” (Entitled Productions, Dir. Andrew Martin, DOP Simon Dennis), is in the can– it was a very interesting experience.

There were a few oddities about the camera itself- there is no back focus plane indicator anywhere on the camera body, so our focus puller had to manually measure where it was and mark it with a piece of tape. The record and power buttons are also very easy to press accidentally when setting up.

The camera internally numbers the clips by reel number and take. The take number is incremented every time you press record, and the reel number is incremented every time you change the flash card. However, there are a couple of things with this method that make it unreliable: first of all, the reel numbering is not consistent; there were several occasions where the number either didn’t change automatically, and even when set manually it seemed to just use whichever number it liked. The other thing is that starting and stopping the camera doesn’t necessarily imply a complete take; for instance on a few occasions we recorded a slate just before a set up, so you really need to go through and properly log everything afterwards. What all of this means is that you can potentially end up with two or more clips that have the same name (I’m looking at a few like that right now), and changing the name just in finder or windows explorer feels like a very bad idea, not least of all because the metadata is also written into the RedCode file itself, and it doesn’t appear that you can change that at all. So what I did straight away was just ignore the reel and clip numbers from the camera and number them sequentially myself. It’s also another reason why it’s critical to slate footage in some way whenever possible, or at the very least, try to log shot material soon after the shoot while it’s still fresh in your mind. We slated some of the shots in the traditional way, but I also kept a log of everything as I went along, including the reel number I’d assigned and the corresponding RedCode filename.

The only serious problem we had with the camera was right towards the end of the shoot, when we were getting errors whenever the record button was pressed, at which point it would instantly stop recording. Initially we suspected it was to do with the flash card we were using, but it turned out the camera was just overheating- 5 minutes later it was up and running again. The flash cards themselves worked well, they’re very robust as you’d suspect- all the precautions that other people have mentioned, like anti-static bags and so on, seemed completely unnecessary; in fact we didn’t lose a single bit of data from the flash cards the whole shoot.

What we did as far as saving the data was basically having a laptop with a USB flash card reader (again, contrary to what other people were saying, any old reader works fine), I would create a folder per reel (the “right” reel number as opposed to the one that the red camera said it was, and move the contents of the flash card to the folder. I moved it rather than copying it, because that way I could guarantee everything had gone across. The way Macs move files is to copy first and delete anyway, so in the event of a transfer error (there was one during the whole shoot, due to someone accidentally cutting off the power to the disk drive we were saving to…) nothing was actually lost and we could pick up where left off.

Finally each of these folders was converted to a DMG mac disk image file for various reasons, the nice thing about doing that was that you end up with a very portable read-only file per reel, which checks itself whenever you open it– and these become the original camera reels. Then it’s a matter of backing them up to tape and copying them to whoever needs them (the other nice thing abour Red is that it’s all very compact, we had just over 250GB of data for the whole shoot).

I’ve created an Automator workflow that does all of the transfer side of things automatically, but only works on Mac OS 10.5, and requires quite a powerful machine.  Grab it here… and submit feedback here…

The next part will look at preparing the rushes for the edit.