I’ve never come across a perfect dust-busting system.

Ok, that isn’t really saying much; it’s an inherently difficult logistical problem to solve. However, what I mean is, there is just no simple system that works adequately, designed for maximum speed and efficiency. It seems that all the software developers are
trying to churn out new algorithms to better spot and fix dirt, rather than to design a system that helps the operator to fix dust manually.

Sure, but that’s because automatic systems are inevitably faster than going through each frame by hand, right?

Well, it depends. Provided I don’t have to wait for data transfers, on an eight-hour shift I can personally fix between 12-20,000 frames of footage (depending upon what kind of state it’s in). I don’t know of any software that gets close to that (but I’d like to hear from anyone who does). It’s not that my brain can analyse images faster than a computer, it’s just that I can do it far more accurately and with less trial and error.

Automatic systems, such as Thomson’s Shout, require the operator to specify a set of parameters to control how the machine detects dirt, and these must be set separately for each new shot, because many factors in the images will contribute to determining what constitutes dirt. For example, in an outdoor scene, raindrops might be detected by the software as dirt, and then eliminated, effectively destroying the image. Most systems require multiple passes- once to build a motion estimation map for each frame, and again to actually detect individual defects. Unfortunately, even this process does not work perfectly, and operators are then required to go through the footage to check the changes made. And as far as I can tell, the checking process is at least as long as the time taken to just fix problems by hand in the first place (at least humans don’t require motion estimation passes), and may even take longer, because now you’re not just looking for stray dirt, you’re also looking for false positives in the fixed information, as well as artefacts created by the fixes. And as far as I can tell, this will always be the case.
On the other hand, an operator goes through everything once and it’s done. If anything gets missed, it can be fixed later with relative ease. To my mind, it makes more sense to support this paradigm for the time being than to keep flogging the proverbial horse.

So how can this simple process be improved?

Well, first of all, a manual paint system must be geared for speed. Playback must be at least 1/3rd real-time, no matter what the resolution. If this means working with proxies for viewing processes, so be it. Anything that takes "foreground time"- i.e. time that the operator has to spend waiting for something to happen, should be avoided at all costs. This might mean running processes such as caching or rendering, in the background.

As for the interface, the display area should be as big as possible, whilst having access to the following tools:

• clone from previous frame
• clone from next frame
• clone from current frame
• clone from specified frame
• interpolate region from surrounding pixels (for correcting scratches)

And yes, each should constitute a separate button, I know it’s not tidy, but it sure is faster. Digital paint is a fairly monotonous process, and having to click multiple buttons to change the clone source each time can be tedious. There should also be some easy method to change the cloned area, (or brush size). Discreet’s Fire system has a very elegant method for doing this: you simply hold down a key on the keyboard, and drag the stylus, or mouse. [Incidentally, a stylus and tablet is much more suited to this kind of work than a mouse, which, in addition to potentially causing RSI, is much slower for making quick fixes than just dabbing at a tablet with a pen.] Other paint tools, such as burn/dodge, could be added, but are mostly superfluous for the bulk of the dust-busting work.
There needs to be an option to change the display, either using specified LUTs, or to ramp up the display contrast and change the brithness, which is very useful for trying to spot the dust in the first place. Many dust-busting operators wrongly believe that they need to look at the footage either in its raw state, or else with the same LUT as the production uses for grading. In fact, all that is needed is to use a display system that makes all the defects stand out. As the vast majority of defects either block light completely (seen as white when the defect is on a negative, and black on a positive), or let it all through (the opposite result), the mid-range of luminosity is much less important in trying to spot dust.
The system should log changes made to each particular footage, and if possible, save the changes on additional layers, or at least embed data into separate files (or into the file header) that allows changes to be undone to the footage, no matter how far down the digital intermediate pipeline they have gone.
Even better, if the paint system can read information from the conforming and QC systems, it may even be possible to, say, have the conform operator highlight defects he or she spots directly on their system, and have that information fed back to the dust-busting system, allowing the operator to quickly make a fix.

Maybe the future will bring such a system to the digital intermediate process. If I had the inclination (to say nothing of the programming experience) I would look into this myself. In the meantime though, I hear rumours of infra-red scanning passes that should be able to create dust mattes as the footage is being scanned, so hopefully this will aid the automated processes somewhat.

Comment from dave - 3/23/2005 at 3:10 pm

I definately find myself erring on the side of manual dustbusting. I do it on flame and being able to playback the clip is extremely important especially when dealing with long scenes.