=Paper=
{{Paper
|id=Vol-233/paper-24
|storemode=property
|title=IP-Racine metadata integration in the digital cinema workflow
|pdfUrl=https://ceur-ws.org/Vol-233/p49.pdf
|volume=Vol-233
|dblpUrl=https://dblp.org/rec/conf/samt/BlatMH06
}}
==IP-Racine metadata integration in the digital cinema workflow==
https://ceur-ws.org/Vol-233/p49.pdf
IP-Racine metadata integration in the digital
cinema workflow
Josep Blat, Benoit Michel, Werner Haas and the IP-Racine consortium
Abstract—IP-Racine is an European commission funded re- III. PRACTICAL IMPLEMENTATION
search program addressing metadata compatibility within the
digital cinema chain. Results of those efforts are visible at the The camera rig is fitted with sensors and sends pan, tilt,
shooting stage with optimized camera parameters transmission to zoom, lens focus, aperture, and Cartesian coordinates (option-
on-set postproduction equipment. ally using GPS) through a serial link to the Grass Valley Viper
camera. The camera inserts the positional metadata in the HD-
I. INTRODUCTION SDI video stream connection. The image and metadata stream
T he digital cinema workflow is extremely complex. At
each of the numerous steps in the movie making chain,
metadata are produced, stored and transmitted. Unfortunately
is then recorded on a DVS Clipster workstation. At the post-
production stage, various processes are achieved such as
compositing the incoming images with synthetic background
at each step lots of those metadata are ignored at the input, or props generated by the eStudio software from Brainstorm.
dropped during a file format conversion or simply lost. When The final images are produced off line but are of very high
everything was analog, the process was slow enough that peo- quality.
ple were able to transmit the image related information by
hand or by fax. But with digital cinematography, we are modi-
fying images in real time and the old models become unsatis-
factory. IP-Racine addresses the problem of real time trans-
mission and use of metadata right at the beginning: within the
camera.
II. METADATA IS KEY TO REALTIME FEEDBACK IN VIRTUAL
STUDIO SCENES SHOOTING
A typical ‘effects’ movie is now shot with actors recorded
Figure 1: Acquisition workflow with real time
in front of a green screen and the background is added digi-
synthetic background feedback
tally at a later stage. To synchronize both pictures positional
data from the camera rig are transmitted to the computer creat- At the same time, the HD-SDI video feed and the metadata
ing the background. The best way to ensure coherency is ob- are transmitted to a hardware chromakeyer inserting the im-
viously to included the positional metadata from the camera ages over a real time generated background similar to the final
rig inside the image data stream as ‘embedded metadata’. The one. The resulting images are displayed on stage on a control
enhanced video signal from the camera may be used subse- monitor. Lower level of details and synthetic lighting condi-
quently by high quality virtual studio software to produce a tions allow a real time rendering of the computer generated
composite image with the real actors and the synthetic back- background matching the positional data of the camera. In
ground perfectly synchronized. The same signal can be used order to perfectly synchronize the camera position and the
directly by a simpler preview computer to display real time virtual background, a small delay is inserted in the camera
images for the director of photography, giving the feedback video stream. This delay is adjusted to the exact duration of
necessary to assert the validity of the produced content. Obvi- the real-time background image generation.
ously, this kind of feedback is a major improvement over tra-
ditional methods where dailies were available only the next IV. METADATA STANDARDS
day. In the digital cinema production workflow a number of het-
erogeneous types of metadata exist, many of them are repre-
sented in specific formats and standards. The scenario de-
scribed here is a practical example of metadata transmission
Work is supported by the European commission under the FP6-IST-511316 contract.
and use at the beginning of the digital cinema chain. IP-
J. Blat is with University Pompeu Fabra, Barcelona, Spain and is the coordinator of Racine is developing and testing transmission of the camera
the IP-Racine project.
B. Michel is responsible for dissemination inside the IP-Racine project. (phone: metadata and other metadata generated at later stages within
+32 498 93 70 58; e-mail: benoit@michel.com). the images themselves. One of the most commonly used im-
W. Haas is with Joanneum Research Institute, Austria.
age formats is
V. APPLICATIONS
A. Metadata Visualization and Annotation
A number of different metadata are created during the digi-
tal cinema production process. Further descriptions are gener-
ated using automatic content analysis tools (shot boundary
detection, camera motion estimation, re-detection of similar
objects, key frame extraction). In order to visualize and mod-
ify the various metadata, which are often stored in a number
of different formats, the WMO tool and its extensible set of
plug-ins can be used to access the metadata descriptions. The
software includes a video player which synchronously dis-
Figure 2: A Grass Valley Viper camera recording a plays the metadata associated with the images.
dancer in front of a green screen background.
B. Browsing Media Collections
1
the single image DPX format whose origin can be tracked In postproduction environments, users deal with a large
back to the pioneering Cineon film scanner from the early amount of images, such as newly shot scenes, archive material
nineties. The DPX format allows the use of several metadata and computer generated sequences. A large portion of the
fields but the exact way of handling those fields vary between material, such as production rushes, is unedited and often re-
equipment manufacturers. dundant. Typically only few metadata annotations are avail-
An important prerequisite for an integrated metadata work- able. To support users in navigating and organizing such con-
flow as envisaged in IP-Racine is thus to establish interopera- tent, IP-Racine develops a summarization and browsing tool.
bility between the different metadata formats that exist The tool uses the available metadata from the production as
throughout the workflow. This includes different types of well as automatic content analysis to filter and cluster the con-
metadata from low-level (e.g. camera position) parameters to tent collection. The presentation of the media items and clus-
high-level textual descriptions. ters is based on key frames extracted from the content. By
Within IP-Racine, components for visualization, editing and selecting relevant clusters and changing the filter and sort cri-
conversion of metadata are being developed. The Workflow teria, the user refines a query which narrows the selection and
Metadata Organizer (WMO) tool developed by Joanneum locates relevant media items.
Research serves as an interface to these components and can C. Data and Metadata Management
be used to visualize and modify the metadata that are available
A high degree of transparency in data management is essen-
throughout the workflow.
tial in the film production business. In IP-Racine DVS Digital
The metadata formats to be considered are the headers of
Video Systems GmbH develops an application called Spy-
commonly used image formats, such as DPX, the elements of
cer™.
the SMPTE Metadata Dictionary (SMPTE RP-210), MPEG-7
[1] and MXF DMS-1 [2], as well as proprietary formats com-
monly used in production. While embedding of metadata into
essence containers is useful during capture, metadata are
stored separately in the postproduction content management
system in order to allow for efficient search and conversion.
IP-Racine will issue recommendations to the industry and
to standard bodies in order to promote metadata interoperabil-
ity between the various postproduction steps. We hope that
the above recommendations and the verification tool will be Figure 3: The DVS Spycer™ data manager
soon adopted by as many manufacturers as possible. An in-
Spycer collects production relevant metadata from media
crease in efficiency in movie production and postproduction is
files automatically and provides this information to a search
expected. A report on standards for digital cinema metadata
tool. Several Spycer™ applications build up a distributed con-
and their implementation is available on the IP-Racine project
tent management network, called SpycerNet, for easy and fast
web site at
file retrieval.
http://www.ipracine.org/documents/documents.html.
Another IP-Racine document of interest is Digital Cinema VI. REFERENCES
Perspectives, a state-of-the-art book covering all aspects of [1] Information Technology—Multimedia Content Description Interface.
production, postproduction, distribution and exhibition as seen ISO/IEC 15938:2001.
by key people of the industry. [2] SMPTE, Material Exchange Format (MXF) Descriptive Metadata
Scheme - 1, SMPTE 380M-2004.
1
[3] Digital Cinema Perspectives, September 2006, ISBN 2-87111-029-8,
DPX stands for Digital Picture eXchange, an SMPTE image format described in the available at www.i6doc.com.
“ANSI/SMPTE 268M-2003 Standard for File Format for Digital Moving-Picture Ex-
change” document.