What a virtual production volume is
In traditional filming against a green screen, actors perform in front of a flat green surface, and the background is added later. Virtual production takes a different approach. A large LED wall, often curved and sometimes extending overhead, shows the background live on set, so actors and cameras can see and react to it as they film. The stage is often called a volume.
The wall can display a photograph, a video or a three-dimensional scene rendered in real time. Because the scene is generated on a computer, it can change on request: the time of day, the weather or the location can be adjusted between takes.
How the illusion works
A background on a flat screen looks flat when the camera moves. To avoid that, the system tracks the camera’s position and direction and redraws the scene on the wall from the camera’s point of view. As the camera shifts, the perspective of objects in the scene shifts too, which creates a convincing sense of depth.
Usually only the area the camera sees, a portion of the wall, is rendered at full quality from the camera’s viewpoint. The rest of the wall shows a lower-detail version and works mainly as lighting. The result is that a camera pointed at the wall sees a believable environment, while the surrounding screens light the scene.
The wall as a light source
LED walls emit light, and in a volume that light falls on the actors, the costumes and the props. This can be an advantage. A sunset on the wall casts warm light on a person standing nearby, and reflections in glossy surfaces, such as glasses or a car body, show the environment. Matching lighting and reflections is difficult with green screen, where green spill is often a nuisance.
It is also a constraint. The wall is not a precise studio light, so crews often add conventional lamps. How accurately the wall’s colors match the camera and the scene matters, as does the white point and color temperature, since the wall’s tint shows up on skin.
What cameras ask of the display
A camera sees a screen differently from a human eye, and several display characteristics matter more than they do on a lobby sign.
Refresh and synchronization. LED walls refresh in cycles. A camera that captures frames at certain times can catch part of a cycle, producing bands or flicker. To avoid this, the wall and camera are typically synchronized, and the wall is run at a refresh rate high enough to avoid visible artifacts. This overlaps with the issues in screen flicker and PWM dimming, which can show up on camera even when it is invisible to the eye.
Moiré. When the camera focuses on the wall, the fine grid of pixels can interact with the camera sensor’s own grid and produce wavy patterns. Crews usually keep the wall out of sharp focus, and a finer pitch helps. The relationship between pitch and distance is covered in the article on pixel pitch.
Color range and consistency. Cameras record color and brightness data that a display must match. A wall with a limited color gamut may not reproduce the colors in the scene, and mismatches between modules show in the footage. Calibration is more demanding here than for signage. Greater bit depth also reduces banding in smooth gradients, such as a sky, which cameras can exaggerate.
Brightness and contrast. High dynamic range content on the wall helps lighting look natural, but the wall must handle both bright highlights and dark scenes without washing out. Black levels are affected by room light falling on the wall.
Viewing angles. The camera may view the wall from an angle, and so may the actors’ eyes. If colors or brightness shift with angle, the image will vary across the frame. Walls designed for this work are chosen partly for stable color across the angles used on set.
Latency. Rendering a scene from the camera’s position and sending it to the wall takes time. If the delay is too long, the background lags behind camera movement. Reducing the delay is a design goal for the whole chain, not just the display.
Panels and roofs
Many volumes use a curved main wall with an overhead section, built from direct-view LED cabinets. This allows the wall to wrap around the set. A ceiling panel can serve as a light source from above and supply reflections. In smaller setups a flat wall behind the talent may be enough, as in broadcast studios and event stages.
What it asks of a crew
Using a volume changes the work. Scenes must be built before shooting, because the backdrop is part of the take. Calibrating cameras, tracking and wall takes time, and technicians keep an eye on the system during filming. Each production weighs the costs of building and operating the volume against the benefits of reduced location travel and post-production work. The balance varies by project, and a volume is not the right tool for every shot.
Beyond film
Similar walls are used in broadcast, live events and concerts, where visuals are part of the staging, and the same display considerations return in simpler form. The article on event video walls covers the marketing side of that use. The point to remember is that the specifications that matter for a camera, such as stable refresh, reliable color and low flicker, are different from, and often stricter than, those for a screen viewed only by people.