Skip to main content

Reality Camera

Reality Camera

Reality Camera bridges the physical and virtual worlds by combining camera tracking, real-time compositing, advanced rendering controls, and AI-assisted performance optimization. It provides the tools required to accurately align virtual content with live video while maintaining cinematic image quality and giving operators precise control over how graphics appear in the final output.

Reality Camera

  • Foreground Pixels Video Mask Foreground Pixels Mask uses depth information from the live video to identify foreground virtual elements and composite them back correctly with keyed talent. It preserves solid and translucent pixels in a single render pass, enabling foreground graphics, reflections, refractions, bloom, and lens flares to blend naturally with the final image. All of this happens in that single render pass.

Foreground Pixels Mask

  • DLSS-assisted performance and image quality NVIDIA DLSS helps Reality 5 maintain real-time performance in visually demanding scenes while improving image quality through AI-based upscaling and ray reconstruction. This is especially valuable in Lumen GI and ray-traced workflows, where it can help reduce noise, flickering, and unstable reflection artifacts.

NVIDIA DLSS

  • Tracking Protocols Tracking input in Reality Camera supports industry-standard tracking protocols and synchronizes physical camera movement with the virtual camera in real time. This helps ensure that virtual graphics, AR elements, and virtual studio environments remain accurately aligned with the live camera feed.
  • Center-Shifted Zoom Lens Support Reality Camera supports broadcast zoom lenses with center-shifted rendering, allowing lens behavior to be matched more accurately while eliminating the need for traditional overscan-based workarounds. This helps preserve image quality and makes camera-aligned rendering more efficient for live virtual production.
  • Camera Projection Projection is how Reality Camera lines up the live camera feed with the 3D virtual world. It uses the tracking data from the physical camera (its position, angle, and lens zoom) to “wrap” the video image onto a defined 3D geometry in the virtual set, so graphics placed inside that space stay locked to the real picture as the camera moves. This is what keeps virtual elements from sliding or drifting off the real footage during a live shot.

Camera Projection

  • AR Ambient Occlusion (AO) Ambient occlusion is the soft, natural shading that appears wherever two surfaces are close together — like the subtle dark line where a wall meets the floor. It’s what stops a 3D scene from looking flat. Reality Camera applies this same shading to virtual graphics placed over live video, so objects sit into the real scene instead of looking pasted on top. It works alongside the Shadow and Reflection passes and can be masked to specific areas, even in real-time ray-traced AR scenes.
  • AR Shadow Pass The shadow pass is a separate layer that contains just the shadows virtual objects cast. Reality Camera blends this layer back into the live video so virtual graphics cast believable shadows onto the real scene, and operators can turn the shadow strength up or down in real time rather than it being baked in permanently.
  • AR Reflection Pass The reflection pass is a separate layer showing how real objects would reflect off virtual surfaces, like a shiny floor. Reality Camera can generate this directly using tracking and depth data, so reflections of the virtual objects appear correctly in virtual reflective surfaces without needing a second dedicated camera or rendering pass just for reflections.
  • Bloom and Lens Flares Preserve and control Unreal Engine post-process effects such as bloom, lens flares, and other camera-based visual enhancements in the final output. Unlike traditional key-and-fill workflows where these effects can be hidden behind keyed talent, Reality composites them in 3D space and reconstructs the final image using dedicated render passes. This allows production teams to retain cinematic visual effects and adjust their contribution in real time, ensuring virtual elements integrate naturally with the live scene.

Bloom Pass