In-camera visual effects in virtual production (VP) use LED walls to provide background imagery and image-based lighting. While this enables on-set compositing, it tightly couples lighting, background and scene appearance, limiting flexibility for downstream editing. Contributing to this, we propose a VP-specific framework for 3D reconstruction and relighting using Gaussian Splatting. Similarly, research on inverse rendering relies on physically-based rendering frameworks that jointly estimate 3D geometry and lighting, with environment maps. However, the maps are typically low-resolution and assume far-field lighting. In VP, with near-field and high-resolution image-based lighting, this leads to inaccurate results as well as a non-trivial editing process. Instead, our framework leverages the known background imagery to condition the relighting process. This avoids relying on environment maps and reduces compositing to a simple background-image editing task. To drive our framework, we introduce a capture process and dataset that captures real VP scenes under varying background content and illumination. This footage is used to decompose a 3D scene into fixed appearance and variable lighting components. The variable lighting process simulates light transport by parameterizing each primitive with a learnable UV coordinate, intensity value and resolution modifier. Using mipmaps, these parameters directly sample the background texture in image space - implicitly capturing reflections and refractions without having to rely on physically-based rendering processes. Combined with the fixed appearance, this allows us to render relit scenes using the vanilla Gaussian Splatting rasterizer. Compared to the baselines, our implementation achieves high-quality 3D reconstruction and controllable relighting. It is efficient (less than 3 GB RAM, less than 5 GB VRAM, less than 2 hours training, ~35 FPS) and supports rendering useful AOVs including depth, lighting intensity, variable lighting color, and unlit renders. A subjective study also demonstrates the practical benefits of using our framework for photorealistic relighting in post production.
We propose a VP-specific framework that avoids physically based IR. We introduce a multi-view, multi-illumination dataset that captures VP scenes under varying LED wall content, and leverage the known background imagery to decompose Gaussian Splatting appearance into fixed and variable lighting components. Variable lighting is modeled by assigning each Gaussian a UV coordinate sampling the background texture, modulated by a learnable per-Gaussian intensity parameter. Combined with a base color representing fixed appearance, this enables relighting through direct manipulation of the background image. Our method achieves high-quality 3D reconstruction and controllable relighting without requiring geometry, normals, or environment maps. It is efficient ( less than 3 GB RAM, less than 5 GB VRAM, les than 2 hours training, ~35 FPS) and supports rendering useful AOVs including depth, lighting intensity, variable lighting color, and unlit renders.