GLEAM captures the light already present in a scene and turns it into a usable illumination model, allowing mobile AR objects to share the visual character of their physical surroundings.
Introduction: Lighting That Belongs in the Scene
Virtual objects only feel integrated into augmented reality when their appearance responds to the physical environment. GLEAM estimates real-world illumination in real time, making reflective and light-sensitive materials such as glass, water, and metal more believable on mobile devices.
System Architecture: Radiance Samples to a Scene Cubemap
GLEAM observes reflective objects in the device camera to gather environmental radiance samples. It combines radiance sampling, optional transfer across devices, and cubemap composition to estimate lighting from all directions and apply it to virtual content.
Results: Balancing Fidelity and Adaptiveness
GLEAM can operate across a spectrum of update rates and cubemap resolutions, enabling responsive lighting for fast-changing scenes or higher-fidelity estimates when conditions allow. In user testing, the framework outperformed standard mobile AR lighting tools.
