AI Image Relighting

What you see in a photo is roughly the result of what a surface is (its base color, or albedo, its shape and its material) combined with the light falling on it. Relighting means changing the light while leaving everything else alone. That is hard because a single photo mixes these factors together. Classical and early learned methods tried to separate the factors explicitly. Intrinsic image decomposition splits an image into albedo and shading. Other models estimate surface normals and depth, then re-shade the scene with a new light described by an environment map or low-order spherical harmonics. Portrait relighting research, such as Google's Total Relighting (2021), trained on data from light stages, which are domes of controllable lights that record a person under many lighting conditions. These methods can be precise but struggle outside their training data, for example with hair, glass or complex scenes. Diffusion-based relighting takes a different approach: a generative model re-renders the image under the requested lighting. IC-Light (Imposing Consistent Light) was released in 2024 by Lvmin Zhang, the developer behind ControlNet. It comes in versions conditioned on a text description of the light or on a new background image. Its key training idea is consistent light transport. In physics, an object's appearance under a mix of two lights equals the mix of its appearances under each light alone. Enforcing this during training helps the model change the lighting without changing the object's identity or details. Two misconceptions are common. First, relighting is not a color filter: a filter cannot move shadows or add highlights on the side facing a new light. Second, the model does not recover true 3D geometry. It produces plausible shading, which can fail on text, fine patterns and reflective surfaces, or when the requested lighting is physically inconsistent. Results can also subtly change details such as skin texture, so inspect outputs closely.