De-lighting

De-lighting is removing the lighting that a photograph has recorded, so that only the surface colour is left. A photo of a rock contains the rock and the sun, sky, shadows and bounce light of that afternoon. A base colour map must contain only the rock, because the engine adds its own light. Unity's guide puts the problem simply: raw photogrammetry textures "often contain a lot of lighting information that you should remove so as to obtain the true base color for rendering" (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity).

It matters for every photo-sourced material and photogrammetry scan; those pages cover capture, this one the de-lighting step.

#What de-lighting removes

A lit texture carries several kinds of light at once. Unity's De-Lighting Tool works through them in order: "Environment lighting reconstruction", "Image-based de-lighting", "Ambient and global illumination lighting removal", and "Exposure compensation and color correction" (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity). In plain terms: sun and cast shadows, soft sky light, darkening in crevices (ambient occlusion) and bounce from nearby surfaces, specular highlights, and the camera's own exposure and white balance.

#Three approaches

#1. Avoiding the light at capture

Light that is never recorded does not have to be removed later. Unity's tool documentation says that strong direct lighting such as the Sun should be avoided during capture, and that "Most of the time, strong cast shadows won't be removed from the texture" by the tool (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity). Flat, overcast light leaves only soft occlusion to fix. Cross-polarised flash goes further and removes specular reflections before they reach the sensor (paulbourke.net: Cross polarisation in photography).

#2. Reconstructing the light from the scan

Software can estimate the lighting from the scan itself and divide it out. Unity's free De-Lighting Tool needs the lit colour texture plus baked AO, normal and bent normal maps, which you can bake in xNormal, Knald or Substance Designer (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity). Instead of needing an HDR capture of the environment, it "extracts environment lighting directly from the original lit texture" (80 Level: Photogrammetry: De-Lighting Tool for Unity). It uses AO to decide how much of each pixel's light is local (bounce from the ground or the object itself) and how much comes from the wider environment (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity).

Two optional inputs improve results (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity):

• A position map, so de-lighting happens in 3D and avoids seams at UV borders.
• A mask map. Red marks parts that are a different material, such as moss, snow or paint on a rock, or areas the reconstruction got wrong. Green forces local de-lighting where AO alone is not enough.

If a colour checker was in the shot, a reference zone placed over it calibrates exposure and colour towards the real albedo (Unity-Technologies/DeLightingTool on GitHub: De-Lighting in Unity). Unity published it as an expert guide alongside a photogrammetry workflow for "authoring game assets on an affordable budget" (Unity Manual 2018.1: Expert Guides).

Agisoft De-Lighter is a free standalone tool for Windows, macOS and Linux that needs only the textured model: you sketch lit and shadowed areas with brush strokes, and it removes cast shadows and AO from the colour (CG Channel: Download free texture de-lighting tool…).

#3. AI filters

Substance 3D Sampler's Delight (AI Powered) filter removes "lighting information from the base color channel" and has no parameters: "it works automatically" (Substance 3D Sampler documentation: Delight). Adobe recommends adding it at the top of the layer stack after the material is extracted and made tileable, and notes that Sampler's AI-powered Image to Material already "includes a delighting pass", so you should not stack both (Substance 3D Sampler documentation: Delight).

#Related terms

cross-polarisation · photo-sourced and scanned materials · photogrammetry · base colour (albedo) map · ambient occlusion · physically based rendering · photogrammetry and scanning tools

#Further reading and watching