Photogrammetry

Photogrammetry builds a 3D model, with its colour texture, from many overlapping photographs of a real object or place. Software finds the same features in different photos, works out where each camera was, and reconstructs the surface. For games it is a way to get real-world shape and colour detail that would take days to sculpt and paint.

DICE's artists, who used it across Star Wars Battlefront, called it "not a magic bullet but instead a tool like any other" (EA Frostbite: Photogrammetry and Star Wars Battlefront). This page covers the pipeline and the cases where photogrammetry helps or does not.

#The pipeline

1. Capture. Photograph the subject from every side with heavy overlap.
2. Processing. Software aligns the photos and builds a dense mesh and a colour texture.
3. Cleanup. Remove the ground, holes and noise; fix scale and orientation.
4. De-lighting. Remove the lighting baked into the photos so the colour works as a base colour.
5. Game asset. Retopologise or decimate to a game mesh, bake the scan's detail to normal and other maps, or, in Unreal, import a dense mesh for Nanite, which is designed to take "film-quality source arts, such as ZBrush sculpts and photogrammetry scans" (Unreal Engine Documentation: Nanite Virtualized Geometry).

Software choices are covered in photogrammetry and scanning tools. Flat surfaces such as ground and walls are often captured as materials instead; see photo-sourced and scanned materials.

#Capture

Adobe's capture guide treats a DSLR in manual mode as the main tool and says "12Mp or more resolution is a good choice" (Adobe Experience League: 3D Capture Equipment). For small objects on a turntable it suggests "16 shots per rotation" in at least five loops at different heights (Adobe Experience League: 3D Capture Equipment). Use diffuse, even light; matte objects capture more easily (Adobe Experience League: 3D Capture Equipment). Overlap is the most important factor: areas not seen in multiple images will not appear in the model, and the subject should fill at least half of each frame (Epic Developer Community: RealityScan step-by-step guide).

Cross-polarisation is the main technique for clean colour. A polarising filter on the light and another on the lens can "remove all specular highlights from an object" (Pixpro: Cross Polarization Photogrammetry), so reflections do not end up painted into the texture. See cross-polarisation.

#Processing and cleanup

Desktop tools such as RealityScan (formerly RealityCapture) and Agisoft Metashape do the reconstruction. RealityScan 2.0 added AI masking, LiDAR merging and coverage heatmaps that show weak areas (Digital Production: RealityScan 2.0: Epic gives RealityCapture a…), and the desktop app is free below a revenue threshold (realityscan.com: RealityScan licensing). Metashape is sold in Standard and Professional editions (agisoft.com: Metashape installer downloads). Phone apps such as RealityScan Mobile run on Android and iOS (CG Channel: Epic Games releases RealityScan 2.0).

Typical cleanup steps:

• Delete everything that is not the subject (ground, turntable, background).
• Set real-world scale from a known measurement, and orient the asset upright. See units, scale and axes.
• Fill holes where the camera could not see, and smooth noisy areas.
• Create the game mesh, unwrap it, and bake. See the high-to-low workflow.

#De-lighting

A photo records the subject and the light of that moment. Software can remove some of it after the fact: Agisoft De-Lighter takes brush-stroke hints of lit and shadowed areas (CG Channel: Download free texture de-lighting tool…), and Substance 3D Sampler's AI filter removes "lighting information from the base color channel" automatically (Substance 3D Sampler documentation: Delight). The de-lighting page compares the methods.

#Cases where photogrammetry is unsuitable

• Shiny, transparent or featureless subjects. KIRI's turntable guide warns that featureless, single-colour, reflective or transparent objects scan badly (kiriengine.app: How to scan with a turntable).
• Subjects you cannot light well. Adobe's guide asks for diffuse, even light (Adobe Experience League: 3D Capture Equipment); harsh sun gives you shadows to remove later.
• When it would not save time. DICE's verdict, "not a magic bullet but instead a tool like any other" (EA Frostbite: Photogrammetry and Star Wars Battlefront), suggests a simple test: scan where it is faster or better than modeling, and model the rest.

#Common mistakes

• Too few photos. A Capturing Reality knowledge-base post advises "Do not limit the number of images" (Unreal Engine Forums: Knowledge Base: Taking pictures for…); missing angles become holes or smeared geometry.
• Uneven or changing light. Adobe asks for diffuse, even light (Adobe Experience League: 3D Capture Equipment); clouds moving over the sun give the texture patches of different brightness.
• Leaving lighting in the base colour. The asset looks wrong under every in-game light. See de-lighting.
• Shipping the raw scan. Millions of triangles and messy UVs are a source, not a game asset, unless your renderer is built for it (Unreal Engine Documentation: Nanite Virtualized Geometry).

#Further reading and watching

• photogrammetry and scanning tools: software and apps compared, with prices and platforms.