Ray tracing

Ray tracing renders an image by following rays of light through the scene: from the camera into the world, and from the surfaces they hit towards lights and other surfaces. NVIDIA's explainer describes it as following "the path of those beams backwards from your eye to the objects that light interacts with" (NVIDIA Blog: What's the Difference Between Ray Tracing and…). It answers questions that ordinary rendering struggles with: is this point in shadow, what does this surface reflect, and where does its bounce light come from?

#Ray tracing and rasterisation

Most real-time rendering is rasterisation: projecting the scene's triangles onto the screen and shading the pixels they cover (NVIDIA Blog: What's the Difference Between Ray Tracing and…). It is fast but only knows what the camera sees. Ray tracing can look anywhere in the scene, at a much higher cost.

NVIDIA traces the idea to Arthur Appel at IBM in 1969, and credits Turner Whitted's 1979 paper with showing "how to capture reflection, shadows and refraction" (NVIDIA Blog: What's the Difference Between Ray Tracing and…). Newer hardware made real-time use possible. Because it is expensive, ray tracing in games is usually applied only where it helps most, with the rest rasterised (NVIDIA Blog: What's the Difference Between Ray Tracing and…). Unity's HDRP works this way: it "uses ray tracing to replace certain rasterized effects", such as screen-space shadows, reflections and global illumination (High Definition Render Pipeline manual: Getting started with ray tracing).

#Hardware and software ray tracing

Hardware ray tracing uses dedicated GPU units to test rays against the actual triangles. Software ray tracing traces against a simplified version of the scene on ordinary shader cores. Unreal's Lumen offers both:

• Software is the default: "Lumen uses Software Ray Tracing against Signed Distance Fields by default" (Unreal Engine Documentation: Lumen Technical Details). See mesh distance fields.
• Hardware "supports a larger range of geometry types", including skinned meshes, and can "evaluate lighting at the ray hit instead of the lower quality Surface Cache" (Unreal Engine Documentation: Lumen Technical Details). It requires DirectX 12 GPUs from the NVIDIA RTX 2000 or AMD RX 6000 series upwards, or PlayStation 5, Xbox Series S/X or Switch 2 (Unreal Engine Documentation: Lumen Technical Details). Its costs grow with many overlapping meshes, and deforming meshes cost "a large cost to update the Ray Tracing acceleration structures each frame" (Unreal Engine Documentation: Lumen Technical Details).

Unity's HDRP ray tracing is narrower in platform terms: it "only supports ray tracing using the DirectX 12 API", in the Editor or Windows player (High Definition Render Pipeline manual: Getting started with ray tracing).

#What it means for artists

• Content rules depend on the method. Software tracing in Lumen does not support World Position Offset, cannot represent "extremely thin features", and needs walls of at least 10 cm (Unreal Engine Documentation: Lumen Technical Details). See light leaking.
• Big single meshes trace badly in software mode. Epic warns that "a mountain or multi-story building, will have a poor distance field representation" (Unreal Engine Documentation: Lumen Technical Details). Split them.
• Fewer lighting workarounds. Fewer fill lights and baked tricks are needed, but exposure matters more because interiors come out darker, as they do in reality.

Related: global illumination, real-time lighting, real-time vs. offline rendering, shadows and virtual shadow maps.

#Further reading and watching