Parallax occlusion mapping

Parallax occlusion mapping (POM) makes a flat surface look deep. For every pixel, the shader follows the view ray down into a height and displacement maps until it hits the stored surface, then shades the pixel with the texture coordinates of that hit point. Bricks appear to stand proud of the mortar and hide it behind them as you walk past, while the polygon underneath stays flat.

Natalya Tatarchuk's 2006 paper, which popularised the technique for games, describes it as "per-pixel ray tracing for inverse displacement mapping with an easy-to-implement, efficient algorithm" (Research) 2006). The height and displacement maps page compares POM with plain parallax and real displacement; this page explains how POM works and what it costs.

#How the ray march works

1. The shader takes the view direction in tangent space and MikkTSpace and splits the depth range into layers.
2. It steps along the ray, one layer at a time, reading the height map at each step. Tatarchuk samples "using a linear search and approximating the height profile as a piecewise linear curve" (Research) 2006).
3. When a sample falls below the stored height, the ray has entered the surface. The shader refines the hit between the last two samples.
4. The hit point's UVs are used to read the base colour, normal and other maps. The visible result is that tall features hide what is behind them.

Steep viewing angles need more steps, because the ray travels further across the texture. Tatarchuk varies the sampling rate "as a linear function of the angle between the geometric normal and the view direction ray" (Research) 2006). Godot does the same: its shader mixes between a maximum and a minimum layer count by view angle, with defaults of 32 and 8 layers (Godot Engine: scene/resources/material.cpp).

#What it costs

Each step is a texture read, in the pixel shader, for every pixel the surface covers.

Common ways to keep it affordable:

• Fade it out with distance. Tatarchuk's shader runs "only a simple bump mapping shader" for mip levels beyond a threshold (Research) 2006). HDRP's equivalent is Fading Mip Level Start, "The mip level at which the pixel displacement effect begins to fade out" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference).
• Cap the steps. HDRP exposes Minimum Steps and Maximum Steps for its pixel displacement (High Definition Render Pipeline manual 17.0: Lit material Inspector reference); Unreal's POM material function has Min Steps and Max Steps inputs (Unreal Community Wiki: Parallax Occlusion Mapping).
• Use it where the eye goes. Floors, cobbles and brick walls near the player; not ceilings or distant facades.

#Self-shadowing and depth

Two additions make the relief more believable, both at extra cost:

• Self-shadowing. A second march towards the light finds whether a point is hidden by taller features. Tatarchuk "heuristically approximate[s] the size of penumbra" to get soft shadows (Research) 2006).
• Depth offset. Writing the marched depth lets other objects and decals intersect the fake relief correctly. HDRP's Depth Offset "modifies the depth buffer according to the displacement" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference), and Unreal's POM function outputs Pixel Depth Offset (Unreal Community Wiki: Parallax Occlusion Mapping).

#Limits

• The silhouette stays flat. POM never changes geometry, so edges and the line where a POM floor meets a wall stay straight. Tatarchuk notes that silhouettes remain low-resolution, and that artists sometimes add border geometry to hide this (Research) 2006).
• Stair-stepping and swimming. "Parallax occlusion mapping is a sampling-based algorithm at its core, and as such, it can exhibit aliasing" (Research) 2006). Too few layers show as slices at grazing angles; too much height scale makes the surface swim. Godot's height scale defaults to 5 cm between black and white and warns that higher values may produce artefacts (Godot Engine: BaseMaterial3D class reference).
• Shadows from other objects ignore the relief. Cast shadows use the flat polygon, so the community tutorial suggests you may need to turn off shadow casting on the mesh (Unreal Community Wiki: Parallax Occlusion Mapping).
• UV layout matters. Godot's heightmap needs normalised heights and correct tangents; if it "looks strange when the camera moves", try flipping the tangent or binormal (Godot Engine: BaseMaterial3D class reference).

#How to set one up

These steps set up POM on a cobblestone floor material in Unreal Engine, with the Unity HDRP and Godot equivalents at the end. The worked numbers assume a 2 × 2 m tiling material with a height map.

1. Prepare the height map. Use a greyscale height map that uses the full black-to-white range; a map that uses only part of its range makes parallax "look strange, especially when the camera moves" (Godot Engine: BaseMaterial3D class reference). Import it with sRGB off.
2. Add the POM function. In the material, add the ParallaxOcclusionMapping material function and connect the height map to Heightmap Texture (Unreal Community Wiki: Parallax Occlusion Mapping).
3. Set the depth. Set Height Ratio to a small value, such as 0.03–0.05 for cobbles, and raise it slowly; too much height makes the surface swim at grazing angles (Godot Engine: BaseMaterial3D class reference).
4. Set the steps. Start with Min Steps 8 and Max Steps 32, the same range as Godot's defaults (Godot Engine: scene/resources/material.cpp; Unreal Community Wiki: Parallax Occlusion Mapping). Lower Max Steps until slicing just starts to show at a low viewing angle, then go back one step.
5. Use the new UVs. Connect the function's Parallax UVs output to the UV input of every other texture in the material (base colour, normal, roughness), so they all shift together (Unreal Community Wiki: Parallax Occlusion Mapping).
6. Add depth offset if objects sit on it. Connect Pixel Depth Offset so props and decals intersect the relief rather than the flat polygon (Unreal Community Wiki: Parallax Occlusion Mapping).
7. Fade it out with distance. Blend to the plain normal-mapped version beyond about 10 m, or at a chosen mip level as Tatarchuk's shader does (Research) 2006).
8. Measure. Compare frame time with POM on and off with the floor filling the screen.

In Unity HDRP, set the Lit material's Displacement Mode to Pixel Displacement, then set Minimum Steps, Maximum Steps, Fading Mip Level Start and Depth Offset (High Definition Render Pipeline manual 17.0: Lit material Inspector reference). In Godot, enable Height and Deep Parallax and set Min Layers and Max Layers (Godot Engine: BaseMaterial3D class reference).

#Video tutorials

#Use in recent games

• The Last of Us Part I (2022), Naughty Dog. Matthew Trevelyan Johns built the ice in the deer-hunt section to look thick, with the riverbed visible below. His caption: "The illusion of depth was created using multiple parallax occlusion effects to simulate varying crack thicknesses"; because the cracks were broken up, few layers were needed, which reduced the render cost (ArtStation: The Last of Us: Part I...snow and ice shaders).
• Star Citizen (Cloud Imperium Games, in open development). Thomas Carter improved the game's shared sheets of POM decals, modelling over "tiny 32 pixel textures" to match their heights and rebuilding the maps so they lined up with the old layout (ArtStation: Star Citizen - Utilitarian Decal Sheet).
• CryEngine 2 (2007), Crytek. An older, historically important case: the engine listed parallax occlusion mapping among its features (Wikipedia: Parallax occlusion mapping).

A related shader trick also fakes depth on a flat surface, but traces view rays against the walls of a virtual room instead of a height field: interior mapping shows rooms behind flat windows.

#In different tools

• Unreal Engine: the ParallaxOcclusionMapping material function, with Heightmap Texture, Height Ratio, Min Steps and Max Steps inputs and Parallax UVs and Pixel Depth Offset outputs (Unreal Community Wiki: Parallax Occlusion Mapping).
• Unity HDRP: Lit material, Displacement Mode Pixel Displacement, which "Displaces the pixels on the mesh surface according to the Height Map" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference).
• Godot: enable Height with Deep Parallax, then set Min Layers and Max Layers (Godot Engine: BaseMaterial3D class reference).

#Further reading and watching

Related: height and displacement maps, normal map, height blending, shaders for artists, virtualised geometry.