Parallax occlusion mapping

Parallax occlusion mapping (POM) makes a flat surface look deep. For every pixel, the shader follows the view ray down into a height and displacement maps until it hits the stored surface, then shades the pixel with the texture coordinates of that hit point. Bricks appear to stand proud of the mortar and hide it behind them as you walk past, while the polygon underneath stays flat.

Natalya Tatarchuk's 2006 paper, which popularised the technique for games, describes it as "per-pixel ray tracing for inverse displacement mapping with an easy-to-implement, efficient algorithm" (Research) 2006). The height and displacement maps page compares POM with plain parallax and real displacement; this page explains how POM works and what it costs.

#How the ray march works

1. The shader takes the view direction in tangent space and MikkTSpace and splits the depth range into layers.
2. It steps along the ray, one layer at a time, reading the height map at each step. Tatarchuk samples "using a linear search and approximating the height profile as a piecewise linear curve" (Research) 2006).
3. When a sample falls below the stored height, the ray has entered the surface. The shader refines the hit between the last two samples.
4. The hit point's UVs are used to read the base colour, normal and other maps. The visible result is that tall features hide what is behind them.

Steep viewing angles need more steps, because the ray travels further across the texture. Tatarchuk varies the sampling rate "as a linear function of the angle between the geometric normal and the view direction ray" (Research) 2006). Godot does the same: its shader mixes between a maximum and a minimum layer count by view angle, with defaults of 32 and 8 layers (Godot Engine: scene/resources/material.cpp).

#What it costs

Each step is a texture read, in the pixel shader, for every pixel the surface covers.

Common ways to keep it affordable:

• Fade it out with distance. Tatarchuk's shader runs "only a simple bump mapping shader" for mip levels beyond a threshold (Research) 2006). HDRP's equivalent is Fading Mip Level Start, "The mip level at which the pixel displacement effect begins to fade out" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference).
• Cap the steps. HDRP exposes Minimum Steps and Maximum Steps for its pixel displacement (High Definition Render Pipeline manual 17.0: Lit material Inspector reference); Unreal's POM material function has Min Steps and Max Steps inputs (Unreal Community Wiki: Parallax Occlusion Mapping).
• Use it where the eye goes. Floors, cobbles and brick walls near the player; not ceilings or distant facades.

#Self-shadowing and depth

Two additions make the relief more believable, both at extra cost:

• Self-shadowing. A second march towards the light finds whether a point is hidden by taller features. Tatarchuk "heuristically approximate[s] the size of penumbra" to get soft shadows (Research) 2006).
• Depth offset. Writing the marched depth lets other objects and decals intersect the fake relief correctly. HDRP's Depth Offset "modifies the depth buffer according to the displacement" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference), and Unreal's POM function outputs Pixel Depth Offset (Unreal Community Wiki: Parallax Occlusion Mapping).

#Limits

• The silhouette stays flat. POM never changes geometry, so edges and the line where a POM floor meets a wall stay straight. Tatarchuk notes that silhouettes remain low-resolution, and that artists sometimes add border geometry to hide this (Research) 2006).
• Stair-stepping and swimming. "Parallax occlusion mapping is a sampling-based algorithm at its core, and as such, it can exhibit aliasing" (Research) 2006). Too few layers show as slices at grazing angles; too much height scale makes the surface swim. Godot's height scale defaults to 5 cm between black and white and warns that higher values may produce artefacts (Godot Engine: BaseMaterial3D class reference).
• Shadows from other objects ignore the relief. Cast shadows use the flat polygon, so the community tutorial suggests you may need to turn off shadow casting on the mesh (Unreal Community Wiki: Parallax Occlusion Mapping).
• UV layout matters. Godot's heightmap needs normalised heights and correct tangents; if it "looks strange when the camera moves", try flipping the tangent or binormal (Godot Engine: BaseMaterial3D class reference).

#In different tools

• Unreal Engine: the ParallaxOcclusionMapping material function, with Heightmap Texture, Height Ratio, Min Steps and Max Steps inputs and Parallax UVs and Pixel Depth Offset outputs (Unreal Community Wiki: Parallax Occlusion Mapping).
• Unity HDRP: Lit material, Displacement Mode Pixel Displacement, which "Displaces the pixels on the mesh surface according to the Height Map" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference).
• Godot: enable Height with Deep Parallax, then set Min Layers and Max Layers (Godot Engine: BaseMaterial3D class reference).

#Further reading and watching

Related: height and displacement maps, normal map, height blending, shaders for artists, virtualised geometry.