Height and displacement maps

A height map is a greyscale texture that stores how far each point of a surface sits above or below its base level. Unity: "A heightmap should be a greyscale image, with white areas representing the high areas of your texture" (Unity Manual: Heightmap). When the map actually moves geometry it is usually called a displacement map; the data is the same.

#Three ways to use it

The same height map can drive three very different techniques, with very different costs.

#Parallax and parallax occlusion mapping (POM)

Parallax mapping shifts the texture coordinates per pixel based on the viewing angle, so high areas appear to stand in front of low ones. Unity: it "actually shifts the areas of the visible surface texture around", giving "a kind of surface-level occlusion effect" (Unity Manual: Heightmap). Parallax occlusion mapping marches a ray into the height field for a much more convincing result; Godot's "deep parallax" option uses it and warns it is "much more expensive on the GPU" (Godot Engine: BaseMaterial3D).

Neither technique adds geometry. Godot: it "only creates an illusion of depth, and does not add real geometry" (Godot documentation: Standard Material 3D and ORM Material 3D). Unity: "the 'silhouette' of the model will never be modified" (Unity Manual: Heightmap). They work best on flat, tiling surfaces like floors and walls, viewed from not too low an angle; Godot warns of artefacts "at oblique angles, especially when the camera moves" (Godot Engine: BaseMaterial3D).

#Displacement and tessellation

Displacement moves real vertices along their normals. HDRP's Vertex displacement "Displaces the mesh's vertices according to the Height Map", while its Pixel displacement mode is a parallax technique (High Definition Render Pipeline manual 17.0: Lit material Inspector reference). Displacement only works if the mesh has enough vertices, which is why it is usually paired with tessellation, subdividing the mesh on the GPU.

In Unreal Engine 5, Nanite tessellation "is dynamic programmable displacement" that modifies Nanite meshes at runtime with a displacement map or a procedural material (Unreal Engine documentation: Nanite Virtualized Geometry). Epic lists its uses as authoring with less source detail, "Material-driven and animated displacement", and "Creating detailed Nanite landscapes" (Unreal Engine documentation: Nanite Virtualized Geometry). A separate feature, Nanite static displacement, bakes the displacement into an optimised Nanite mesh offline (Unreal Engine documentation: Nanite Virtualized Geometry). See virtualised geometry.

#Height blending

Height maps are also used purely as data: to blend two materials so that the higher one wins, for example sand settling only in the gaps between cobbles. See height blending and vertex painting and blending.

#Authoring and storage

• Mid-grey as zero. When baking, Marmoset suggests setting min and max values so that the zero point is 50% (Marmoset: The Toolbag Baking Tutorial). Then the surface can push both in and out.
• Use the full range. Godot: "For best results, the texture should be normalized"; a map that uses only a small part of its range makes parallax "look strange, especially when the camera moves" (Godot Engine: BaseMaterial3D).
• Bit depth. Unreal has a dedicated Displacementmap compression setting, listed as "8 / 16bit" (Unreal Engine documentation: Texture Compression Settings). 8-bit gives only 256 height steps, which shows as terracing on real displacement; 16-bit is safer for anything that moves geometry.
• Resolution. Godot notes you may be able to use a lower-resolution height map because "most heightmaps are only comprised of low-frequency data" (Godot Engine: BaseMaterial3D).
• Pair it with a normal map. Godot and Unity both recommend using height together with a normal map (Godot documentation: Standard Material 3D and ORM Material 3D; Unity Manual: Heightmap).

#Example

#Common mistakes

• Expecting POM to change the silhouette. The silhouette stays unchanged (Unity Manual: Heightmap).
• Too much depth. High parallax scales create swimming artefacts at grazing angles (Godot Engine: BaseMaterial3D).
• 8-bit displacement on large, close surfaces, showing stepped terraces.
• Displacing a low-poly mesh without tessellation; there are not enough vertices to move.
• Height map and normal map that disagree, because they were baked from different high polys.
• Using height maps with triplanar mapping in Godot, where height mapping is not supported on the same material (Godot Engine: BaseMaterial3D).

#In different tools

• Unreal Engine: Nanite tessellation driven by a displacement map or procedural material (Unreal Engine documentation: Nanite Virtualized Geometry); parallax is built in the material graph.
• Unity: Height Map in URP Lit (Unity Manual: Lit shader material Inspector window…); HDRP Displacement Mode with vertex and pixel options (High Definition Render Pipeline manual 17.0: Lit material Inspector reference).
• Godot: Height, with Deep Parallax for POM (Godot Engine: BaseMaterial3D).
• Bakers: Marmoset's Height output "will bake a height or displacement map" (Marmoset: The Toolbag Baking Tutorial).

#Related terms

normal map · height blending · virtualised geometry · baking · tileable materials · terrain and landscapes

#Further reading and watching