Opacity, masked and translucent materials

Engines have three ways to draw a surface you can partly see through. Masked materials (alpha test, cutout) keep or discard each pixel. Translucent materials (alpha blend) mix each pixel with what is behind it. Dithered or hashed materials fake partial opacity with a pattern of kept and discarded pixels. glTF names the first two: in MASK mode the output "is either fully opaque or fully transparent", used for "tree leaves or wire fences", while BLEND is "combined with the background", used for "gauze cloth or animal fur" (Khronos Group: glTF 2.0 Specification).

Picking the right one is one of the most common performance and quality decisions in foliage, hair, fences, glass and effects.

#Masked (alpha test)

Each pixel compares its alpha against a threshold. glTF: "If the alpha value is greater than or equal to the alphaCutoff value then it is rendered as fully opaque, otherwise, it is rendered as fully transparent" (Khronos Group: glTF 2.0 Specification). Unreal calls this the Opacity Mask Clip Value: pixels above it are drawn and pixels below are discarded (Unreal Engine documentation: Unreal Engine Material Properties). Unreal adds that masked pixels "simply do not draw; you will not see any reflections in those areas" (Unreal Engine documentation: Material Blend Modes).

Because surviving pixels are opaque, masked materials write depth and need no sorting (Khronos Group: glTF 2.0 Specification), and they cast shadows. Godot's Alpha Scissor mode "supports casting shadows" and is "faster to render than alpha blending, but slower than opaque rendering" (Godot Engine: BaseMaterial3D class reference).

The downside is a hard, aliased edge. MSAA does not help, because shaders still run once per pixel, so it "does not reduce transparency aliasing for materials using the Alpha Scissor transparency mode" (Godot documentation: 3D antialiasing). Alpha to coverage fixes this by feeding alpha into the MSAA sample mask; Godot says it is "effective at reducing aliasing on transparent materials without introducing any blurriness", at a moderate cost (Godot documentation: 3D antialiasing).

#Why foliage disappears at a distance

Mipmaps average the alpha channel, so fewer pixels pass the cutoff in each smaller level. Ignacio Castaño saw this when building trees: "As the trees moved farther away from the camera, the leafs faded out becoming almost transparent" (Ludicon: Computing Alpha Mipmaps).

#Translucent (alpha blend)

Blending gives real partial opacity: smoke, glass, water surfaces, soft hair tips. It is also the most expensive and least reliable mode.

• Cost and shadows. Godot's Alpha mode "is the slowest to render, and disables shadow casting" (Godot Engine: BaseMaterial3D class reference).
• Sorting. For blended surfaces, glTF notes "There is no perfect and fast solution that works for all cases" (Khronos Group: glTF 2.0 Specification). Overlapping translucent objects can draw in the wrong order. HDRP offers a Transparent Depth Prepass to "improve their sorting" and Back Then Front Rendering, which draws the material "in two separate draw calls" (High Definition Render Pipeline manual 17.0: Lit material Inspector reference).
• Lighting limits. Unreal's blend modes page says translucent materials "do not currently support specularity" in that mode and lists separate lighting modes, from the cheap Volumetric NonDirectional to the most expensive Surface Forward Shading (Unreal Engine documentation: Material Blend Modes; Unreal Engine documentation: Unreal Engine Material Properties).
• Overdraw. Every blended layer shades the pixel again; a stack of smoke cards or leaf cards multiplies the work. See overdraw.

Other blend modes are cheaper special cases: Additive adds to the background (fire, glows) and Modulate multiplies it (stains, tinted glass) (Unreal Engine documentation: Material Blend Modes).

#Dithered and hashed

A dithered material is a masked material whose threshold changes from pixel to pixel in a pattern. Unreal's Dither Opacity Mask, combined with temporal antialiasing, gives a limited form of translucency that supports all lighting features (Unreal Engine documentation: Unreal Engine Material Properties). Godot's Alpha Hash uses "a spatially-deterministic threshold", supports shadows, and "is suited for hair rendering" (Godot Engine: BaseMaterial3D class reference).

Chris Wyman's GDC 2017 talk on hashed alpha testing explains why: ordinary "alpha-tested geom can disappear w/ distance", so hashing uses "stable, procedural noise" for the threshold (GDC 2017: Hashed Alpha Testing). The price is visible noise; the talk lists temporal antialiasing, which "integrates sub pixel noise samples", as one way to deal with it (GDC 2017: Hashed Alpha Testing). The same technique is used to fade LODs in and out; Unreal has a per-material Dithered LOD Transition option (Unreal Engine documentation: Unreal Engine Material Properties).

#In different tools

#Further reading and watching

Related: foliage and vegetation, overdraw, impostors and billboards, decals, mipmaps.