Telemetry and heatmaps

Telemetry is data a game records automatically about how it is played: positions, deaths, kills, time spent, items used. A heatmap plots one of those values over a top-down view of the level, so the hot spots show where something happens most. Bungie's 2007 Halo 3 maps, built from kill and death statistics across "thousands of Halo 3 online faceoffs", made the technique famous (GamesRadar: Where not to stand in Halo 3).

For level designers it shows where players die, which routes they take and which sightlines dominate, which observation alone cannot show reliably. It complements playtesting spaces rather than replacing it.

#Where it came from

Microsoft's games user research group grew out of psychology and human–computer interaction. Randy Pagulayan and colleagues recall that, at the time, game "playtesting" typically meant inviting friends to try an unfinished game, often in exchange for food (Pagulayan et al. 2018). By Halo 3, the lab combined observation, think-aloud testing and "complex computer programs" that helped staff see when players were and weren't having fun (NPR: Microsoft to Debut Final Halo Game). Designer Paul Bertone summed up the effect of aggregated data: "You can't hide from a hundred people playing the game, and everybody got stuck there" (NPR: Microsoft to Debut Final Halo Game).

Changes that came out of that testing included flashing objectives on screen, changing a gun's firing angle because players kept shooting themselves, and adding a barrier on a highway where many players flew off to their deaths (NPR: Microsoft to Debut Final Halo Game). NPR reported "hundreds of changes" in total (NPR: Microsoft to Debut Final Halo Game).

Academic work followed. Anders Drachen and Alessandro Canossa's 2009 paper applied geographic information systems (GIS), the tools used for real-world maps, to spatial player behaviour (Drachen & Canossa 2009).

#Other telemetry measures

• Time spent. At GDC 2008, Dan Arey and Chris Swain presented BioWare's time-spent reports for Mass Effect: players found the Noveria level too conversation-heavy and slow, and it was rebalanced with more combat (raphkoster.com: GDC2008: Master Metrics).
• Designer intent versus reality. The same talk described comparing heatmaps with design intent, for example players moving slowly through a gauntlet meant to feel fast (raphkoster.com: GDC2008: Master Metrics).
• Kills by weapon showed how players used items on multiplayer maps (raphkoster.com: GDC2008: Master Metrics).
• Balance and flow maps. Sean Houghton of High Moon Studios subtracts deaths from kills per cell to make a balance map, and traces the average direction of kills to make a flow map, whose lines "trace good 'run and gun' paths" (Game Developer: Opinion: Balance and Flow Maps). His example used 30 million rows of Team Deathmatch data from Transformers: War for Cybertron's Molten map (Game Developer: Opinion: Balance and Flow Maps).
• Development telemetry. At GDC 2010, Georg Zoeller described a "developer-facing telemetry solution" at BioWare for managing large projects such as Dragon Age: Origins and Mass Effect (GDC Vault: Development Telemetry in Video Games Projects).

#Reading a heatmap

• Chokepoints and dominant positions show as dense clusters (see the Valhalla example).
• Unused space shows as cold areas: art budget spent where nobody goes.
• Unexpected routes show where players' desire lines differ from your critical path.
• Height is lost in a top-down view; check multi-level areas separately (Game Developer: Opinion: Balance and Flow Maps).

#Related terms

playtesting spaces, layout and flow, critical path, sightlines and vistas, performance and profiling.

#Further reading and watching