Empirical Grounding Improves the Realism of LLM Agents Simulating Human Behavior During Disruptions

2026-07-19

AI Summary

This work addresses the tendency of existing large language model (LLM) agents to generate human behaviors during perturbation events that lack statistical realism and empirical grounding. To overcome this limitation, the authors propose an empirically anchored LLM agent framework that deeply integrates U.S. Census demographic data, time-use survey behavioral baselines, and urban spatial context throughout the agent lifecycle—including initialization, memory formation, decision-making prompts, and behavior execution. This integration substantially enhances the statistical fidelity of simulated behaviors: under normal conditions, the activity profile correlation coefficient improves from 0.528 to 0.912; during heatwaves, it rises from 0.349 to 0.836, and the capture rate of heatwave response magnitude increases from 20.6% to 46.4%.

Abstract

Large language model (LLM) agents offer a generative approach to simulating human behavior under conditions that may have few or no direct historical analogues, a common challenge in disaster and infrastructure-disruption planning. However, this generative capacity creates a validity problem: individually plausible agent reasoning may fail to reproduce empirical population behavior. We evaluate whether empirical grounding improves the statistical realism of LLM-agent simulations during disruptions. Specifically, we develop an empirically grounded LLM-agent framework that embeds demographic profiles from the American Community Survey, baseline routines from the American Time Use Survey, and urban spatial context into agent initialization, memory, decision prompts, and activity execution. An independent household survey conducted during the July 2024 Philadelphia heatwave is reserved as an external validation benchmark. Compared with an ungrounded LLM-agent baseline, the grounded model improved reconstruction of normal daily routines, increasing mean correlation with empirical activity profiles from 0.528 to 0.912 and reducing mean squared error from 0.066 to 0.008. Under heatwave conditions, the grounded model better reproduced survey-derived activity profiles, increasing mean correlation from 0.349 to 0.836 and reducing mean squared error from 0.098 to 0.012. The grounded model captured 46.4% of observed heatwave response amplitude, compared with 20.6% for the ungrounded baseline. These findings show that empirical grounding can make LLM agents more statistically credible simulators of population behavior while revealing remaining gaps in modeling human adaptation during disruptions.

Problem

• LLM agents
• human behavior simulation
• empirical grounding
• disruption response
• statistical realism

Innovation

• empirical grounding
• LLM agents
• human behavior simulation
• disruption response
• statistical realism