← Paper Digest

CaFe: Causally Faithful Conditional Traffic Scenario Generation

Steven Yu Tung, Shubham Vaijanath Phoolari, Christoph Lauer, Abhinav Valada, Steven Peters

November 2026

TL;DR CaFe is a language-conditioned traffic simulator in which each car reads only its own instruction and sees other cars only through a language-blind prediction from their executed motion, so instructions cannot leak to other agents. In a paired replay test where existing designs displace 23-83% of nearby uninstructed agents, CaFe displaces none while keeping WOSAC realism at 0.763.

What they do

Language-conditioned traffic simulators let a test engineer instruct some cars (“cut in late”) while the rest of traffic reacts. The authors show that current designs leak these private instructions: uninstructed cars react before the instructed manoeuvre is visible, which defuses the scenario. They name this intent leakage and contribute a test that measures it, a dataset of private per-vehicle instructions, and CaFe, a simulator that rules out leakage by construction.

How it works

  • Paired replay test. Roll a scene out with instructions, then again with the same random seed, no instructions, and the instructed cars replaying their executed motion. Any uninstructed agent displaced by more than 0.5 m reveals leakage; a faithful simulator scores exactly zero.
  • WOMD-PACT. Built from 531,099 Waymo Open Motion scenes: timed behaviour tags, a counterfactual “ghost” test that attributes each reaction to its cause, and LLM-written, mechanically checked clauses that describe only the vehicle’s own behaviour (1.49 million clauses for 0.81 million vehicles).
  • Two streams. On a frozen diffusion-forcing simulator (MDG), a private stream adds a text adapter for each instructed agent, and a language-blind public stream predicts every agent from executed motion. In inter-agent attention, agents read only the public beliefs of others.
  • Closed-loop fine-tuning with an energy score over eight one-second re-plans.

Results

  • At 8 s in the paired test, broadcast instructions displace 82.7% of nearby uninstructed agents, reading others’ private future tokens 26.7% and ECoSim 22.6%; CaFe 0.0%.
  • WOSAC realism 0.7631 versus 0.7628 for the leaky single-stream variant, minADE 1.276 versus 1.362 m, collision rate 4.81% versus 5.82%.
  • Against a leak-free variant without inter-agent attention, public beliefs cut collisions between the reacting and the instructed car from 9.8% to 5.8%.
  • Instruction following holds: 40.8% of timed clauses satisfied versus 40.9%, joint success 32.4% versus 33.9%.

Limitations

Only language conditioning is tested; goal points and route sketches remain untested. Public beliefs come from one simple design, the frozen backbone. All evaluation is in simulation on Waymo data, and no variant satisfies more than 41% of timed instructions.

Figures

From Figure 2: The paired leakage test. The scene runs twice with the same seed, once with car A following its privat...
From Figure 2: The paired leakage test. The scene runs twice with the same seed, once with car A following its private instruction and once with A replaying that motion without it. Nearby agents displaced by more than 0.5 m count as affected: here a leaky model moves car B by 1.5 m at 4 s, while CaFe affects 0% of agents.
From Figure 3: CaFe. The private stream conditions each agent on its own instruction, while a language-blind pass of ...
From Figure 3: CaFe. The private stream conditions each agent on its own instruction, while a language-blind pass of the frozen backbone predicts public beliefs from executed motion. In inter-agent attention, an agent reads its own private features and only the public beliefs of others. On the right, the public beliefs change as an agent's motion reveals its intention.
Abstract

Language-conditioned traffic simulation enables scalable test scenarios steered by natural language. The goal is to control selected agents while surrounding traffic reacts to their observed behaviour. For example, a vehicle instructed to cut in late should cause the following vehicle to brake only when the manoeuvre becomes observable. Yet simulators can leak private instructions across agents, causing surrounding vehicles to react before the instructed behaviour becomes observable. Such premature reactions defuse desired events and alter the ensuing closed-loop scenario. Existing benchmarks measure what agents do, not what they could have known, leaving this failure unmeasured. We formalise it as intent leakage and address it through evaluation, data and modelling. Our paired intervention test holds an instructed agent's executed trajectory fixed while varying its private instruction, isolating leakage-induced changes in surrounding agents. We also construct WOMD-PACT, an automatically generated, verifiable dataset of 531,099 WOMD scenes with timed behaviour tags with reaction episodes identified through a counterfactual ghost test and grounded private instructions. Finally, we introduce CaFe, a dual-stream diffusion-forcing simulator fine-tuned in closed loop that restricts inter-agent interactions to public beliefs, preserving legitimate anticipation without leaking private intent. Our evaluation reveals that existing conditioning designs displace 23-83% of nearby uninstructed agents through private-intent leakage, even when the instructed agent's executed motion is held fixed. CaFe eliminates measurable leakage while maintaining comparable WOSAC realism (0.763), demonstrating that causal faithfulness can be enforced without sacrificing measured simulation realism.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.