Asynchronous Robot Policies

Sombit Dey, Luc Van Gool, Danda Pani Paudel

November 2026

Two sped-up real Franka runs of putting toy vegetables into a bowl.

TL;DR ARP splits a vision-language-action policy into two processes. A slow VLM keeps refreshing a compact memory, while a small vision-action expert reads it and acts at a much higher rate, trained to cope with stale memory. On LIBERO it keeps 81.8% success with the VLM updating at only 0.5 Hz, and when the target shifts 5 cm mid-reach it succeeds 51.5% of the time against 9.5% for a π0-style baseline.

What they do

A standard VLA runs its vision-language model and action expert one after the other inside the control loop. Latency therefore adds up and grows with VLM size. ARP decouples the two into asynchronous processes, borrowing ROS’s publisher-subscriber pattern: a slow semantic loop and a fast action loop whose latency is only the action expert’s.

How it works

Semantic expert. The VLM’s key-value activations from several layers pass through a learned adapter into a fixed-size memory, tagged with an embedding of its age. Because the memory size is fixed, the action side does not depend on the VLM’s scale.
Vision-action expert. A transformer over current camera, proprioception and action tokens cross-attends to the memory in every layer and generates action chunks with flow matching. Its normalization is conditioned on the memory’s staleness and the execution delay.
Staleness training. Each training frame is paired with VLM input from an earlier frame of the same demonstration (lag up to 40 steps by default). Action-prefix conditioning smooths the switch from one chunk to the next.

Results

LIBERO, one multitask policy: ARP-1B reaches 84.8% against 80.2% for a replicated π0-1B, and ARP-8B reaches 89.3%.
Disturbances in simulation: with a 5 cm target shift mid-reach, ARP-1B succeeds 51.5% of the time against 9.5% for the baseline. When bumps redirect a rolling target, it succeeds 44.7% against 12.7%.
Throttling the VLM: the π0-style baseline falls from 81.5% to 45.0%, while ARP-1B stays between 79.0% and 84.8%. Without staleness training, ARP drops to 2.0% at 0.5 Hz.
DROID-trained Franka, 3 real tasks × 10 trials: higher progress and throughput than π0.5-DROID on the two step-budget-limited bowl tasks, and comparable on cup stacking. In RoboLab simulation, ARP reaches 20.0% average success against 18.4% for LAP-3B, with episodes 20% shorter than π0.5’s.

Limitations

π0.5 still leads overall in RoboLab (37.9%), which the authors attribute to its larger proprietary training data. The real-robot evaluation is small and reported only in a figure, and the dynamic tests are simulation-only. How slowly the VLM can run before performance degrades is left open.

Videos

Dynamic Bump task in simulation, grasping a rolling ball, from the scene and wrist cameras.

Figures

From Figure 1: A standard VLA runs the VLM and the action expert in sequence, so its perception-to-action latency is ... — From Figure 1: A standard VLA runs the VLM and the action expert in sequence, so its perception-to-action latency is their sum. ARP updates the VLM asynchronously and lets a vision-action expert act on every new observation, so control latency is the expert's alone, independent of VLM size.

From Figure 2: ARP architecture. The semantic expert fills the memory from the latest available observation, while th... — From Figure 2: ARP architecture. The semantic expert fills the memory from the latest available observation, while the vision-action expert cross-attends to this possibly stale memory. Its adaptive normalization is conditioned on the flow-matching step and on staleness.

From Figure 3: Franka results for ARP and π0.5, both trained on DROID, on three real-world tasks and in RoboLab simul... — From Figure 3: Franka results for ARP and π0.5, both trained on DROID, on three real-world tasks and in RoboLab simulation.

From Figure 7: Success rate of ARP-1B and the π0-1B baseline as the VLM inference rate drops from 5 Hz to 0.5 Hz. ARP... — From Figure 7: Success rate of ARP-1B and the π0-1B baseline as the VLM inference rate drops from 5 Hz to 0.5 Hz. ARP-1B holds steady and leads by up to 18.2 points at 0.5 Hz.

Abstract

Scaling vision-language foundation models has significantly advanced generalist robot manipulation, but the increased model scale introduces computational latency that degrades real-time physical reactivity. Existing Vision-Language-Action (VLA) architectures execute high-level semantic reasoning and action execution sequentially within the primary control loop, creating a fundamental tension between semantic-reasoning capability and robot agility. To address this, we propose Asynchronous Robot Policy (ARP), an architectural framework that decouples semantic reasoning from low-level action generation using asynchronous processes operating at independent frequencies. In ARP, a high-capacity Vision-Language Model (VLM) asynchronously updates a semantic memory at low frequency, while a lightweight Vision-Action Expert, conditioned on this semantic memory, generates robot actions at a much higher control frequency. Such a formulation, enables having a fast perception-action loop without compromising on the semantic capability. To overcome asynchronous memory staleness, ARP trains the vision-action expert on the semantic staleness and execution delay, with temporally mismatched observation across processes. On Franka manipulation tasks, ARP achieves an 18% higher success rate and 20% less time need for execution in quasi-static setups, while 3-5 times baseline performance in dynamic simulation environments. Furthermore, ARP enables operating the VLM at 10 times lower than its nominal frequency with no significant performance loss making it much more efficient than the baseline. These results highlight the architectural advantages of modeling robot policies as asynchronous processes, enabling real-world reactive deployment without sacrificing the scale of foundation models.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong