← Paper Digest

Memory-Efficient Parallel Policy Learning for Collapse-Reactive Retreat via Hybrid Terrain Modeling

Yuki Kadokawa, Ritsuki Matsunaga, Yuqi Yang, Edgar Anarossi, Takamitsu Matsubara

November 2026

TL;DR Train a Go2W wheeled quadruped to back out of ground that collapses under it, by replacing collapsible terrain with per-robot virtual contact forces in simulation. This fits over 100x more parallel environments than rigid-block terrain, converges over 50x faster, and the policy transfers zero-shot to four physical collapse scenarios.

What they do

At disaster sites, ground can give way only once a robot is on it. The paper trains one proprioceptive policy for a Unitree Go2W wheeled quadruped that drives normally and, when a wheel loses support, keeps its balance and backs out to firm ground. The obstacle is training: thousands of parallel robots each need their own collapse. If they share one collapsible rigid terrain, one robot’s collapse changes another’s ground (“indirect interference”); giving every robot its own block or particle terrain exhausts GPU memory.

How it works

  • Hybrid terrain modeling. Isaac Lab simulates the robot and firm terrain as rigid bodies. The collapsible region is a real hole in the floor, where virtual contact forces support the wheels.
  • State-Multiplexed Contact Model. All environments share the reference geometry, a penalty contact law with regularized friction, and the state-transition rule. Each keeps only a small collapse state (trigger flags, support ratio, subsidence, slip direction), so no terrain objects are duplicated.
  • Three collapse primitives: Touch (support loss after a load trigger), Sink (subsidence under load, then collapse) and Slip (a lateral push, then support loss).
  • Policy. PPO with an asymmetric actor-critic: the actor sees only proprioception, while the critic also gets distances to the opening. Training uses a terrain curriculum and domain randomization.

Results

  • Only this model ran 16,384 environments on one RTX 4090. Convergence took about 1/4 the time of 50 cm rigid blocks and 1/70 that of 10 cm blocks, at about 90% sim-to-sim success, comparable to the finest grid.
  • Sharing one rigid terrain between just two environments caused interference in about 28% of episodes and cut success from about 70% to 50%.
  • In simulation it avoided falls in over 90% of trials at every speed, compared with under 20% for estimator-triggered retreat (Naive, RTDD). It reached over 80% on unseen particle terrain.
  • On the real robot, with no extra training, policies averaged over 4 of 5 retreats under Touch, Sink and Particle Slip, and fewer under Slip. Naive almost never succeeded under Touch and Sink.

Limitations

Proprioception alone can mistake a downward step for a collapse: Stepped Downhill had the most false retreats, and training steps were at most 0.08 m. How faithfully the virtual contact model matches real ground failure is not quantified. Retreat works mainly after forward entry, and virtual forces act only on the wheels.

Videos

Full video (YouTube)

Figures

From Figure 1: Top, a wheeled quadruped meets collapse-prone ground that cannot be identified before contact and must...
From Figure 1: Top, a wheeled quadruped meets collapse-prone ground that cannot be identified before contact and must retreat to firm ground without falling. Bottom, three ways to model collapsible terrain for parallel training: rigid collapse on shared terrain (one robot's collapse affects others), rigid collapse on per-robot terrain (no interference, but costly), and the proposed virtual collapse on shared terrain.
From Figure 2: Framework overview. Left, an asymmetric actor-critic in which only the proprioceptive actor is transfe...
From Figure 2: Framework overview. Left, an asymmetric actor-critic in which only the proprioceptive actor is transferred to the real robot. Center, parallel environments share one collider-free opening where the State-Multiplexed Contact Model supplies independent virtual contact forces, so robots can be supported, sinking or unsupported at the same spot. Right, a shared contact-response law maps each wheel contact and its environment's own collapse state to forces, and a shared transition updates that state.
From Figure 3: The collapse primitives. (a) The collapse-capable opening with a randomly shaped collapse region, and ...
From Figure 3: The collapse primitives. (a) The collapse-capable opening with a randomly shaped collapse region, and a wheel's support ratio across the region boundary. (b)-(d) Touch (contact-triggered support loss), Sink (load-induced subsidence) and Slip (lateral slip), each in three stages.
From Figure 15: Real-robot retreat successes of Ours, Naive and RTDD in four physical collapse scenarios (Touch, Sink...
From Figure 15: Real-robot retreat successes of Ours, Naive and RTDD in four physical collapse scenarios (Touch, Sink, Slip, Particle Slip). Five independently trained policies per method were each tested five times; bars show the mean successes out of five, error bars the standard deviation across policies.
Abstract

At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. Massively parallel reinforcement learning of collapse-reactive retreat requires independent state--action experiences. However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. We call this indirect interference. Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. A supplementary video is available at: https://youtu.be/_RrhwjzEPE0

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.