Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

BiPOP: Partial-Order Planning for Long-Horizon Bimanual Robotic Manipulation

Kun Chu, Xufeng Zhao, Jan-Gerrit Habekost, Cornelius Weber, Stefan Wermter

November 2026

TL;DR An LLM writes a bimanual task as a PDDL problem and a temporal planner decides which arm does what and what runs in parallel. Built on non-reasoning GPT-4.1, BiPOP succeeds on 100%, 100% and 95% of three simulated task domains, against 98%, 87% and 86% for open-loop planning with o3.

What they do

Many long-horizon bimanual tasks are a sequence of one-arm skills plus a few two-handed ones. The hard part is deciding which arm does what, in which order, and what can safely run at the same time. BiPOP has an LLM write the task as a PDDL problem and leaves arm assignment, ordering and concurrency to a temporal planner, which returns a partial-order plan.

How it works

  • Spatio-temporal model. The table is split by reachability into a left-only, a right-only and an overlap region. Two actions run concurrently only if they share no arm, object or region; two-handed skills, such as carrying a large bowl, are possible only in the overlap region. Skills are durative PDDL actions with mutual-exclusion predicates.
  • LLM as PDDL writer. An open-vocabulary detector (LocateAnything) assigns objects to regions. GPT-4.1 writes Python that builds the PDDL problem through the Unified-Planning library; interpreter errors and “no plan found” are fed back for regeneration.
  • Temporal planning. Temporal Fast Downward runs under several configurations, the VAL validator rejects invalid schedules, and the validated plan with the fewest parallel steps is sent to the arm controllers.

Results

  • 100 randomized instances per domain in CoppeliaSim (ServeFruit, ServeWater, StackBlock) on the NICOL robot (two 8-DoF arms with five-fingered hands). Success rates: BiPOP 100/100/95%, open-loop o3 98/87/86%, DeepSeek-R1 66/82/52%, open-loop GPT-4.1 25/0/0%, ReAct with GPT-4.1 100/45/36%.
  • Planning time: BiPOP 6.6, 6.0 and 35.6 s, against 33-73 s for o3 and 227-368 s for R1. On StackBlock the temporal planner takes 28.9 s of that.
  • BiPOP almost always produces the shortest plan on ServeFruit and ServeWater. LLM-only planners mostly fail by violating spatial or temporal coordination constraints, and ReAct’s successful plans are longer.
  • On the real robot the detector localized all task objects in 100 real images each of ServeFruit and ServeWater, and BiPOP ran ServeWater end to end across varied configurations.

Limitations

The real-robot run covers only ServeWater, gives no success rate, and uses hand-crafted skills. All actions are given the same duration, there is no execution monitoring or replanning after a failed grasp, and the PDDL domain is written by hand. On StackBlock the planner does not always converge within its time budget.

Figures

From Figure 1: Overview on the ServeWater task. The scene description, the bimanual domain knowledge and the instruct...
From Figure 1: Overview on the ServeWater task. The scene description, the bimanual domain knowledge and the instruction go to an LLM that writes a PDDL problem; a temporal planner solves it, and the validated partial-order plan is executed by the NICOL robot.
From Figure 2: Partial-order plans BiPOP generated for ServeWater on the real robot under three initial configuration...
From Figure 2: Partial-order plans BiPOP generated for ServeWater on the real robot under three initial configurations. Blue and red boxes are left- and right-arm actions; boxes side by side run in parallel.
From Figure 5: Outcomes per method over 100 instances of each task domain. Successes (green) are shaded by optimality...
From Figure 5: Outcomes per method over 100 instances of each task domain. Successes (green) are shaded by optimality gap, darkest being the shortest plan found; failures are coloured by failure class.
Abstract

Bimanual robotic manipulation is versatile, yet its central challenge lies not only in executing dexterous motion with each hand, but also in coordinating the two arms for long-horizon manipulation under coupled spatial and temporal constraints. Prior work has largely focused on endowing robots with human-level manipulation skills, paying less attention to the structure of bimanual behaviour itself: many practical tasks are a coordinated composition of reusable unimanual skills and a small set of synchronous bimanual skills. This makes high-level spatio-temporal coordination, rather than skill acquisition alone, a key bottleneck for efficient bimanual execution. We propose BiPOP, a framework that couples Large Language Models (LLMs) with Temporal Partial-Order Planning for long-horizon bimanual manipulation. At its core is an explicit spatio-temporal model that partitions the workspace by arm reachability and captures inter-arm dependencies, arm assignment, synchronization requirements, and opportunities for safe concurrent execution. Building on this model, an LLM translates natural-language instructions into a formal PDDL specification, while a temporal planner enforces the constraints and schedules actions, yielding plans that are logically consistent at the symbolic level and efficient through parallel execution. On simulated long-horizon tasks of varying complexity, BiPOP outperforms open-loop planning with strong reasoning models and closed-loop ReAct-style planning in success rate, planning time, and plan efficiency, and the full pipeline runs end to end on the real bimanual robot NICOL. A failure analysis further shows that LLM-only planners mainly violate spatial and temporal coordination constraints, which motivates delegating these constraints to a symbolic planner. Code: https://github.com/Kchu/BiPOP

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.