← Paper Digest

RoboKit: Accelerating Parallel Differentiable Optimization for Robot Learning

Jianglong Ye, Xuanbin Peng, Conor Mc Gartoll, Changwei Jing, Lai Wei, Jiali Chen, Xueyan Zou, Xiaolong Wang

November 2026

TL;DR RoboKit is a GPU toolkit, written in Warp, that runs many optimizations in parallel for IK, trajectory optimization, motion retargeting, grasp synthesis, camera calibration and sampling-based MPC. It runs 2-60x faster than task-specific baselines, for example 4,096 dexterous grasps in 2.97 s versus 178.45 s. Policies distilled from it run 9-240x faster than the solver itself.

What they do

Robot learning still leans on classical optimization for teleoperation, data generation and planning. RoboKit is one GPU toolkit for these optimization tasks: inverse kinematics, trajectory optimization, hand and humanoid retargeting, dexterous grasp synthesis, rendering-based hand-eye calibration and physics-based MPC. Its cost terms are differentiable, so the same objectives can train policies.

How it works

  • Parallel solver. Built in Warp, it pairs multi-seed particle sampling with a choice of optimizer per task: Adam, L-BFGS, Gauss-Newton/Levenberg-Marquardt, or MPPI. Cost terms are composable and support sparse Jacobians.
  • Acceleration structures. A voxel SDF switches to exact mesh queries only near the surface, and one bounding sphere per link prunes collision checks. Calibration uses a coarse-to-fine rendering pyramid, and physics rollouts start coarse.
  • Distillation. Offline distillation trains on filtered solver outputs. Online distillation backpropagates the cost residuals straight into the policy. Grasping needs an offline warm-up before online fine-tuning.

Results

  • IK is 7.1 vs 86.2 ms for PyRoKi at batch 2,000. Trajectory planning is 0.016 vs 0.057 s for cuRobo, at comparable success.
  • Grasp synthesis succeeds 46.83% of the time vs 23.63% for Dex1B, though Dex1B has higher grasp quality (Q1 0.109 vs 0.077). From a 35 cm / 45 degree initial error, calibration reaches 1.10 cm in 2.0 s, while EasyHEC does not converge.
  • Physics-based hand retargeting raises ADD-AUC on OakInk from 51.3 to 65.8 over SPIDER and runs 5.6x faster.
  • A distilled trajectory policy takes 0.63 vs 150.61 ms per frame, and a distilled hand-retargeting policy lowers position error from 7.79 to 4.80 mm. RL reaches mjlab’s final score 1.3-2.1x sooner.

Limitations

RoboKit targets batched GPU workloads. For single-instance kinematics on a CPU, kernel-launch overhead dominates and a C++ solver would be faster. Trajectory optimization has no graph-search stage, and the toolkit lacks inverse dynamics. All evaluations use existing simulated benchmarks and datasets; the paper reports no real-robot experiments.

Figures

From Figure 1: Representative RoboKit tasks: whole-body grasping, trajectory optimization, whole-body retargeting and...
From Figure 1: Representative RoboKit tasks: whole-body grasping, trajectory optimization, whole-body retargeting and rendering-based calibration.
From Figure 2: RoboKit has three components: a parallel solver over differentiable objectives, acceleration structure...
From Figure 2: RoboKit has three components: a parallel solver over differentiable objectives, acceleration structures for geometry, rendering and physics, and policy distillation through offline and online learning.
From Figure 3: Distilled policies on trajectory optimization, hand and full-body retargeting, and dexterous grasp syn...
From Figure 3: Distilled policies on trajectory optimization, hand and full-body retargeting, and dexterous grasp synthesis.
Abstract

We present RoboKit, a high-performance, general-purpose toolkit for parallel differentiable optimization in robotics, built on three principles: (i) Performance and flexibility: GPU-parallel multi-seed solvers pair sampling with versatile optimizers, running 2-60$\times$ faster than task-specific baselines across seven tasks; (ii) Geometry, rendering and physics: hybrid and coarse-to-fine representations accelerate collision, differentiable rendering and simulation, optimizing 4096 grasps in 2.97 s; (iii) Integration with learning: costs are differentiable and simulation is parallel, so RoboKit both distills optimization into policies and trains RL; distilled policies run 9-240$\times$ faster at comparable accuracy, and RL reaches a GPU baseline's scores up to 2.1$\times$ sooner on the same simulator.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.