PolicyIK: High-Precision Inverse Kinematics for Robot Arms with Reinforcement Learning

Yuhan Liu, Xinqi Liu, Zixuan Dong, Keith W. Ross, Shenji Wan

November 2026

TL;DR An RL policy does inverse kinematics by nudging joint-position references, trained to reach and then hold full poses within 1 mm and 0.2°. Trained directly on a physical OpenArm, it holds 43 of 60 poses through the final 5 s (71.7%), against at most 28.3% for four classical IK controllers.

What they do

Many VLA policies output end-effector poses and leave the conversion to joint commands to an IK controller, so IK accuracy limits every policy built on top of it. On a position-controlled arm, gravity, finite gains and joint limits make the arm settle away from the goal. PolicyIK learns closed-loop IK with goal-conditioned RL. Its target is full-pose precision of 1 mm and 0.2°, which the arm must reach and then hold.

How it works

Reference accumulation. Each action is an increment to a joint-position reference that a PD controller tracks. The policy also sees the reference minus the measured joints, so a zero action keeps the offset needed to hold a pose against gravity.
Two-scale pose errors. The pose error is observed clipped at a coarse (1 cm, 20°) and a fine (0.5 mm, 0.25°) scale, and an error-dependent gain shrinks steps near the goal.
Reward. A progress term pays only when the best error so far improves; a dwell term pays for every step inside the tolerance.
Training. PPO in 4,096 Isaac Sim environments, or FlashSAC directly on hardware with hindsight goal relabeling, near-goal starts, 150 scripted demonstrations and a latent-alignment critic for large-rotation detours.

Results

Simulated 7-DoF OpenArm, 4,120 held-out tasks: 93.5% strict success in one attempt (median 0.25 mm, 0.04°), 98.9% with retries. DLS, QP and MPC score 43, 37 and 37 points lower, and an adapted UMI on Legs RL recipe gets no strict successes.
Physical OpenArm, trained on hardware for 120,000 transitions: reaches 54 of 60 tasks and holds 43 through the final 5 s.
Physical xArm7: 60/60 on standard and 60/60 on large-rotation tasks, for both simulation- and hardware-trained policies.
Simulated LIBERO-Object pick-and-place driven by π0.5: 95.7% success, against 96.7% for hand-tuned DLS and 90.8% for default DLS and hand-tuned QP.

Limitations

The paper has no limitations section. Hardware errors come from joint encoders through forward kinematics, not from an external measurement, and the VLA pick-and-place test is simulation-only. On the OpenArm, a 15 s budget leaves only 27 of 60 tasks held for 5 s, and a separate policy is trained for each arm and mounting.

Figures

From Figure 1: (a) full-pose reaching across the simulated workspace; (b) reaching and then holding within 1 mm and 0... — From Figure 1: (a) full-pose reaching across the simulated workspace; (b) reaching and then holding within 1 mm and 0.2 degrees; (c) on hardware, PolicyIK holds the pose through the final 5 s on more tasks than classical controllers.

From Figure 2: At each step the policy maps the observation to a joint-reference increment, which is scaled by an err... — From Figure 2: At each step the policy maps the observation to a joint-reference increment, which is scaled by an error-dependent gain, accumulated in the joint reference and tracked by a joint PD controller.

From Figure 6: On the physical arm, MPC keeps correcting after reaching the goal and repeatedly leaves the 1 mm toler... — From Figure 6: On the physical arm, MPC keeps correcting after reaching the goal and repeatedly leaves the 1 mm tolerance, while PolicyIK holds the pose.

Abstract

Inverse kinematics (IK) converts end-effector pose commands into joint motion, and many vision-language-action (VLA) policies rely on it to act on a given arm and to transfer across embodiments. Precise IK remains laborious: redundancy and singularities complicate the inverse mapping, execution depends on hand-tuned controllers, and training data on physical robots are costly to collect. We present PolicyIK, a goal-conditioned reinforcement learning policy for high-precision IK. It updates joint-position references from pose feedback and observes its accumulated reference relative to the measured joints, so it can retain the offset that the position controller needs at rest. Two-scale pose errors with an error-dependent action gain resolve millimeter residuals while preserving large motions. Progress and dwell rewards teach it to both reach the tolerance and stay within it. In simulation, a single attempt by the trained PPO policy keeps the mean errors over the final second within 1 mm and 0.2° on 93.5% of held-out tasks, and a deployment protocol raises success to 98.9%. When trained directly on a physical arm with an off-policy RL learner, PolicyIK achieves higher reaching and holding success than all four classical IK controllers under the same time budgets. In simulated pick-and-place, PolicyIK executes the end-effector commands of the $π_{0.5}$ VLA policy with 95.7% success. These results show that reinforcement learning can provide high-precision inverse kinematics for robot arms. Code is available at https://github.com/Tinaliu0123/PolicyIK.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong