AnchorHEC: Targetless Hand-Eye Calibration of Wrist and Fixed Cameras

Guebin Hwang, Taeyeop Lee, Yeonseo Lee, Hyosup Shin, Sungho Jo

November 2026

Estimated camera poses against ground truth in simulation, then the calibrated cameras on the physical robot.

TL;DR AnchorHEC calibrates a robot's wrist camera and fixed cameras in one run from ordinary RGB images of the scene and the arm's forward kinematics, with no calibration board. On a physical xArm6 with 15 wrist poses it reaches 1.2-1.4 cm wrist-camera and 1.6-1.8 cm fixed-camera translation error, versus 2.4-10.2 cm and 3.4-8.8 cm for the JCR baseline.

What they do

Many manipulation setups pair a wrist camera with fixed cameras, and each needs its transform to the robot: wrist camera to end effector, fixed camera to robot base. Target-based calibration needs a board in view, robot-body methods need CAD models, trained models or a chosen reference point, and scene-based methods built on 3D reconstruction handle only wrist cameras. AnchorHEC calibrates both camera types in one procedure from RGB images of a static, textured scene, forward kinematics (FK) and known intrinsics.

How it works

Shared initialization. A 3D foundation model (DUSt3R, MASt3R or VGGT) reconstructs all wrist and fixed-camera images in one frame. FK selects wrist-view pairs with diverse rotation axes to solve for the wrist transform and reconstruction scale, which also places the fixed cameras in the robot base frame.
Wrist refinement. Bundle adjustment on SIFT tracks across the wrist images, with exact FK supplying metric scale. If it fails or keeps under 30% of the observations, it reruns with priors that let FK poses and lens parameters adjust.
Fixed-camera registration. Dense matches between fixed and wrist images are triangulated into metric points using the refined wrist transform and FK; PnP with RANSAC then registers each fixed camera. A second pose fit on the rejected matches detects repetitive scenes that support two poses, and the method falls back to the reconstruction-based estimate.

Results

Simulation (Isaac Sim, Franka, room and tabletop scenes, 24 backbone and pose-budget conditions): wrist translation and rotation errors are 84.6% and 70.0% lower than JCR’s on average. On the 55 of 60 runs per backbone where bundle adjustment succeeds, every fixed-camera registration is accepted, with mean errors of 1.29-1.31 cm.
Physical xArm6 with two Azure Kinects, 15 wrist poses: wrist translation error 1.35 cm (room) and 1.22 cm (tabletop) versus 10.16 and 2.41 cm for JCR; fixed camera 1.63 and 1.75 cm versus 8.76 and 3.42 cm for JCR + SF.
Over 880 simulated fixed-camera estimates, the ambiguity test cuts mean error from 11.32 cm to 2.24 cm.

Limitations

Physical validation covers two acquisitions with one fixed camera each, and calibration after remounting the wrist camera is untested. The method assumes rigid mounts, known intrinsics, a static textured scene and wrist rotations about non-parallel axes. With only 5 wrist poses, physical errors rise to 3.5-8.0 cm. Relaxing FK and lens parameters helps in some acquisitions and hurts in others, and an accepted registration can still be worse than the fallback (62 of 440 simulated room registrations by more than 1 cm).

Figures

From Figure 1: AnchorHEC overview. A joint 3D reconstruction and forward kinematics (FK) initialize the camera-to-rob... — From Figure 1: AnchorHEC overview. A joint 3D reconstruction and forward kinematics (FK) initialize the camera-to-robot transforms of the wrist and fixed cameras; the refined wrist transform and FK then anchor metric scene geometry for registering the fixed cameras. Green markers are end-effector poses; orange and blue are wrist and fixed cameras.

From Figure 5: Physical-robot setup and results in the (a) room and (b) tabletop acquisitions. Left: the robot with i... — From Figure 5: Physical-robot setup and results in the (a) room and (b) tabletop acquisitions. Left: the robot with its wrist and fixed cameras. Right: the reconstructed pointmap placed in the robot base frame with the estimated wrist (orange) and fixed (blue) cameras, for a 15-pose example.

From Figure 4: Simulation results in (a) room and (b) tabletop scenes with 15 wrist poses and four fixed cameras. Lef... — From Figure 4: Simulation results in (a) room and (b) tabletop scenes with 15 wrist poses and four fixed cameras. Left: the pointmap in the robot base frame with the estimated wrist (orange) and fixed (blue) cameras. Right: ground-truth and estimated poses for one wrist view and four fixed cameras, AnchorHEC versus JCR and JCR + SF.

Abstract

Recent robot systems combine wrist and fixed cameras, requiring camera-to-robot transforms for both. Most hand-eye calibration methods rely on targets or robot-body observations. Scene-based methods using 3D reconstruction require neither but address only wrist cameras. We introduce AnchorHEC, a targetless hand-eye calibration framework for wrist and fixed cameras in a single procedure using RGB images, forward kinematics (FK), and known camera intrinsics. We initialize both camera types using FK and a shared 3D foundation model reconstruction. FK-constrained wrist-image bundle adjustment reduces the impact of reconstructed pose and scale errors. If this refinement fails or retains too few inliers, we relax the constraints on potentially inaccurate FK poses and lens parameters. We use the refined wrist transform and FK to triangulate metric points and estimate fixed-camera poses. Across three backbones, calibration errors in simulation decrease by 86.4% in translation and 71.6% in rotation, averaged over both camera types, relative to reconstruction-based baselines. We also validate AnchorHEC on a physical robot. Project page: https://hwanggb.github.io/AnchorHEC/

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong