ForceHarness: Turning Force into Action and Physics for Frozen Frontier Models
TL;DR A harness lets frozen, API-only frontier models use force. Self-terminating force primitives run the contact loop under the model, and each force trace comes back as physical quantities such as mass, friction and contact height. On a Franka FR3 that estimates force from joint torques, GPT-6 Astra goes from 2/60 to 54/60 successes on six contact-rich tasks, with no demonstrations or finetuning.
What they do
Frontier multimodal models can drive robots zero-shot from images. Tasks like driving a screw or sorting look-alike bottles by weight, though, depend on force that images don’t show. A model behind an API decides only every few seconds and can’t read raw force streams. ForceHarness puts three layers between a frozen model and the robot so it can use force without demonstrations or training. The authors also build HiddenPhys, a seven-task MuJoCo benchmark where each task hides one physical variable that decides success.
How it works
- Calibration. Hand force comes from a wrist sensor, or from joint torques on arms without one. Because the torque estimate drifts with pose, the layer re-zeros at rest, reads only after the arm settles and reports changes within an episode.
- Self-terminating primitives. Twelve tools (move_until_contact, slide, search, insert, screw, lift, …) run admittance control through the stock impedance controller and hold the force the model asked for. Each stops on an event such as contact, stall or a resistance drop, not on a timer.
- Physics translation. Fixed estimators turn each force trace into quantities with uncertainty and flags, such as mass, friction, contact height or insertion advance. They apply no task thresholds.
- Loop. Each turn the model reads the images, the history and the last tool result, then makes one tool call.
Results
- On a Franka FR3 without a wrist sensor (10 episodes per task), GPT-6 Astra goes from 2/60 without force to 54/60 across six tasks. GPT-6.1 Sol reaches 47/60.
- On ForceBench, Astra rises from 70.0% to 87.5%, above the 35.31% of the best trained policy reported by ForceBench. Most of the gain from force comes on Drawer Pull (0/10 to 10/10).
- On HiddenPhys, Astra and Claude Fable 5.2 both reach 135/140 with zero handle damage. Hand-written expert controllers reach 136/140, and Diffusion Policy, π0.5 and ForceVLA trained on about 200 demonstrations per task reach 65–83/140.
- For smaller GPT-5.x models, returning physical quantities instead of raw numbers raises correct verdicts on Weight, CoM and Lock from 0.69 to 0.85.
Limitations
Model queries add seconds between primitives, which rules out reactive coordination, and the fixed menu of primitives limits new motion laws. Ablations run only in simulation. On ForceBench, tools and thresholds were tuned on eight of the ten test seeds, and the trained baselines are quoted from the ForceBench paper rather than rerun.
Videos
Figures



Abstract
Frontier multimodal models can now control robots zero-shot from images, yet many manipulation tasks depend on contact forces that images cannot show. Bringing force to an API-only frozen model raises two challenges. First, contact evolves much faster than an API model can decide, so the system must close the loop on force beneath the model. Second, a frozen model cannot interpret raw force streams, so force must be returned in a form the model can reason about. Therefore, we present ForceHarness, a harness that lets frozen models use force without demonstrations or finetuning. After calibrating hand force from a wrist sensor or joint torques alone, ForceHarness addresses the first challenge with self-terminating force primitives that regulate force on their own and return at a contact event. To address the second challenge, a physics translation layer turns each force trace into physical quantities with uncertainty as the tool result. To evaluate ForceHarness against expert force controllers and learned policies under identical conditions, we also build HiddenPhys, a seven-task MuJoCo benchmark with hidden physical variables. On a real arm without a wrist force sensor, ForceHarness raises the zero-shot success rate of GPT-6 Astra from 3.3% to 90.0%. In simulation, it adds 17.5 percentage points on the public ForceBench suite and 34.3 percentage points on HiddenPhys, surpassing force-aware policies trained on demonstrations on both benchmarks and matching hand-written expert controllers on HiddenPhys. Ablations further show that smaller models make significantly better decisions when given physical quantities rather than raw force values.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
