← Paper Digest

DreamFAR: Real-Time and Long-Horizon Interaction with Diverse Objects

Yiming Dou, Botao He, Antonio Loquercio, Andrew Owens, Jitendra Malik, Sumedh Anand Sontakke, Ruoshi Liu

November 2026

TL;DR A video world model trained from scratch on bimanual teleoperation data, which remembers the frames where the robot makes or breaks contact and streams at 10 Hz. Over 60-s rollouts on 1,000 held-out episodes it averages 16.4 dB PSNR, against 15.8 for Cosmos-Predict 2.5 and 14.5 for a size-matched WEAVER.

What they do

DreamFAR is an action-conditioned video world model for a bimanual robot, trained from scratch on teleoperation data with no internet-video pretraining. It streams frames at 10 Hz, so a policy or a human on leader arms can act inside it in closed loop, and keeps contact-rich rollouts coherent for a minute or more.

How it works

  • Tokenizer. A fine-tuned SSDD encoder maps each of three 256x256 camera views to a 16x16 grid of latents projected onto the unit sphere, which limits scale drift.
  • Contact-based memory. An auxiliary head predicts one contact bit per arm; when one flips (a grasp or a release), that frame enters a keyframe bank. Up to eight keyframes, retrieved by end-effector-pose similarity, join a 5-s window of recent frames.
  • Streaming transformer. 1.9B parameters with a learned, never-evicted sink frame, trained with per-frame noise levels (diffusion forcing). At inference 16 frames at staggered noise levels are denoised together over a rolling KV cache, so cost per step stays constant.
  • Data. An internal teleoperation dataset on YAM arms; 256 H200 GPUs for six days.

Results

  • Against baselines retrained on the public ABC dataset (DreamFAR and WEAVER both at about 0.8B), 60-s rollouts on 1,000 held-out episodes average 16.4 dB PSNR, versus 15.8 for Cosmos-Predict 2.5, 14.5 for WEAVER and 11.0 for IWS. FID rises from 15 to 32 over the minute, versus 30 to 73 for Cosmos-Predict 2.5.
  • Policy evaluation with two π0 checkpoints on three tasks (32 episodes each): mean task progress inside the model and on the real robot correlates at Pearson r = 0.71.
  • Training π0 on 75 real plus 75 demonstrations collected inside DreamFAR nearly doubles real-only task progress, to 63% over 10 real trials.
  • Contact memory beats fixed-stride and no memory on wrist-view PSNR (19.25 vs 18.99 vs 18.66).

Limitations

One YAM station, one embodiment and parallel grippers only. The model is optimistic: it reports higher progress than the real robot and preserves the policy ranking on two of three tasks. It is trained on mostly successful demonstrations, so the authors warn it should not be the only check before deploying a policy.

Figures

From Figure 1: DreamFAR, a world model trained on robot-native data, takes live teleoperation at 10 Hz, rolls out for...
From Figure 1: DreamFAR, a world model trained on robot-native data, takes live teleoperation at 10 Hz, rolls out for minutes autoregressively and generalizes across more than 1,000 tasks.
From Figure 2: (a) Each camera view is encoded as latents projected onto the unit sphere and decoded by a flow decode...
From Figure 2: (a) Each camera view is encoded as latents projected onto the unit sphere and decoded by a flow decoder. (b) Frames where the robot makes or breaks contact with an object go into a contact memory indexed by end-effector pose. (c) A streaming transformer predicts the next frames.
From Figure 5: PSNR, LPIPS and FID of 60-s rollouts on 1,000 held-out ABC episodes, averaged in 10-s bins. DreamFAR d...
From Figure 5: PSNR, LPIPS and FID of 60-s rollouts on 1,000 held-out ABC episodes, averaged in 10-s bins. DreamFAR degrades the slowest on all three metrics.
From Figure 8: One rollout frame from three models that differ only in memory. Without memory the bin is empty, time-...
From Figure 8: One rollout frame from three models that differ only in memory. Without memory the bin is empty, time-strided memory shows the wrong contents, and contact-based memory matches the real observation.
Abstract

World models have shown great potential in serving as a simulator for robot manipulation. However, existing models are either fit to a single environment or lack ability to simulate long-horizon contact-rich events. We present DreamFAR, a world model whose dynamics model is trained from scratch on a large robot-native teleoperation dataset, with no internet-video pretraining. We present an architecture and learning recipe designed for training on robot-native data. DreamFAR is significantly smaller than the high-performance video generation models, yet shows better understanding of contact and causality, and runs 10 FPS on a commercial GPU. Experimentally, DreamFAR significantly outperforms existing models in predicting the long-horizon future. We further demonstrate various applications where DreamFAR can be used, including policy evaluation and mixed-data co-training. Project page: https://www.yimingdou.com/DreamFAR/

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.