Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

SPARK: Simple Post-training for Adapting pRetrained Knowledge to Robot Control

Taekyung Kim, Jeongeun Park, Byeongho Heo, Sungjoon Choi, Sangdoo Yun, Dongyoon Han

November 2026

TL;DR SPARK post-trains off-the-shelf vision encoders (CLIP, SigLIP, DINOv2, Theia) on ordinary video so their class token captures how a scene changes, while patch features stay close to the original model. Frozen SPARK features raise average imitation-learning scores by 4.0-5.3 points across MetaWorld, DMC and Franka Kitchen, and success by 5-10 points on three real-robot tasks.

What they do

Vision encoders such as CLIP, SigLIP and DINOv2 are trained for recognition and language alignment, not for the scene changes that matter for control. Training robot encoders from scratch, on the other hand, needs costly robot data. SPARK is a post-training recipe that adapts an existing encoder on Kinetics-400 video, with no captions or actions, so that its class token becomes a compact visual state for policies while the encoder keeps its pretrained semantics.

How it works

  • Temporal abstraction. The encoder reads a reference frame and outputs a class token. From that token plus 10% of a later frame’s patches (4-96 frames ahead), an eight-block ViT decoder predicts the later frame’s remaining tokens. These predictions are matched to the frozen original encoder’s features.
  • Knowledge preservation. A second loss keeps the adapted encoder’s patch tokens close to the frozen original’s on the same frame.
  • Use. After post-training, the encoder is frozen and a behavior-cloning policy reads its class token, plus proprioception where available.

Results

  • Simulation (five tasks each in MetaWorld, DMC and Franka Kitchen): average gains over the vanilla encoder are 5.3 points for CLIP, 5.1 for SigLIP, 4.0 for DINOv2 and 4.3 for Theia, largest on Franka Kitchen (+11.0 for SigLIP). SPARK-Theia reaches 89.2% on MetaWorld; the best robot-specific encoder (data4robotics) reaches 87.0%.
  • Real robot (CLIP backbone, 20 demonstrations and 20 trials per task): cloth folding 65 → 75%, cup stacking 45 → 55%, cabinet opening 50 → 55%.
  • Ablations on Franka Kitchen with SigLIP: either loss alone falls below the vanilla encoder (14.6% and 22.8% vs 31.2%), while both together reach 42.2%. Starting from CLIP (31.5%), MAE-style post-training scores 23.0% and SPARK 39.8%.
  • Flickr30K retrieval with the original SigLIP text encoder drops by at most 2.8 points (image-to-text R@1).

Limitations

Absolute performance still depends mostly on the capacity of the base encoder, and the temporal objective predicts only one offset frame. The real-robot test uses only a CLIP backbone on three tasks with 20 trials each. The encoder stays frozen during policy learning; training it jointly inside a full VLA is left to future work.

Figures

From Figure 1: Overview of SPARK. The SPARK encoder turns a reference scene into a class token, a compact abstraction...
From Figure 1: Overview of SPARK. The SPARK encoder turns a reference scene into a class token, a compact abstraction from which a decoder predicts the target scene's tokens with hints from a few target patches. The temporal abstraction loss matches these predictions to a frozen foundation model's target tokens, and the knowledge preservation loss keeps the SPARK encoder's target patch tokens close to the frozen model's.
From Figure 2: One frame of a real-world Cup Stacking rollout by the SPARK-adapted policy.
From Figure 2: One frame of a real-world Cup Stacking rollout by the SPARK-adapted policy.
Abstract

Large-scale representation learning has produced public foundation models that provide strong general-purpose visual features. However, their pretraining objectives are not designed for robot representation learning, so the resulting embeddings tend to emphasize global semantics rather than the temporally sensitive representations required for robot control. Training robot models from scratch is also undesirable due to the limited scale and high collection cost of robotics data, as well as the substantial computational burden of pre-training. In this paper, we propose SPARK (Simple Post-training for Adapting pRetrained Knowledge), a post-training method that adapts foundation models to robot control. SPARK is built on two principles: temporal abstraction, which encourages the encoder to derive compact features that preserve information essential for understanding temporal changes and action-relevant scene structure, and knowledge preservation, which aligns patch-level representations with those of the original foundation model to retain useful pretrained semantics. These objectives yield compact visual state representations that remain semantically meaningful while preserving useful prior knowledge. Experiments across multiple robotics benchmarks show that SPARK consistently improves success rates and generalization over vanilla foundation models, and further demonstrate that these gains transfer to real-world robot manipulation.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.