ALIVE: Bring Foundation Features to Life with Events

TL;DR ALIVE uses an event camera to keep frozen DINOv3 features up to date between RGB frames, so detection, segmentation and depth heads stay current without re-running the encoder. On DSEC with a 200 ms-old frame it adds 30.9 mAP in detection over the stale features, at 82.4% fewer FLOPs than re-encoding.
What they do
Features from a visual foundation model describe the scene only at the moment a frame was captured, and every prediction built on them goes stale until the next frame is encoded. ALIVE pairs the frame camera with an event camera and uses the events in between to update the features of a frozen DINOv3 encoder directly. Because all task heads read those features, one update refreshes detection, segmentation and depth at once, with no task labels.
How it works
- Anchor and events. DINOv3 ViT-B/16 encodes the latest frame into features from four layers. Events arrive in 5 ms packets, each turned into a voxel grid and encoded by a small CNN.
- Updater (2.5M parameters). Per packet it predicts a per-token displacement that moves the features and a ConvGRU memory along the motion, adds a residual correction for changes motion does not explain, and refines tokens at object boundaries with a zero-sum 3x3 kernel.
- Training. Only on synthetic TartanAir V2 sequences, supervised by DINOv3 features of later frames (tokenwise, token-relation and temporal-increment losses), without task heads.
- Transfer. At test time, parameter-free moment matching aligns each channel’s mean and variance with the anchor features before frozen heads read them.
Results
- Zero-shot on three real datasets (DSEC, DDD17, CoSEC) and five task-dataset settings, it beats all eight event-based baselines in every metric.
- Against the stale anchor features on DSEC, pooled over anchor ages: +24.6 mAP in detection, +9.1 mIoU in segmentation and 11.0% lower depth AbsRel. With a 200 ms-old anchor the gains are 30.9 mAP, 11.6 mIoU and 15.9%.
- An update takes 65.0 GFLOPs and 4.66 ms: 82.4% fewer FLOPs than re-encoding with DINOv3 and 2.5x faster. It is the only evaluated method that can update at 200 Hz.
- Ablations: removing feature transport costs 10.5 mAP; without moment matching, depth AbsRel rises from 0.129 to 0.162.
Limitations
The paper has no limitations section. Its own results leave a gap to encoding the current frame (52.6 vs 64.4 mAP in DSEC detection), accuracy still falls as the RGB anchor ages (more slowly than the stale features), and the evaluation is offline on recorded datasets with no robot in the loop. Code is not yet released.
Figures




Abstract
Visual foundation models provide powerful representations for diverse tasks, but their features become stale as visual content changes between image observations. Refreshing them requires new frames and repeated runs of a computationally expensive image encoder. We introduce ALIVE, an event-based method that incrementally updates native foundation model features between image observations. Given an anchor image, ALIVE uses subsequent events to transport and correct existing multi-layer features extracted by a frozen DINOv3 encoder. Its lightweight updater is trained end-to-end using only synthetic data, without downstream task supervision, and transfers directly to real-world event streams without any adaptation. Across three real-world datasets and five task-dataset settings spanning object detection, semantic segmentation, and depth estimation, ALIVE outperforms all evaluated event-based methods in every metric. On DSEC, when the anchor frame is 200 ms old, ALIVE improves over anchor-image predictions by 30.9 mAP in detection, 11.6 mIoU in segmentation, and 15.9% in depth AbsRel. Compared with image re-encoding, ALIVE reduces FLOPs by 82.4% and latency by a factor of 2.5.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
