← Paper Digest

Déjà-Events: Multi-view 3D Geometry with Event Cameras via Sparsity-Aware Adaptive Gating

Arda Cinar Demirtas, Pablo Varela Bernal, Marc Pollefeys, Sunghwan Hong

November 2026

TL;DR An event-camera-only model that predicts depth, camera poses and pointmaps from several views in one forward pass, distilled from a frozen RGB reconstruction model without any depth or pose labels. Trained only in simulation, it halves the pointmap error of the strongest event baseline on real DSEC driving data (0.15 vs. 0.30 AbsRel) with about a tenth of its parameters.

What they do

Event cameras keep seeing structure where frame cameras blur or saturate, but event datasets with dense 3D labels are small. Déjà-Events sidesteps the labels: it distils Déjà View (DVLT), a feed-forward RGB model that predicts cameras, depth and pointmaps from many views, into an event-only student, using paired RGB frames and events.

How it works

  • Event input. Events in each frame’s window go into a five-bin voxel grid, which a stem turns into a three-channel image (positive, absolute and negative polarity) that the DINOv2 encoder can read.
  • Token distillation. The student copies the teacher and adds LoRA adapters, training 1.6 M of 118 M parameters. It learns to match the teacher’s final tokens, so the frozen decoders give cameras, rays and depth with one loss and no ground truth.
  • Sparsity-aware adaptive gating. DVLT reuses one transformer block K times, scaled by step-dependent gates. A small MLP (2,114 parameters per attention block) corrects each patch’s gate by its event density.

Results

  • Distilled only on simulated EventScape, it has the lowest pointmap error of the event-based methods on EventScape (0.202 AbsRel vs. 0.356 for EventVGGT) and, zero-shot, on MVSEC (0.297 vs. 0.347) and DSEC (0.148 vs. 0.301).
  • Depth AbsRel on DSEC is 0.145 vs. 0.282 for EventVGGT; on MVSEC EventVGGT is better (0.275 vs. 0.298).
  • On a dark MVSEC night drive the student beats its RGB teacher on trajectory error (0.065 vs. 0.156 m).
  • It uses 118 M parameters against EventVGGT’s 1.16 B, at less than half the compute.

Limitations

Outside MVSEC the student stays well behind its RGB teacher (DSEC depth AbsRel 0.145 vs. 0.060), and it can only overtake the teacher where frames fail. On the real datasets DEVO odometry has lower trajectory error. Predicted focal lengths can be far off (1.6x too long on MVSEC). Evaluation is driving data only, and summing the time bins discards when motion happened inside a window.

Figures

From Figure 1: At a tunnel exit the RGB frame is saturated while the event stream keeps its structure. In the blown-o...
From Figure 1: At a tunnel exit the RGB frame is saturated while the event stream keeps its structure. In the blown-out area the RGB teacher predicts flat, ground-plane-like depth, while the event student keeps the vehicles and the tunnel.
From Figure 2: A frozen RGB teacher (top) and an event student with the same architecture (bottom) run the shared blo...
From Figure 2: A frozen RGB teacher (top) and an event student with the same architecture (bottom) run the shared block for the same sampled number of steps, and the loss matches their final tokens. The student trains low-rank adapters and per-patch gate corrections.
From Figure 4: Zero-shot depth on a DSEC sequence from a 50 ms event window, compared with EventVGGT, E2Depth and Dep...
From Figure 4: Zero-shot depth on a DSEC sequence from a 50 ms event window, compared with EventVGGT, E2Depth and DepthAnyEvent. No model was trained on DSEC.
Abstract

Event cameras keep capturing structure where conventional cameras blur or saturate, which makes them attractive for 3D perception in fast or poorly lit scenes. Event datasets with dense 3D ground truth, however, are far smaller and less diverse than the image collections behind feed-forward 3D reconstruction models. We present Déjà-Events, an event-only model that predicts depth, camera pose and pointmaps from multiple views in a single forward pass. Rather than relying on scarce 3D labels, we distil it from Déjà View, a feed-forward RGB model that refines its tokens by repeatedly applying one shared transformer block. On paired RGB and event recordings, the event student learns to reproduce the frozen teacher's final tokens, which requires no depth or pose ground truth and trains only about 1% of its parameters. Because events are dense at edges and moving texture but nearly absent in textureless regions, we further introduce sparsity-aware adaptive gating, which adapts the shared block's gates to events and adjusts them per patch by event density, so that accuracy holds as refinement steps are added. Trained only on simulated data, Déjà-Events achieves the lowest pointmap error among the event-based methods we evaluate, on EventScape and zero-shot on the real MVSEC and DSEC datasets. On DSEC it halves the error of the strongest baseline (0.15 vs. 0.30 AbsRel) with about a tenth of its parameters. It also has the lowest depth error on EventScape and DSEC and the lowest trajectory error on EventScape.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.