Déjà View: 117M-Parameter Looped Transformer Beats Billion-Parameter 3D Reconstruction Baselines

Déjà View is a new transformer model for multi-view 3D reconstruction that replaces stacked unique layers with a single transformer block applied recurrently at inference time. At just 117M parameters, it matches or outperforms feed-forward baselines ranging from 356M to 1.26B parameters across five benchmarks covering indoor, outdoor, object-centric, and driving scenes — using 8–10× fewer parameters and 1.9–2.3× less compute.

The key architectural insight is that depth in conventional reconstruction transformers effectively buys iterative refinement, but pays for it inefficiently in unique parameters. By making iteration explicit through a looped block conditioned on a continuous time interval, Déjà View exposes the number of refinement steps K as an inference-time compute knob. A single trained checkpoint covers any step count within the training range, letting users trade speed for quality without retraining. The authors also report that the looped formulation outperforms an otherwise identical model with independent per-step parameters under matched training data and compute, suggesting explicit iteration is a stronger inductive bias — not merely a parameter-efficiency trick.