GenIA Aligns Frozen 3D Generative Models to Real-World Inputs at Test Time Without Retraining

Researchers have introduced GenIA, a test-time framework that adapts the frozen SAM3D generative 3D foundation model to monocular images, multi-view images, and video without any retraining. By applying geometric and photometric constraints during inference — including visibility-biased attention, cross-observation fusion, and differentiable rendering guidance — GenIA grounds SAM3D's generative prior in observed data, correcting pose, geometry, and appearance alignment that the base model may otherwise miss.

The framework operates in four stages per scene: coarse shape acquisition (or external shape injection), per-frame pose denoising, observation-aligned appearance prediction, and an optional test-time refinement of the appearance latent and a low-rank decoder adapter. Across synthetic and real benchmarks, GenIA outperforms recent optimization-based, per-frame image-to-3D, and video-to-4D methods on held-out PSNR, LPIPS, and CLIP-I metrics, while adding only minimal runtime overhead over the base SAM3D model running on an A100 GPU.