GaussianGPT: Autoregressive Transformer Generates Full 3D Scenes via Next-Token Prediction
![[ECCV'26] Our method creates 3D Gaussian scenes completely autoregressively, allowing for generation, completion, and outpainting with the same model. - nicolasvonluetzow/GaussianGPT](https://genbuzz.news/media/representations/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MjMyMiwicHVyIjoiYmxvYl9pZCJ9fQ==--15d04f33268d6732adaaa7e3dd9ac4a23a679e6a/eyJfcmFpbHMiOnsiZGF0YSI6eyJmb3JtYXQiOiJwbmciLCJyZXNpemVfdG9fZmlsbCI6WzEyMDAsNjAwXX0sInB1ciI6InZhcmlhdGlvbiJ9fQ==--a513445e9fd289d825a28e8cddd66184d60366bc/GaussianGPT.png)
Image via github.com
Researchers from TU Munich have introduced GaussianGPT, an autoregressive transformer model that generates 3D Gaussian scenes through next-token prediction — offering an alternative to the diffusion and flow-matching approaches that dominate current 3D generative modeling. The pipeline first compresses Gaussian primitives into discrete tokens using a sparse 3D convolutional VQ-VAE, then models those token sequences with a causal transformer using 3D rotary positional embeddings.
Because scenes are built sequentially rather than refined holistically, GaussianGPT naturally supports scene completion, outpainting, temperature-controlled sampling, and flexible generation horizons. The work has been accepted to ECCV 2026, with training and inference code, pre-trained VQ-VAE, and GPT checkpoints now publicly available.