ClothTransformer: A Unified Latent-Space Transformer Cuts Cloth Simulation Error by Up to 9×

Researchers have introduced ClothTransformer, a framework that reframes cloth simulation as autoregressive sequence modeling inside a learned latent space. A single Transformer model handles three distinct scenarios — body-driven garments, robotic manipulation, and general rigid-object collisions — without per-scenario fine-tuning, achieving 4–9× lower error than prior state-of-the-art neural simulators.

Three technical contributions underpin the system: a unified Transformer architecture that encodes collision geometry as generic triangle tokens; a scalable latent-space formulation that compresses arbitrary-resolution meshes into a fixed-size token set, decoupling temporal computation from mesh resolution; and a ~493,400-frame penetration-free training dataset spanning all three scenarios, paired with a differentiable Continuous Collision Detection (CCD) module to eliminate tunneling artifacts. The resolution-agnostic design makes ClothTransformer a notable step toward general-purpose physics simulation for VFX and robotics applications.