Thinking Machines Releases Inkling: 1-Trillion-Parameter Open Multimodal LLM with 1M Context Window

Thinking Machines has launched Inkling, a 1-trillion-parameter open multimodal large language model that natively ingests text, image, and audio inputs within a 1-million-token context window. The decoder-only Mixture-of-Experts model carries 975B total parameters but activates only 41B at inference time, and was trained on 45 trillion tokens spanning text, images, audio, and video. It ships in full BF16 and an NVFP4 quantized variant, with speculative MTP layers for faster throughput.

Inkling is available on Hugging Face with day-0 integration across the Transformers library, SGLang, and llama.cpp. The model is positioned for domain adaptation via fine-tuning and is aimed at developers building multimodal reasoning applications. Architectural highlights include relative attention (in place of RoPE), a hybrid global/sliding-window attention scheme, short 1D convolutions over hidden states, and lightweight hierarchical MLP towers for vision and audio encoding rather than separate dedicated encoders.