NVIDIA Engineer Builds Interactive Talking Garage Door for Halloween Using Nemotron Voice Agent Stack

NVIDIA engineer Rick Grandy built MonsterHouse, a Halloween projection that animates a grumpy, pirate-voiced garage door capable of holding real conversations with trick-or-treaters. The system chains together four Nemotron open models — Nemotron ASR for speech recognition, Nemotron 3.5 Bolt E4B 4GB as the conversational brain, Magpie TTS Zeroshot for live voice synthesis, and Audio2Face-3D to drive mouth animation in Unreal Engine 5 — all coordinated by the Nemotron Voice Agent and running on a single RTX PRO workstation.

The character rig, a wooden garage-door mouth built in Autodesk Maya and exported into Unreal Engine, was backstoried as a house constructed from sunken pirate-ship timbers. A LoRA adapter shaped the gruff personality, while a YOLO/ByteTrack vision layer counted and tracked visitors and Nemotron Nano Omni described their costumes. The project doubles as a practical showcase of the full Nemotron voice-agent pipeline running on consumer-grade hardware.