A 500ms Pipeline Turns LLMs Into Real-Time 3D Storytellers That Kids Can Interrupt
Most LLM demos stop at a chatbox, but real-time embodied interaction — sub-second latency, interruptibility, lip-sync — is what makes AI usable for children, retail kiosks, and service robots. This stack shows that a single developer can wire off-the-shelf Chinese LLMs and a commercial embodied SDK into a production-like pipeline without building the animation or TTS layer from scratch.
A Vue 3 + Vite frontend wires a large language model to a 3D digital human SDK so that a child's spoken request produces a story performed by a cartoon avatar with real-time lip-sync, expressions, and gestures. The pipeline runs speech recognition through the browser's Web Speech API, sends the transcript to a configurable LLM backend (DeepSeek, Tongyi Qianwen, or others via OpenAI-compatible endpoints), and feeds the generated text to Mofa Nebula's XmovAvatar SDK, which drives the avatar on-screen. End-to-end latency sits around 500 ms, and the SDK's `interactiveidle` + `speak` pattern lets the child interrupt the avatar mid-story to ask a new question.
Trae Work generated the initial project skeleton and boilerplate composables, including edge-case handling for unsupported browsers and permission denials in the speech-recognition module. One notable integration pitfall: a Qwen model returned empty `content` fields and buried the story inside a `reasoning` payload, requiring a regex-based extraction function that hunts for Chinese text blocks across several fallback patterns.
The deployed app wraps all technical configuration in child-friendly language (AppID becomes "Magic Key Number") and stores API keys only in localStorage. Parents configure model credentials and avatar keychains through a settings panel; the child sees only a press-to-talk button and a 3D character that wakes up, tells stories, and switches between monkey, bear, and bunny avatars.
Interruptibility is the feature that separates a toy from a usable interactive agent; without it, the avatar just becomes a slow text-to-speech player.
Empty `content` fields in otherwise OpenAI-compatible endpoints are a real integration hazard — the fix is a regex fallback chain, not a prompt tweak.
Wrapping technical configuration in fantasy language isn't just cosmetic; it's what lets a non-technical parent set up an LLM-backed app without calling it an LLM.
The same composable that drives a browser avatar can drive a physical robot, because the SDK abstracts the embodiment layer behind a uniform `speak` interface.