A 3D AI English Tutor That Runs in a Single HTML File
A convincing AI tutor needs sub-second responsiveness, accurate lip sync, and the ability to be interrupted—three things that cloud-rendered video-stream digital humans consistently fail at. This parameter-stream approach shows that a laptop browser can deliver all three, which lowers the barrier for anyone building interactive learning tools without a GPU server.
Most AI speaking tutors are stuck in a chatbox-and-voice-bar loop that kills the feedback cycle needed for language learning. This build replaces that with a 3D embodied avatar driven by a parameter-stream architecture: the server sends facial muscle, skeletal, and PCM audio parameters, and the browser renders them locally in WebGL. End-to-end latency sits around 1200ms, and mouth shapes stay aligned with speech because animation and audio parameters arrive in the same frame.
The whole thing is a single HTML file. Credentials for the Mofa Nebula 3D platform and any OpenAI-compatible LLM are entered in a collapsible panel and persisted to localStorage. Browser-native Web Speech ASR handles English voice input on a push-to-talk button, and the LLM's streaming response is segmented at punctuation boundaries so the avatar begins speaking the first sentence while the rest is still generating.
Four preset personas—daily conversation, IELTS examiner, business interview, and travel English—swap the system prompt without touching the code. The SDK's speak method is treated as a strict state machine with explicit first-sentence, mid-sentence, and end-sentence calls, and an interactiveIdle() interrupt button clears both the client-side animation buffer and the cloud TTSA session so the student can jump in mid-correction.
Streaming sentence segmentation is counterintuitively better for short-form dialogue than waiting for a full paragraph, because the first sentence's acknowledgment buys time for the correction that follows.
The platform's one-voice-per-application constraint is a real architectural limitation that forces multi-language products into a multi-application setup, but the localStorage credential pattern makes switching nearly frictionless.
Explicitly calling speak('', false, true) when the text buffer is empty at stream-end is an easy-to-miss edge case that silently breaks subsequent conversation turns if skipped.