A 3D Digital Human That Speaks in Your Browser, Driven by a Parameter Stream

5 Minutes to Put a Talking 3D Digital Human in Your Webpage

Let's start with the result: this effect is not a video I recorded—it's a 3D digital human rendered in real time in the browser. You give it a piece of text, and it synthesizes speech in real time, syncs the lip movements, and adds expressions and actions.

Insert image description here

You can open the online version directly to experience it (just fill in an App ID/Secret to play):

🔗 https://likebeans.github.io/xingyun3D/

If you have a Mofa Xingyun account, using the invitation code XDZARL7NEP to register will also give you 1000 credits, enough for you to run demos for a long time.

Below, I'll guide you from scratch on how to integrate this into your own webpage.

1. What It Actually Does

This is not "a pre-recorded video," but a real-time driven 3D digital human:

You call a speak(text);
The cloud synthesizes the text into a parameter stream of speech, expressions, lip movements, and actions, and sends it down to the browser;
The browser renders (AI end-rendering) the visuals, aligned with the audio.

So it achieves "what you see is what you hear, with millisecond-level response." This is also the foundation for connecting a large model later to make a real-time AI anchor (there will be a dedicated article on that later).

2. Environment Requirements

Let's state the prerequisites first to save you from pitfalls:

Browser: The latest version of Chrome / Edge / Safari is recommended (the SDK relies on WebGL2 hardware-accelerated rendering);
Access Method: The SDK only supports localhost or https. Directly using "IP + port" or a bare http domain will cause an error (this is a pitfall I recorded in my pitfalls article);
Account Credentials: Log in to https://xingyun3d.com, create a driving application in the "Application Center," choose a character, voice, and performance style, and you will get an App ID / App Secret;
Runtime Environment (when running my open-source project): Node ≥ 18.17 is sufficient, zero dependencies.

3. Minimum Runnable Code

The entire integration only takes three steps: include the script, create an instance, and speak.

1. Include the SDK

<div style="width: 540px; height: 960px">
  <div id="sdk"></div>
</div>
<script src="https://media.xingyun3d.com/xingyun3d/general/litesdk/[email protected]"></script>

2. Create an Instance

const sdk = new XmovAvatar({
  containerId: '#sdk',           // Digital human rendering container
  appId: 'Your App ID',           // Obtained after creating a driving application in the Application Center
  appSecret: 'Your App Secret',
  gatewayServer: 'https://nebula-agent.xingyun3d.com/user/v1/ttsa/session',
  hardwareAcceleration: 'prefer-hardware', // Enable hardware acceleration
  onMessage(message) { /* Handle errors/messages */ },
  onVoiceStateChange(status) { /* status: start / end */ },
});

// Initialize and listen for resource download progress
sdk.init({
  initModel: 'normal',
  onDownloadProgress: (progress) => console.log(progress + '%'), // Required
});

3. Make It Speak

sdk.speak('Welcome to Mofa Xingyun', true, true);

Just like that, a talking 3D digital human stands in your webpage.

3. I Wrapped It into a "Performance Console"

To make the demo more presentable (and more suitable for product promotion), I wrote an open-source project:

📦 https://github.com/likebeans/xingyun3D

It wraps the SDK's capabilities into a zero-dependency, single-page demo console:

Impromptu Speaking: It says whatever you type in the input box, ⌘/Ctrl + Enter to send
Opening Show / Dancing / Greeting: SSML + KA action commands, performing emotions, actions, and lines together
Streaming Broadcast: Simulates a large model outputting segment by segment, with subtitles and lip movements synced in real time
Auto Demo: One-click runs the full program of "Greeting → Self-introduction → Streaming Broadcast → Dancing → Curtain Call"
Status Remote Control: Standby / Standby Interactive / Online / Offline / Invisible, plus volume adjustment
Real-time Event Stream: Voice, network latency, status, and credit consumption are all visualized
Light / Dark Dual Themes: Default light mode, one-click switch if you prefer dark mode

When speaking, subtitles and stage lighting effects are linked in real time, and the effect looks like this:

Insert image description here

To run it locally, you only need:

# 1. Fill in environment variables
cp .env.example .env   # Edit .env and fill in XMOV_APP_ID / XMOV_APP_SECRET
# 2. Start (zero dependencies, Node ≥ 18.17)
npm start
# 3. Open http://localhost:3000

Or directly with Docker:

docker run -d -p 3000:3000 \
  -e XMOV_APP_ID=YourAppID \
  -e XMOV_APP_SECRET=YourAppSecret \
  ghcr.io/likebeans/xingyun3d:latest

4. Why I Think This Is Worth Paying Attention To

(One side note: if you prefer a dark interface, the project has a one-click switch, the effect is as follows—)

External image transfer failed, the source site may have an anti-leech mechanism, it is recommended to save the image and upload it directly

You might not have heard of the company Mofa (xmov), but what's behind it is not simple: the official positioning is "a globally leading 3D embodied interactive intelligent agent AI technology company," with the core being its self-developed LAM text-to-3D multimodal large model. The "Mofa Xingyun" it built is an embodied intelligence infrastructure for all terminals.

A few keywords worth noting:

Parameter Stream + AI End-Rendering: It doesn't send down a video; it sends down "parameters" (speech, expression, action, lip-sync data), and the client side renders, so it can be lightweight and fast;
End-to-end 500ms ultra-low latency: The interaction is real-time enough that connecting a large model won't feel "laggy";
Ten-million-level concurrency: This is prepared for large-scale scenarios, not a toy;
Lightweight deployment on hundred-yuan chips: Doesn't require high-end GPUs;
One SDK adapts to all terminals: Screens, humanoid robots, AR/VR glasses—all use the same speak(), from webpages to robots.

I will break down and verify these one by one in subsequent articles, not just reciting the official website copy.

5. Benefits

🎁 Invitation Code XDZARL7NEP: Register for Mofa Xingyun and get 1000 credits
🔗 Online Experience: https://likebeans.github.io/xingyun3D/
📦 Open Source Repository: https://github.com/likebeans/xingyun3D
📖 Official Documentation: https://xingyun3d.com/developers/52-183

In the next article, I will deeply dissect the technical details of the SDK: how speech, lip movements, expressions, and actions are linked together within 500ms.