跪拜 Guibai
← Back to the summary

A 3D Virtual Human Lands on the Desktop, Built Entirely by Talking to an AI

Introduction

In the previous article, "Nebula SDK + Tampermonkey: Giving LLMs a Body to Accompany You on Every Webpage," I used a Tampermonkey script and the Mofa Nebula SDK to make a 3D virtual human appear on any webpage, giving AI a body to perceive the world, understand the environment, and express itself naturally.

This attempt made me increasingly convinced: The next phase of AI is not just about thinking better, but about interacting better. This is also the core concept of Mofa Nebula's "Embodied Interaction Intelligence": giving AI a body, allowing it to perceive the world, understand the environment, and communicate naturally with people through voice, expressions, actions, and real-time responses.

But a Tampermonkey script ultimately runs inside a browser. Once you leave the browser, the virtual human disappears. It is parasitic on a tab, not truly accompanying you on your desktop.

So this time, I used TRAE SOLO to build a true desktop application—letting the 3D virtual human step out of the browser and become an independent companion on your desktop at all times.

Effect Demonstration

Let's first look at the final result.

Standalone Desktop Application

The entire program is just a single MofaNebula3DVirtualHuman.exe file. It's green, portable, and runs with a double-click, requiring no environment configuration.

How to get it:

Baidu Netdisk download link for the program: https://pan.baidu.com/s/15RLrdH0OK92sR1M6Cv7Pzw?pwd=1234

Add me on WeChat shc1139874527, note "Nebula", and I'll send it to you privately.

Lightning-Fast Onboarding Experience

Double-click MofaNebula3DVirtualHuman.exe, enter the APP_ID and APP_SECRET from the Mofa Nebula SDK, and you can start experiencing it immediately, with zero barriers.

Getting the key takes only 4 steps:

  1. Go to the [Mofa Nebula Platform] to register an account, fill in the invitation code JM2A7C9LSW, and receive 1000 points.
  2. After successful registration, enter the "Console" and click "Embodied Drive" to create an application.
  3. Choose your preferred digital human avatar, scene, and voice, then click "Create and Enter Debugging."
  4. Click "Access SDK" in the upper right corner of the page to get the APP_ID and APP_SECRET, and fill them into the program.

💡 Using the invitation code to register gives you 1000 points; otherwise, you get 100 points. Points are similar to tokens for large models.

Smooth 3D Interaction

The 3D digital human renders in real-time, supports voice broadcasting and real-time interruption, with synchronized expressions and actions, and a response latency of less than 500ms.

Intelligent Dialogue Capability

It connects to any LLM, supports streaming responses, and the digital human listens and speaks simultaneously, providing a natural and smooth experience.

Core Technical Architecture

The entire project relies on three core technical components:

Technical Component Role Description
TRAE SOLO Development Tool A development mode of TRAE IDE that directly generates complete desktop application code through natural language conversation, going from idea to product in one step.
Mofa Nebula SDK 3D Rendering & Interaction Platform-level capability for the AI terminal era, allowing AI in all terminals to serve, accompany, and entertain people in an embodied form.
Electron Desktop Container Packages web applications into cross-platform desktop applications.

Mofa Nebula Embodied Drive SDK

The Mofa Nebula SDK is the core of the project, responsible for the rendering and interaction of the 3D digital human. It is not a simple digital human tool, nor an Agent wrapper tool, but an Embodied Interaction Intelligence Open Platform.

It fills the missing "interaction layer" when large models and Agents are deployed in real terminals:

Simply put, large models solve the AI's "brain," and Mofa Nebula completes the AI's "body, expression, and interaction."

TRAE SOLO: A Sharp Tool for Solo Developers

In traditional development processes, moving from requirements analysis to code implementation and then to testing and deployment often requires team collaboration. But for solo developers, TRAE SOLO offers a new development model—one person, one tool, completing the entire development process.

Core advantages of TRAE SOLO:

  1. Intelligent Code Generation: Based on the code generation capability of large models, just describe the requirements to quickly generate high-quality code.
  2. Real-time Debugging: Write and debug simultaneously, with code changes taking effect instantly, greatly improving development efficiency.
  3. Full-stack Support: Frontend, backend, desktop applications—one tool handles all scenarios.
  4. Knowledge Accumulation: Experience from the development process is automatically accumulated into a knowledge base, making subsequent development faster.

Electron: Cross-Platform Desktop Container

Electron allows us to build cross-platform desktop applications using HTML, CSS, and JavaScript. Its core advantages:

Practical Tutorial: Building a Desktop Virtual Human from Scratch

Step 1: Install the nebula-avatar-demo Skill

Before starting development, install a key skill for TRAE SOLO: nebula-avatar-demo.

Reference article for the nebula-avatar-demo skill: https://juejin.cn/post/7664905579035410441

Skills are extension capability packs for TRAE SOLO, equivalent to giving the AI "expert memory." After installing a skill, TRAE SOLO will automatically reference the best practices and code standards within that skill when generating code, resulting in higher quality and more consistent output.

The nebula-avatar-demo skill encapsulates the basic calling examples and code styles of the Mofa Nebula SDK. After installation, TRAE SOLO knows how to correctly call the Nebula SDK's API, avoiding incorrect interface calls.

Installation Method: Enter the following in the TRAE SOLO dialogue window:

Help me install the nebula-avatar-demo skill, skill address: https://gitee.com/sxshi/nebula-avatar-demo

TRAE SOLO will automatically search for and install the skill. Once installed, it can be used in subsequent development.

Step 2: Launch TRAE SOLO and Initialize the Project

Open TRAE SOLO and create a new project named xinyun. Next, let TRAE SOLO help us initialize the project.

If you have questions about the overall technical solution or are unclear about the requirements boundaries, you can first use the /openspec opsx:/explore command to explore implementation paths and organize the task list. We can also combine skills like superpowers to let TRAE SOLO reference more best practices when generating code, resulting in higher quality output.

Step 3: Describe Requirements in Natural Language

This is the most core step of the entire development process—no need to write code manually, just tell TRAE SOLO what you want to do in natural language.

First, switch the IDE to SOLO mode:

Then, present the requirements sequentially in the dialogue window:

Help me create an Electron + Vue 3 + TypeScript project, built with Vite, and install element-plus and less dependencies.

TRAE SOLO automatically completed the project structure setup, dependency installation, and configuration file generation.

Help me encapsulate the calling logic for the Nebula SDK, referencing the code style of the nebula-avatar-demo skill,
including: creating an SDK instance, connecting and disconnecting, 3D model loading progress callback,
and voice playback method, encapsulated as a Vue composable function useAvatar.

TRAE SOLO referenced the code style of the nebula-avatar-demo skill and automatically generated the complete SDK integration code:

// src/composables/useAvatar.ts
export function useAvatar(containerId: string = "#td-person") {
  const { sdkConfig } = useGlobalData();
  const sdk = shallowRef<any>(null);

  const connectInfo = ref({
    status: "disconnected",
    statusText: "Not Connected",
    progress: 0,
    isConnected: false,
  });

  async function connect() {
    connectInfo.value.statusText = "3D Model Loading...";
    await sdk.value.init({
      onDownloadProgress: (p: number) => {
        connectInfo.value.statusText = `3D Model Loading... ${p.toFixed(2)}%`;
      },
    });
    connectInfo.value.isConnected = true;
    speak("Hello host, I am your virtual assistant~");
  }

  function speak(text: string, isStart = true, isEnd = true) {
    sdk.value?.speak(text, isStart, isEnd);
  }

  return { connect, disconnect, connectInfo, speak };
}

③ Integrate DeepSeek Dialogue

Help me implement a composable function useDeepSeekChat for DeepSeek streaming dialogue,
supporting SSE streaming reception, message history management, and triggering an onSpeakChunk callback for each piece of text received to drive the virtual human to speak.

④ Build the Main Interface

Help me create the main interface App.vue, including: a frameless window (custom title bar, minimize/close buttons),
a left-side 3D virtual human rendering area, and a right-side function panel (connection panel/dialogue panel/settings panel switching).

⑤ Package as a Desktop Application

Help me configure electron-builder to support packaging as a Windows installer.

Step 4: Run and Debug, Fix Issues Through Direct Dialogue

After code generation is complete, run npm run dev to start development mode. The application ran, but a problem was quickly discovered—the window couldn't be dragged and moved.

This was because a frameless window was used, and the default drag area was not set. In TRAE SOLO, I just needed to say one sentence:

The window can't be dragged and moved, help me fix it.

TRAE SOLO immediately located the problem and automatically added the -webkit-app-region: drag style to the title bar area, while ensuring the button areas were not affected by dragging. After re-running, the window could be dragged normally.

Similarly, several other minor issues were encountered during development, all resolved through dialogue with TRAE SOLO:

Problem Encountered My Description TRAE SOLO's Fix
Window can't be dragged "The window can't be dragged and moved" Automatically added -webkit-app-region: drag
3D model loading stuck "Model keeps loading after SDK connection" Checked gatewayServer configuration, corrected the interface address
White screen after packaging "White screen when opening after packaging into exe" Modified loadFile path, corrected __dirname reference

Step 5: Package and Release

After all features are debugged and passed, execute packaging:

Bash

npm run dist    # Package as .exe installer

The generated installation package is just a single exe file, green and portable, runnable with a double-click.

Development Experience

Throughout the entire process, I did not manually write a single line of code. From project initialization to final packaging, all code was automatically generated by TRAE SOLO based on natural language requirements—Electron main process configuration, Nebula SDK integration, DeepSeek dialogue connection, UI interface construction, packaging configuration, and even problem fixes, all completed through dialogue.

The nebula-avatar-demo skill played a key role in this—it allowed TRAE SOLO to directly reference official best practices when generating SDK calling code, avoiding repeated trial and error.

The only three things I had to do were: clearly describe the requirements, say a sentence when encountering a problem, and provide the Nebula SDK key.

Developer's Perspective: Real Experience and Thoughts

Why Choose Mofa Nebula

Before starting this project, I actually tried other digital human solutions and stepped on quite a few pitfalls.

Traditional video streaming solutions rely on server-side rendering, where every frame of the picture is pushed from the cloud to the client—high latency, high bandwidth consumption, and fundamentally unable to achieve true real-time interaction; a user would have to wait several seconds to see a response after speaking.

Parameter streaming solutions are lighter, but require building a complete rendering pipeline yourself: skeletal animation, expression driving, lip-syncing... The development cost is extremely high, making it difficult for ordinary developers to get it running in a short time.

The emergence of Mofa Nebula truly solved this pain point—a single SDK for one-stop integration, using parameter streaming technology at the bottom layer for client-side rendering, with an end-to-end response latency of about 500ms, allowing ordinary developers to quickly build high-quality embodied interaction applications. I don't need to care about how the rendering engine runs or how models are loaded; I just need to call a few APIs, and the 3D virtual human can speak and make expressions on the screen.

Expandable Scenarios

Although this Demo is just a desktop application, Mofa Nebula's capabilities go far beyond this. Based on the same SDK and interaction logic, it can be easily expanded to more real-world scenarios:

Summary

Large models have taught AI to think, but if AI is to truly enter terminal scenarios, it also needs a "body," "expression," and "interaction." As an embodied interaction intelligence open platform, Mofa Nebula completes this last piece of the puzzle.

Through TRAE SOLO, I successfully upgraded the web-based 3D virtual human into a standalone desktop application. The entire development process was efficient and smooth, mainly due to the following points:

  1. The powerful capabilities of the Mofa Nebula SDK: It provides a mature 3D digital human rendering and interaction solution, allowing AI to move from a chat box to real terminals.
  2. The intelligent development capability of TRAE SOLO: It significantly reduced repetitive work, allowing me to focus on core logic.
  3. The cross-platform capability of Electron: It easily packaged the web application into a desktop application.

If you also want to experience the charm of embodied interaction intelligence, why not try the Mofa Nebula SDK—using the invitation code JM2A7C9LSW during registration gives you 1000 points, and a few lines of code can get your first digital human Demo running.

Comments

Top 1 from juejin.cn, machine-translated. The original thread is authoritative.

用户10336183740

how do I download it