Cocos Video Puzzle Now Runs on WeChat Mini Games with a Single Codebase

Great news!

Introduction

Hello everyone, I am Yiyuan Programmer, a lead programmer with 8 years of experience in the game industry.

I wonder if you still remember my last article, "Cocos Video Puzzle, the Last Piece of the Puzzle Game, Now Supports Native!", where we tackled video texture extraction for both Web and Android Native platforms. The puzzle gameplay was directly reusable, and the results were excellent.

However, shortly after the article was published, the comments and private messages were flooded with the same question:

Screenshot of user inquiries

Since that's the case, I had to step in again and fill in the last platform piece for the puzzle game in Cocos — WeChat Mini Games!

Getting down to business, this issue will take you through a practical battle: how to decode video, extract textures, and complete a puzzle game within a WeChat Mini Game, all while sharing a single set of business code with Web and Native platforms.

The complete practical source code for this article is available at the end. Those who already own the collection can update directly.

Previous Review: How Did Web and Native Do It?

A brief review: In the last issue, we successfully established two paths:

1. Web Environment

Using document.createElement('video') to create a video tag, drawing each frame to a Canvas via drawImage, and then uploadData to a Texture2D.

2. Android Native

Modifying the official Creator3.7.3_VideoTexture example, decoding mp4 via window.gfx.Video + FFmpeg, and uploading the texture in the frame_update callback.

The puzzle core (Board, Piece, Shader slicing) required no changes at all. The only variable was "how the video becomes a dynamic texture."

So, WeChat Mini Games have neither document nor our custom gfx.Video. What to do?

Why Are WeChat Mini Games Difficult?

I believe anyone who has worked on interactive video mini-games has experienced this: within the WeChat environment, options for getting video pixel data are limited.

Let me list the solutions evaluated at the time:

1. VideoPlayer Component

Same problem as Web / Native — cannot get video frame data, plus old issues like native layer occlusion.

The puzzle requires sampling the texture, so this is ruled out directly.

2. Video to Image Sequence Playback

Last time, I already tested this: package size balloons, numerous loading requests, and inexplicable acceleration on native.

Mini Games certainly can't handle it either, ruled out.

3. wx.createVideoDecoder

WeChat's provided low-level video decoder can getFrameData() to obtain RGBA frame data, then upload it to Cocos's Texture2D — chosen for this issue.

Solution	WeChat Feasibility	Conclusion
VideoPlayer Component	Cannot get texture, layer issues	Ruled Out
Video to Image Sequence	Package size balloons	Ruled Out
wx.createVideoDecoder	Can get frame data	Chosen for this issue

Actually, the idea is consistent with the last issue: Decode video → Upload to Texture2D → Puzzle Shader slices and displays. Only the decoding method has changed.

Core Practice: WeChat Mini Game Video Texture

1. Platform Branching: One Codebase, Three Platforms

First, perform platform detection in VideoTexturePlayer. WeChat Mini Games use the MINIGAME macro, which takes priority over native JSB:

One codebase, three platforms

The external API remains unchanged. GameManager just calls videoPlayer.load(), and Board gets the Texture2D to start slicing the puzzle.

2. Create Decoder, Upload Texture Frame by Frame

The core flow has four steps:

Step one, create the WeChat video decoder:

wx.createVideoDecoder

Step two, listen for the start event, get the video width and height, and create a Texture2D:

Listen for start event

Step three, start decoding:

decoder.start

Step four, in update(), fetch data frame by frame and upload the texture:

this.texture.uploadData

Pitfall Record: Looping Playback

After the first version ran playback successfully, I confidently added looping logic:

// ❌ Wrong approach
decoder.on('stop', () => {
    this.wechatDecoder.seek(0);
});

I never expected that when a video finishes playing naturally, it triggers ended, not stop!

Even worse: calling seek(0) alone cannot resume decoding, the screen freezes directly, and the video stops moving even after the puzzle is completed:

Insert image description here

The correct approach:

Listen for the ended event, not stop
Call decoder.stop(), then decoder.start({ source }) to restart decoding
Use wechatLooping to prevent re-entry, and wechatDestroying to prevent race conditions during destruction
If stop().then(start) fails, only then fallback to seek(0)

stop+start loop success

With this fix, the video loops smoothly after the puzzle is completed, and the artist won't be "frozen".

Other Details

1. Core Flow

The core of this article remains: WeChat-side video decoding → Extract texture → Display on puzzle pieces.

The puzzle gameplay directly reuses the previous issue's approach. Board uses Shader parameters to slice different regions of the same dynamic texture. Friends can refer to the old article or source code.

2. Component Integration

The complete component is already included in the collection's source code.

Last issue's native folder (Android native decoding) is kept as is;

For this issue's WeChat side, no additional native plugin is needed. Just ensure the VideoTexturePlayer script and types/wx.d.ts are in the project, and call load():

this.videoPlayer.load

3. One Codebase, Three Platforms

VideoTexturePlayer encapsulates three paths: Web / Native / WeChat. GameManager, Board, and the puzzle Shader require no changes at all.

Final Result

After the puzzle is completed, the video loops, and the artist comes to life!

3*3 Dog Puzzle:

Insert image description here

10*10 Artist Puzzle:

Insert image description here

Conclusion

The complete practical source code for this article has been integrated into Yiyuan Cocos Mini Game Practical Collection 2.0. Those who already own it can update directly.

I am "Yiyuan Programmer", a lead programmer with 8 years of experience in the game industry. In game development, I hope to help you, and through you, help everyone.

To be honest, I'd like a like and a heart! Please share this article with other friends you think might need it. Thank you!