Rapidly Iterating the Spine Skeletal Animation Library with Trae Work: Adding ASTC Format Support

① Pain Point Origins

• Premise: The current client project implements some iOS/Android pages using the Flutter cross-platform framework.
• Requirement: The client needs to implement 3D room decoration capabilities, user character customization (similar to QQ Show), and support for character actions.
• Solution Selection: Based on the requirements, introducing a game engine to implement the 3D space seemed appropriate. The tech team ultimately planned to introduce Cocos Creator as the technical framework for the 3D room. For character customization, since Cocos Creator has built-in support for Spine skeletal animation, and Spine officially provides versions for iOS/Android and Flutter, Spine was chosen as the technical solution for character dressing (Live2D is a similar skeletal animation but lacks a Flutter version).

The above introduction is mainly to help everyone understand the background of the entire requirement, which is also the origin of the pain points.

1.1 - Technical Optimization

Early development of the overall requirement went smoothly, but later, as the 3D space model resources in the Cocos game engine increased and the built-in Spine loaded more materials, memory surged, causing lag and even crashes on low-end devices. This signaled that the project had reached a stage requiring technical optimization.

Targeted technical investigation and deep optimization were carried out within the Cocos environment. The optimization process involved resource compression, asynchronous incremental resource loading, and Spine material reuse to minimize memory growth. The final results showed significant improvement but were not yet optimal.

Since Spine material resources in the Cocos environment, the native environment, and the Flutter platform use the same resources—image resources in PNG format—and the Cocos environment supports texture compression, this can significantly reduce memory usage, lower bandwidth requirements, and improve the game's rendering performance and loading speed.

Similarly, Spine resources can also convert texture pixel data into a GPU-specific compressed format within Cocos. Our tech team lead first practiced converting all existing Spine resources to ASTC format and successfully loaded them in the Cocos environment, achieving a breakthrough improvement in memory performance.

1.2 - ASTC Compression Format

Cocos's official documentation introduces compressed textures: (Compressed Textures | Cocos Creator)

ASTC is one such texture compression format. These compressed formats can be used directly in GPU memory, eliminating the intermediate processing from upper-level image formats to compressed formats, improving efficiency while reducing image resource decoding time and game memory.

1.3 - Format Unification

The original technical solution shared a single set of global Spine resources: iOS/Android, Flutter, Cocos. The native side downloaded and managed the same source files. The reason is obvious: convenient cache management, avoiding redundant downloads, and saving bandwidth resources. Now that the Cocos environment has implemented loading in ASTC format, can other platforms' Spine capabilities also be compatible with loading this format?

② Implementing Android Spine Support for ASTC Loading

2.1 - Solving the Android Pain Point

The team tech lead had previously upgraded and extended the iOS Spine library to support ASTC format loading, and the underlying iOS implementation was well-compatible with few changes. However, the Android Spine library's implementation differed significantly from iOS, making modification somewhat difficult.

2.2 - Analysis of Underlying Technical Principles

Having some prior understanding of Android Spine, its implementation depends on the following two important repositories.

    api "com.badlogicgames.gdx:gdx:1.14.0"
    api files('libs/spine-libgdx-4.2.12.jar')

libGDX is a Java-based cross-platform 2D/3D game development framework, relying on OpenGL (ES) graphics interface at its core, supporting six major platforms: Windows, Linux, macOS, Android, iOS, and Web browsers. It is one of the most mature and widely used game development frameworks in the Java ecosystem.

spine-libgdx is an implementation based on the libGDX framework, using OpenGL ES to render skeletal animations.

Examining the dependency library source code revealed OpenGL-related code. The initial guess was that Android Spine's underlying rendering should use GLSurfaceView or TextureView. However, deeper reading of the source code found that Android Spine actually uses Canvas rendering.

This raised a question: the dependency library's underlying rendering uses OpenGL, but the upper layer does not fully adopt this approach. Is there a reason for this, and does it affect the future possibility of supporting the ASTC format (ASTC inevitably requires OpenGL texture rendering)?

When in doubt, ask AI first: Open Trae Work, switch to Code mode, create a new task, select the folder, and finally input your question.

First Question

https://github.com/libgdx/libgdx Help me analyze the underlying technical implementation principles of this project's Cross-platform Game Development Framework, how it achieves cross-platform capability, and output the technical architecture as an HTML document. Focus on how to integrate and use it in an Android project, and explain how the underlying rendering and display works in an Android project.

Trae Output:

Trae directly output a detailed introductory document for libgdx, reconfirming that the rendering library's underlying implementation on the Android platform is based on OpenGL.

Second Question

https://github.com/EsotericSoftware/spine-runtimes/tree/4.3/spine-android spine-android uses com.esotericsoftware.spine:spine-libgdx and com.badlogicgames.gdx at its core. However, the onDraw method in SpineView.java uses canvas for drawing content. This differs from the underlying use of libgdx, which is an OpenGL implementation. Help me generate another HTML document explaining how the native Android SpineView renders Spine content, and whether OpenGL is used at the bottom layer.

Trae Output:

As mentioned earlier, why doesn't Android Spine use OpenGL implementation, given that the libgdx rendering library fully supports it?

From the document explanation, we know:

• Reuses spine-libgdx core logic: Skeletal animation updates, slot transformations, attachment calculations, etc., all come from spine-libgdx.
• Reuses gdx data structures: Array, FloatArray, IntArray, Pool, etc., come from com.badlogicgames.gdx.
• Replaces the rendering backend: Does not use libGDX's OpenGL pipeline but renders via the Android Canvas API.

Key point: spine-android only depends on the logic layer of spine-libgdx, not its rendering layer (GL20 / Batch / ShaderProgram). Rendering is done by Android's native Canvas.

From the document's solution comparison, Android Spine uses the Canvas.drawVertices() method for drawing. Although the code level does not call OpenGL, the Android rendering system's underlying layer might involve GPU acceleration.

The rendering summary for SpineView is as follows:

SpineView.onDraw(Canvas)
    └── SkeletonRenderer.renderToCanvas(Canvas, RenderCommand[])
            └── Canvas.drawVertices(TRIANGLES, ...)
                    ├── [Hardware Acceleration] → OpenGL ES → GPU → SurfaceFlinger
                    └── [Software Rendering] → Skia CPU Rasterization → Bitmap → Copy to Surface

But the document concludes: Compared to the libgdx OpenGL solution, although performance is not as good as the GPU direct path, the integration cost is low, making it more suitable for UI embedding and simple animation playback scenarios. This indirectly suggests that adopting an OpenGL solution might be feasible.

2.3 - Technical Modification Solution Selection

Third Question

If spine-android were to adopt the libgdx OpenGL rendering solution, would the amount of modification be large?

Trae Output:

Analysis of the modification effort for changing spine-android to the libgdx OpenGL rendering solution. Modification Assessment: Medium to large, core logic can be reused.

Looking at the target modification points, the overall amount of change is significant: from the Renderer drawing layer, texture object AndroidTexture, Atlas, etc., core parts of the code need to be refactored.

However, the final summary recommends using spine-libgdx for modification.

spine-libgdx already provides a complete OpenGL ES rendering implementation:

• com.esotericsoftware.spine.SkeletonRenderer (OpenGL version)
• com.esotericsoftware.spine.SkeletonDrawable (libgdx version)
• Integrates libGDX's SpriteBatch, Texture, TextureAtlas

Fourth Question

Are there any open-source projects on the market for spine-android that use OpenGL ES rendering?

Trae Output:

Based on public searches, no mature open-source project specifically providing a "pure Android OpenGL ES rendering alternative for spine-android" has been found.

At the same time, the final summary still recommends directly using spine-libgdx (recommended, minimal changes). Finally, it concludes that adopting an OpenGL solution for Android Spine is feasible.

Fifth Question

spine_android_spineview_rendering.html Based on the technical solution description, if the Android version of SpineView were to be modified to directly load ASTC, how should the code be rewritten? Output as an HTML document.

Trae Output:

The original spine-android's SpineView renders 2D via Canvas.drawVertices(), with the texture loading chain being:

• High video memory usage: Bitmap decompresses to RGBA8888, 4 bytes/pixel, a 2048×2048 texture occupies 16 MB.
• Does not support compressed textures: BitmapFactory only supports PNG/JPEG/WebP, cannot directly decode GPU compressed formats like ASTC/ETC2.
• Sampling quality limited by the software layer: API < 29 forces LAYER_TYPE_SOFTWARE, unable to enjoy GPU mipmap and anisotropic filtering.
• Multi-page texture switching overhead: BitmapShader requires frequent Paint switching in multi-page atlases, making draw calls difficult to merge.

Modification plan target effects:

• Directly load ASTC compressed textures, eliminating PNG decoding and RGBA intermediate representation.
• Texture data resides entirely in GPU memory in compressed form, reducing video memory usage by 4~8 times.
• Retain spine-libgdx logic layer (Skeleton / AnimationState) and AndroidSkeletonDrawable interface unchanged, only replace the rendering backend.
• External API remains consistent with the original SpineView.Builder, minimizing migration costs.

ASTC is a GPU compressed texture; Bitmap and BitmapShader cannot consume it. This means the modification must switch the rendering backend from Canvas to OpenGL ES. This is a fundamental change to the solution, not a partial optimization.

Then, the output modification plan introduces that the core logic is basically unchanged. The difference lies in the view hosting method: View changes to GLSurfaceView; Bitmap becomes textureId; SkeletonRenderer (Canvas) is replaced by GLSkeletonRenderer; blending modes change, etc.

In the technical solution modification plan, the biggest anticipated pain point lies in code translation, replacing the entire original Canvas drawing process with a GL environment: ASTC format file loading; GL environment setup; drawing process rendering; blending mode mapping. Among these, the GL environment is particularly cumbersome and complex. Anyone who has done audio/video development knows that OpenGL shader compilation, frame data processing, vertex data processing, texture creation, off-screen rendering, etc., are all lengthy and tedious formulaic code.

The technical modification plan was mature, and it was time to implement the code for functional verification! The document was fed to the AI to analyze and formulate a development plan for implementation.

2.4 - Technical Detail Optimization and Fixes

Everything is hard at the beginning. AI writes code quite fast, but it can also stumble when implementing features of a certain difficulty. As shown in the image below, the following conversation records are all about communicating with AI on how to fix bugs and issues.

Optimizing the First Problem

Using other AIs consistently failed to solve the black screen issue where no content was displayed.

I have already used AI to implement an ASTC-loading SpineView, android-spine/spine-android/src/main/java/com/esotericsoftware/spine/android/gl/AstcSpineView.java. But currently, it cannot display actual content. Are there any unresolved bugs or missing features causing the loading failure? Essentially, it's an AstcSpineView implemented by referencing SpineView, but the underlying rendering framework has been switched to OpenGL. Help me investigate the specific reason.

Trae Output:

Bug 1 (Fatal): GLSkeletonRenderer.createProjectionMatrix matrix transformation error

The original code directly added pixel-unit translateX/translateY to ortho/ortho. These two elements are the translation components of the NDC clip space, and their values must be within [-1, 1]. The passed translation amounts are usually hundreds of pixels, causing all vertices to be mapped outside the NDC range and clipped entirely by the GPU, resulting in a completely black screen.

Fix: Change to directly constructing the correct MVP matrix, first transforming Spine world coordinates to screen pixel coordinates via scale/translate, then mapping to NDC using orthographic projection.

Bug 2: Incorrect translateY sign in AstcSpineView.updateCanvasTransform

Compared with the original SpineView's Canvas transformation chain, the Y-axis translation should be + boundsCenterY * scale instead of - boundsCenterY * scale. This causes the skeleton to shift out of the visible area in the Y-axis direction.

Now recompile and run, AstcSpineView should be able to render Spine animations with ASTC textures normally. If problems persist, it is recommended to enable GLUtils.checkGLError to check for OpenGL errors, or check the debug logs of GLSkeletonRenderer and AstcSpineView in logcat to confirm whether vertex data reaches the GPU.

Optimizing the Second Problem

Is there room for optimization inside AstcSpineView? I see a GC triggered every 15 seconds.

Trae Output:

There is significant room for optimization. After reviewing AstcSpineView.java, AstcSkeletonRenderer.java, and GLSkeletonRenderer.java, new objects are being allocated almost every frame. The 15-second GC trigger is because the allocation rate exceeds the young gen's collection pace.

• High-frequency allocation (every frame, every slot)
• Rendering layer per-frame allocation
• Cache commands Array
• MVP matrix reuse
• Cache getWidth/getHeight in onSurfaceChanged
• And other detailed optimizations

Memory usage before optimization:

Memory usage after optimization:

Optimizing the Third Problem

android-spine/spine-android/docs/GLSurfaceView_vs_TextureView_Solution_Comparison.md According to the document, add a new AstcTextureSpineView (based on TextureView) to implement new spine loading capabilities, reusing existing reusable capabilities as much as possible.

Trae Output:

Only the View container layer (EGL management + rendering thread + SurfaceTexture handling) is new; the rendering core is reused with zero modifications.

PS: AstcSpineView has already been implemented, so why add AstcTextureSpineView? Because the Flutter environment also plans to adopt the ASTC loading mode and use Flutter Platform View for compatibility handling. However, it was discovered that using GLSurfaceView has serious issues with Flutter adaptation (the independent Surface conflicts with Flutter's (Skia/Impeller) rendering pipeline), causing the displayed content coordinates to be misaligned and unable to fit the display area.

Subsequently, AI was used again to re-implement AstcTextureSpineView using TextureView for rendering. Although the initial research didn't consider this and encountered a pitfall, with AI's help, the rewriting efficiency was extremely high, almost replicating the functionality in a single conversation.

Display effect of bridging native SurfaceView in Flutter environment:

Flutter's Texture widget is wrapped in a parent layout with a red background, but the rendered content still exceeds the constraint display.

Display effect of bridging native TextureView in Flutter environment:

③ Results Display

Spine source code display on the native side:

Final ASTC format effect display on the Android native side:

④ Experience Summary

After all this rambling, thank you very much for sticking with it to the end!!!

The initial reason was that SpineView had video memory and memory issues. The original logic used Bitmap, which occupied a large amount of memory. Using ASTC compressed textures not only reduces file size but also optimizes memory consumption. At the same time, the drawing was upgraded from Canvas to an independent GL thread rendering. Later, because GLSurfaceView had issues in the Flutter environment (unable to overlay native UI, does not support View transformations, etc.), it was switched to TextureView to improve overall compatibility.

In summary: From "software Canvas drawing PNG" to "GPU direct drawing ASTC compressed textures" to solve performance and video memory issues, and then from "GLSurfaceView" to "TextureView + self-managed EGL" to solve transparent overlay, View transformations, and lifecycle persistence, ultimately allowing Spine animations on Android to both save resources and be used freely like ordinary Views.

The R&D process hasn't fundamentally changed from the past:

1. First, research and understand the requirement background, formulate methods, and weigh pros and cons and pain points.
2. Design technical solution options, choose the most suitable and effective solution.
3. Implement the required functionality, meeting basic conditions and achieving the expected effect.
4. Optimize and iterate to complete the final goal.

Using Trae Work for development is the same. Before letting AI assist in development, the developer needs to understand the development requirements first. Only with a clear goal can AI achieve results more accurately.

Trae Work Usage Experience

The above are all personal real-world experiences successfully using Trae Work in a work environment. Nowadays, having AI and applying it to actual work scenarios truly allows one to feel the power of AI, and it indeed helps developers solve some tricky and tedious work content, improving work efficiency.

Additionally, the architecture diagrams and technical frameworks in the full text were generated using Trae Work, which greatly improved writing efficiency (in the past, I would silently draw diagrams myself to complete them).

Other Thoughts 🤔

• Perhaps AI will replace humans in the future, but for now, it's still too early to say. Humans created AI, and there is certainly a future that humans can continue to create.
• AI can lose intelligence and make mistakes, just like humans. But humans serving as the last line of defense is still very important.
• Having AI does not mean one should stop thinking. One must have clear ideas in mind; AI is a tool, not a clone that completely replaces me.
• The above only represents personal thoughts and ramblings. In the AI era, let's encourage each other 🙏🏻.