FFmpeg for HarmonyOS Ships with Native Hardware Decode and Dual ABI Packages
A FFmpeg Development Base with Native Hardware Decode Support for HarmonyOS
Author: Sirius_Ji
Project Intro: ffmpeg-for-HarmonyOS is an audio/video base library built for pure HarmonyOS (HarmonyOS NEXT / OpenHarmony). It not only opens up FFmpeg's full audio/video processing capabilities, but also natively connects to HarmonyOS's hardware decode/encode interfaces, provided in both HAR and C++ SDK forms.
Project: ffmpeg-for-HarmonyOS Current Version: v0.2.2 Positioning: An audio/video base library for HarmonyOS NEXT / OpenHarmony, not a finished player app.
1. Development Background and Motivation
When doing HarmonyOS audio/video development last year, the most direct problem I hit was: I couldn't find a complete FFmpeg library that would run directly on pure HarmonyOS.
The solutions you can find online broadly fall into a few categories:
- Taking Linux/Android
.sofiles and changing the extension, with ABI and system libraries not matching; - Compiling only one or two FFmpeg components, missing demux, resampling, filters;
- Only example code, with no reproducible dual-ABI build and no release packages;
- Can run software decoding, but has not integrated HarmonyOS's native hardware
OH_AudioCodec/OH_VideoDecoder, easily causing heating under high load.
The first two categories are "won't compile"; the latter two are "compiles but isn't usable." For someone actually building audio/video features, what's missing is not FFmpeg source code, but a base that can interface with HarmonyOS system capabilities and be integrated into engineering workflows.
This is also why the project includes hardware decoding.
First, pure software decoding has a direct cost on mobile devices. FFmpeg's software decoding continuously occupies the CPU, especially with high-resolution, high-bitrate video, and then device heating, battery drain, and frequency throttling follow. For players, transcoders, camera processing, and similar scenarios, relying only on software decoding makes it hard to maintain experience over time.
Second, many similar open-source products on the market still focus on "getting FFmpeg to compile on HarmonyOS," but have incomplete support for HarmonyOS's native hardware decoding capabilities. System interfaces like OH_AudioCodec and OH_VideoDecoder don't naturally connect with FFmpeg's AVCodecContext; they require additional decoder wrapping, Surface binding, timestamp and EOS handling. Without that work, hardware decoding is just a noun in documentation.
So this project from the start was not "recompile FFmpeg again," but putting three things together:
- FFmpeg's full demux, resample, filter, and mux capabilities;
- HarmonyOS native hardware decode/encode interfaces;
- Distribution forms directly consumable by ArkTS and C++ projects.
The project stays LGPL-2.1-or-later, with no GPL, nonfree, x264, or x265 enabled. Currently provides arm64-v8a and x86_64 dual ABI, with dynamic libraries aligned to 16KB pages.
2. Core Capabilities and Boundaries
Available Capabilities
- Audio: Decode, transcode, and resample of common formats like AAC / MP3 / FLAC / Opus / WAV; supports HarmonyOS native audio hardware decode, falling back to software decode when hardware decode is unavailable.
- Video: Demux, decode, and Surface rendering of H.264 / H.265; basic H.264 hardware encoding; transcoding and extraction for common containers like MP4 / MKV / FLV.
- Watermark: Text and image watermarks can be overlaid simultaneously, with position, font size, color, opacity, bold, and stroke support; images support PNG / JPEG decoding.
- Filters:
scale,overlay,drawtext,formatchains based onlibavfilter, supportingoriginal/720p/1080poutput. - Engineering Form: HAR package and ZIP SDK dual forms; 8 dynamic libraries; 23 ArkTS methods.
Boundaries That Must Be Stated Clearly
Current v0.2.2 is just an initial public beta, not the final completed version.
- Video processing currently covers only basic capabilities, far from full native FFmpeg coverage. Complex filter combinations, more encoders, more containers, and more bitstream features are still in the follow-up plan.
- Image watermarking is not yet complete. PNG/JPEG decoding, scaling, and overlay already work, but details like alpha channels, edge blending, and abnormal image input still need polishing.
- Hardware decode capabilities depend on the specific device and system version. Different chips and different HarmonyOS versions support different codecs, resolutions, and pixel formats. The project does capability detection and software fallback, but cannot guarantee consistent behavior across all devices.
- The API may still change; v0.x does not promise binary compatibility. Before using in production, run complete validation with your own media, target devices, and target containers.
- It does not cover all FFmpeg components, nor does it promise to replace the system media framework. It is better suited as "a base component when you need FFmpeg capabilities" rather than a universal player.
3. Dual-Form Distribution Design
Different developers integrate FFmpeg in different ways, so v0.2.2 provides two forms simultaneously.
HAR Package: For ArkTS Frontend Developers
ffmpeg_ohos-v0.2.2.har, about 18MB, includes ArkTS interfaces and dual-ABI native libraries.
Suitable for:
- Already have a HarmonyOS project and just want to call transcoding, playback, watermark from pages;
- Don't want to maintain CMake or manually copy
.sofiles; - Want to manage dependencies with
ohpm.
The HAR does not include libc++_shared.so; the host app needs to provide the C++ runtime with its own NDK version to avoid conflicts with the host project's C++ runtime.
ZIP SDK: For C++ Lower-Level Developers
ffmpeg-ohos-sdk-v0.2.2.zip, about 18.5MB, includes:
include/ FFmpeg + freetype + harfbuzz + lame header files
lib/arm64-v8a/ 7 FFmpeg/lame libraries + libffz.so.1
lib/x86_64/
CMakeLists.txt Example that can be added via add_subdirectory directly
BUILD-INFO.txt ABI, SONAME, sha256 manifest
NOTICE.md
LICENSES/
Suitable for:
- Standalone transcoders, offline processing tools, C++ services;
- Need direct access to
AVFormatContextandAVCodecContextfor lower-level control; - Want to integrate FFmpeg as an ordinary C/C++ dependency into an existing build system.
Among these, libffz.so.1 is renamed zlib. The reason is that there is no stable libz.so available in the HarmonyOS system environment, while the host project may already bundle zlib, and naming it libz directly would cause conflicts. After renaming, the NEEDED entries of libavcodec.so.63 and libavformat.so.63 both point to libffz.so.1.
4. Minimal Integration Guide
4.1 ArkTS: Calling Transcoder via HAR
First put the HAR into the project as a local dependency:
// entry/oh-package.json5
{
"dependencies": {
"@sirius/ffmpeg_ohos": "file:../ffmpeg_ohos"
}
}
Then call it:
import { FFmpegOhos } from '@sirius/ffmpeg_ohos';
const src = `${ctx.filesDir}/input.mp4`;
const dst = `${ctx.filesDir}/output.mp4`;
const r = FFmpegOhos.videoTranscode(
src,
dst,
'mp4', // container format
'720p', // original | 720p | 1080p
1.0 // volume 0.0 - 2.0
);
if (!r.ok) {
console.error('transcode start failed:', r.data);
}
When adding text + image watermark, format uses this parameter string:
watermark|<text>|<position>|<fontSize>|<color>|<fontPath>|<alpha>|<bold>
Example:
const fontPath = `${ctx.filesDir}/SourceHanSansCN-Regular.otf`;
const logoPath = `${ctx.filesDir}/logo.png`;
const format =
`watermark|FFmpeg-ohos|bottom-right|42|[email protected]|${fontPath}|80|1`;
FFmpegOhos.videoTranscode(
src,
dst,
format,
'1080p',
1.0,
logoPath // 6th parameter: image watermark
);
position supports top-left / top-right / bottom-left / bottom-right / center; alpha is 0–100; bold is 0 or 1. Fonts and images need to be placed in an app-readable sandbox path first.
Transcoding is an asynchronous task; progress can be queried via the status interface:
const status = JSON.parse(FFmpegOhos.videoTranscodeStatus().data);
// status.state: 1 running, 2 completed, 3 failed
4.2 C++: Linking via ZIP SDK
After unzipping the SDK, directly reference it in a CMake project:
cmake_minimum_required(VERSION 3.16)
project(my_media_tool CXX)
set(OHOS_ARCH arm64-v8a)
add_subdirectory(ffmpeg-ohos-sdk)
add_executable(my_tool main.cpp)
target_link_libraries(my_tool PRIVATE ffmpeg_ohos_sdk)
target_link_libraries(my_tool PRIVATE c++_shared)
ffmpeg_ohos_sdk automatically includes the include paths and 8 imported libraries. Business code can directly use FFmpeg native headers:
extern "C" {
#include <libavformat/avformat.h>
#include <libavcodec/avcodec.h>
}
AVFormatContext *fmt = nullptr;
if (avformat_open_input(&fmt, path, nullptr, nullptr) < 0) {
// error
}
If the host project has its own C++ runtime, ensure the version of libc++_shared.so matches the SDK's compiler toolchain; the SDK does not bundle an extra C++ runtime.
5. Pitfalls and Technical Depth
The harder parts of this project are mostly not "getting an API to work," but problems at system boundaries.
- Random screen tearing on HiSilicon chips: Early hand-written AVCC mux caused random screen tearing on certain frames; it was ultimately changed to a two-stage mux of AnnexB → MPEG-TS → FFmpeg native remux → MP4, avoiding the risk of maintaining AVCC myself.
- Mono upmix swallowing audio: When a mono source was force-upmixed to stereo, transient peaks dropped from 31517 to 22286, exactly -3dB. After switching to preserving the source channel count, FLAC/WAV/Opus peaks and attack recovered.
- Fake zlib stub: There was no directly usable
libz.soin the system; early linking went to an 8.9KB empty shell, and PNG decoding madeavcodec_open2returnAVERROR_EXTERNAL. Finally cross-compiled real zlib and renamed it tolibffz.so.1. - Full relink: Incremental builds once left a
libavformat.so.63that depended on the oldlibz, so nowbuild/<abi>is forcibly deleted before every FFmpeg build, and the release gate checks the DT_NEEDED closure of all.sofiles.
These efforts may not be reflected in the API count, but they determine whether the library can run directly in an unfamiliar project.
6. Open Source Call
Open-source infrastructure work is not easy. One person maintaining cross-compilation, dual ABI, hardware decode adaptation, filter chains, release packages, and documentation means progress won't be as fast as a commercial project, but each step tries to leave reproducible build and verification records.
Current v0.2.2 is just an initial version. Fellow developers are welcome to:
- Try the HAR or ZIP SDK on your own HarmonyOS devices;
- File an Issue when encountering problems, ideally with device model, system version, input media, and full logs;
- Suggest improvements on API design, container support, image watermarking, and hardware decode adaptation;
- If this project saves you a few days of compilation and troubleshooting time, a Star is helpful.
Repositories and Downloads
- GitHub:
https://github.com/Sirius-Ji/ffmpeg-for-HarmonyOS - GitCode:
https://gitcode.com/Sirius_Ji/ffmpeg-for-HarmonyOS - Release (HAR / SDK ZIP / per-ABI ZIP / SHA256SUMS):
https://github.com/Sirius-Ji/ffmpeg-for-HarmonyOS/releases/tag/v0.2.2
Top 1 from juejin.cn, machine-translated. The original thread is authoritative.
Repository address: https://github.com/Sirius-Ji/ffmpeg-for-HarmonyOS https://github.com/Sirius-Ji/ffmpeg-for-HarmonyOS The content on both ends is the same. Everyone is welcome to star and raise issues. [applause] A newbie really needs everyone's support.