Frontend · Code Standards · Frontend Framework

Camera2 to NV21: The Silent Face-Detection Failure in ArcSoft's Android SDK

By 进击的丸子 · Oct 26, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Android face-recognition integrations fail silently when Camera2 frames are converted to NV21 without aligning dimensions and orientation; the SDK's detector simply returns nothing. Matching the demo's conversion, size, and resource-release patterns is what separates a working integration from an empty result.

Summary

ArcSoft's APP Face Recognition SDK V5.0 on Android returns no detections for many developers even after they convert Camera2 frames to NV21. The cause: the conversion only repacks bytes, it does not rotate or mirror the image, and the algorithm's input dimensions must match the configured preview size. The integration path runs through two outputs from the same capture session: a TextureView for preview and an ImageReader for detection frames. The ImageReader outputs YUV_420_888, which is turned into NV21 with ArcSoftImageUtil.yuvToNv21, using each plane's row and pixel strides.

Beyond format, the guide spells out the call sequence: open the camera, create a capture session, receive onImageAvailable callbacks, convert, detect, draw, and close the image. The ImageReader's maxImages argument (5 in the demo) is a buffer count, not a frame rate. Images must be closed after synchronous processing or copied before async work. The guide also covers runtime permission checks, including re-checking after returning from settings and not treating shouldShowRequestPermissionRationale as proof of permanent denial. Lifecycle handling requires pausing and resuming the camera in onPause/onResume, and coordinating resource release to avoid double frees.

Takeaways

— Camera2 provides two outputs in one capture session: a TextureView for preview and an ImageReader for detection frames.

— ImageReader images arrive as YUV_420_888 and must be converted to NV21 using ArcSoftImageUtil.yuvToNv21 with each plane's row and pixel strides.

— NV21 conversion does not rotate or mirror the image; a normal preview does not mean the algorithm input is correctly oriented.

— The final integer in ImageReader.newInstance is the maximum number of simultaneously acquired images (maxImages), set to 5 in the demo; it is not a frame rate.

— FaceHelper must be initialized with the same width and height as the algorithm input; changing resolution requires rebuilding it.

— Image.close() must be called after the listener returns, wrapped in try/finally, or the data must be copied if processing becomes asynchronous.

— Runtime camera permission must be checked and re-confirmed before every camera open, including after returning from settings; shouldShowRequestPermissionRationale does not indicate permanent denial.

— onConfigured() in the capture session confirms the session is ready; onCamera2Opened() only confirms the device is open.

— Image and face information passed to liveness processing must come from the same frame to avoid mismatched data.

Conclusions

The most common reason no face is detected is that developers assume NV21 conversion handles orientation; the guide makes clear it doesn't, which explains many failed integrations.

The article's insistence on synchronous image release and copying data for async processing highlights a real memory-safety issue in Camera2's callback model.

The permission advice about re-checking after settings and not looping dialogs reflects Android's fragmented runtime permission behavior across OEMs.

Concepts & terms

NV21

A YUV color format where the Y plane is followed by interleaved U and V components. Unlike Camera2's default YUV_420_888, many face detection SDKs expect this byte layout, requiring a conversion step.

ImageReader

An Android class that captures frames from a camera session into a buffer for CPU processing. It supplies a Surface that can be added to a capture request, and fires onImageAvailableListener when a frame is ready.

YUV_420_888

A multi-plane YUV format used by Camera2. It provides separate planes for Y, U, and V, with row and pixel strides that must be read when converting to a packed format like NV21.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗