跪拜 Guibai
← All articles
Frontend · Agent · Audio/Video Development

An AI Podcast Agent Gains Video Recognition and a Self-Cleaning Storage Layer

By 京东云开发者 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Turning a video or audio file into a podcast episode is a messy pipeline problem: recognition, script generation, voice design, and synthesis all produce intermediate artifacts that can silently fill a disk. The temp-to-permanent storage pattern with automatic cleanup solves that without requiring a separate media asset manager.

Summary

The third iteration of this AI audio creation agent adds content recognition as a first-class input path. Video and audio uploads are processed through Alibaba's Qwen ASR and multimodal models, which extract spoken content and visual context into a podcast script. A new temporary-permanent storage architecture keeps intermediate voice designs in a temp directory with a 10-minute TTL, while only user-approved timbres get promoted to permanent storage. A scheduled cleanup task recursively prunes expired temp files and empty directories, returning deletion statistics. The system prompt was restructured into six modular sections to reduce tool-calling mistakes, and a file-locking mechanism prevents index corruption during concurrent voice saves. Large videos over 21MB are automatically split into ≤10MB segments before upload, sidestepping API size limits.

Takeaways
Audio and video files are now direct inputs: Qwen ASR transcribes speech while a multimodal model extracts visual and contextual understanding.
All intermediate tool outputs land in storage/temp/ and are automatically deleted after 10 minutes by a scheduled cleanup task that also removes empty directories.
Only user-confirmed voice designs get promoted to storage/audios/ via a save_voice tool that copies files and updates a JSON index under an fcntl file lock.
Videos larger than 21MB are split into ≤10MB segments with moviepy before being sent to the multimodal API, avoiding single-file size limits.
The system prompt was reorganized into six modules (capability, workflow, tool rules, communication, context, execution) and now includes a table mapping tools to responsibilities.
Qwen ASR supports inverse text normalization, converting spoken forms like "二零二五年" to written "2025年" automatically.
The frontend required no changes; existing upload and playback components already handle the new backend capabilities.
Conclusions

Treating temp storage as a staging area with a hard TTL is a lightweight alternative to a full job-state machine for agent workflows where users make explicit keep/discard decisions.

Modularizing a system prompt into a responsibility table reduces LLM tool misinvocation more reliably than adding more natural-language guardrails, because it constrains the model's action space structurally.

Segmenting large media client-side before API calls is a practical workaround for model provider size limits, but it introduces a tradeoff: context that spans segment boundaries may be lost unless the segments are stitched back together logically.

Concepts & terms
ITN (Inverse Text Normalization)
A post-processing step in ASR that converts spoken number and date expressions into their standard written forms, e.g., "twenty twenty-five" becomes "2025".
Temporary-Permanent Dual-Layer Storage
A pattern where agent-generated artifacts first land in a temp directory with automatic expiration; only artifacts explicitly approved by the user are promoted to permanent storage.
fcntl.flock
A Unix system call used here to place an exclusive file lock on the voice index JSON, preventing concurrent writes from corrupting the file when multiple requests try to save a voice simultaneously.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗