Interview · GitHub · Algorithm

A Command-Line Pipeline That Turns Historical Stories into Vertical Videos

By 怕浪猫 · Oct 28, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

This pipeline demonstrates that AI-generated vertical video is now a fully scriptable, repeatable workflow, cutting production time from hours to minutes. It also exposes the regulatory reality of mandatory AI watermarks in China and shows a practical local workaround for self-use, while still requiring the declaration on public posts.

Summary

The pipeline replaces video editing GUIs with code. One storyboard.json holds the script, style suffix, and scene prompts; gen_assets.py calls MOSS's OpenAI-compatible image and TTS APIs to create visuals and voiceovers, then uses OpenCV inpaint to strip the mandatory "AI-generated" watermark. build_subs.py computes the timeline from measured audio durations and renders ASS subtitles, while assemble.py applies Ken Burns motion, mixes audio, and burns in the subtitles.

A single `speed` parameter raised the narration rate from 4.2 to 4.9 characters per second without changing the voice. The whole run takes about three minutes for asset generation and produces an 84.6-second 1080×1920 H.264 video. The system is reusable: swap the storyboard JSON and rerun the scripts to get a new video in about fifteen minutes, as demonstrated with two Edison-themed versions.

Takeaways

— The pipeline uses one storyboard JSON and three Python scripts: gen_assets.py, build_subs.py, and assemble.py.

— Each scene in the storyboard contains one narration line, two subtitle lines, an image prompt, and a shared style suffix appended to every prompt.

— Image generation runs at 896×1536 with 3 concurrent requests for 8 scenes, taking about 3 minutes for all assets.

— The "AI-generated" watermark cannot be disabled via API because it is mandated by Chinese regulations; it is removed locally with OpenCV inpaint (telea) after thresholding bright pixels.

— A TTS `speed` parameter of 1.15 increases speech rate from 4.2 to 4.9 characters per second without changing the voice.

— Scene duration is calculated as audio duration plus 0.7 seconds of breathing room, and subtitle timing is split proportionally by character count.

— Ken Burns effect is implemented with ffmpeg's zoompan filter; audio is placed with adelay and mixed with amix.

— QC checks include ffprobe for stream info, volumedetect for loudness (peak -0.4 dB, no clipping), and manual frame extraction from three points.

— The workflow is reusable: rewriting only the storyboard.json lets you regenerate a new video in about fifteen minutes.

Conclusions

The watermark removal sidesteps a legal requirement; the article itself insists on declaring AI generation when publishing, so the practical workaround is only for internal review, not for circumventing compliance.

The timing heuristic of splitting by character count works because Chinese speech is uniform; for tonal or variable-rate languages it would likely drift and need per-word timestamps.

The entire pipeline is a testament to moving from GUI to code: reproducibility and reuse, but it is still tied to the specific MOSS API and its rate limits.

Concepts & terms

Ken Burns effect

A motion technique that slowly zooms and pans across a still image to create a sense of movement, commonly used in documentary and video production.

Inpainting (OpenCV)

A method to fill in missing or unwanted parts of an image by interpolating from surrounding pixels; here used with the telea algorithm to erase a watermark cleanly.

ASS subtitles

A subtitle format that supports rich styling like fonts, colors, outlines, and fades, enabling the 'styled text' look common in short videos.

Storyboard JSON

A structured data file that defines the narrative, image prompts, and timing parameters for each scene, serving as the single source of truth for the pipeline.

From the discussion

Featured comments

szp2005

We got burned on the three-frame sampling step: the export showed success, but a segment of Chinese subtitles was entirely blank. The reason: Chinese text without spaces was measured with a too-short line height, so not a single line fit, and the sampled frames happened to dodge it. Later we changed to sampling one frame at the midpoint of each Dialogue, counting the pixels in the subtitle band, and erroring out when it's near zero, which caught it before delivery. (AI-assisted writing)

怕浪猫

Which model do you use?

See top comments, translated →

Source: juejin.cn ↗ Google Translate ↗ Backup ↗