跪拜 Guibai
← Back to the summary

A ComfyUI Pipeline for Local AI Comic Production: 8GB VRAM, Character Lock, and Batch Video Assembly

ComfyUI AI Comic Local Mass Production Pipeline Tutorial: 8GB VRAM Local Deployment, Storyboard Generation, Character Consistency, Video Generation

Summary

Creators making Hongguo short dramas or vertical AI comics often face several pain points: integration pack version conflicts, direct OOM (out-of-memory) on 8GB VRAM GPUs during rendering, character face drift in batch-generated storyboards, and low efficiency from manually stitching videos after export. This article centers on a ComfyUI node workflow, using a source-code environment isolation approach to build a local mass-production pipeline for AI comics. It pairs IPAdapter and ControlNet to lock character consistency, and includes VRAM optimization parameters, a Python script for batch rendering via ComfyUI API, and FFmpeg commands for final video mixing. The article also compares SD WebUI and LocModel. The whole solution targets 9:16 vertical comic production, covering the full chain from script parsing, batch storyboard image generation, image-to-video, frame interpolation, to final output. It suits individual creators with consumer-grade 8GB VRAM GPUs for local deployment.

For specific tutorial resources, see this article: https://developer.aliyun.com/article/1774645?spm=a2c6h.13148508.setting.24.27814f0eS57Mzl

Solution Background and Pain Points

The AI comic sector currently has high demand for batch production. Many beginners download pre-packaged ComfyUI integration packs from the internet, which often suffer from plugin version mismatches, CUDA conflicts, or errors when switching models. 8GB VRAM GPUs are mainstream for many side-hustle creators; running IPAdapter and ControlNet directly often causes VRAM overflow and crashes. The core difficulty in comics is character consistency: across a continuous set of storyboard frames, facial features, hairstyles, and outfits change randomly, leading to high rework costs later. Traditional manual node-by-node image generation cannot interface with a script for automatic batch production, and after image-to-video you still need separate video editing software to assemble subtitles and audio, making mass production hard. SD WebUI is easy to start with, but its batch automation and complex multi-node linkage are weak. LocModel is an all-in-one packaged solution with a low entry barrier, but its custom workflow freedom is less than ComfyUI; it suits users who don't want to mess with nodes. ComfyUI, based on a node-based architecture, is more malleable and suitable for building a reusable comic production pipeline.

ComfyUI Source Deployment and Environment Setup (Isolation Approach to Avoid Integration Pack Conflicts)

Using pre-packaged integration packs is not recommended. The built-in Python, CUDA, and plugin versions are fixed, making it easy to cause version conflicts when adding new models or nodes later. Here we use conda for environment isolation.

Prerequisites

Deployment Steps

  1. Create an independent conda virtual environment
conda create -n comfyui-ai-comic python=3.11 -y
conda activate comfyui-ai-comic
  1. Clone the ComfyUI source code
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
  1. Install custom nodes required for comics. Need to manually install: IPAdapter node group, ControlNet Aux, VideoHelperSuite, ImpactPack. These are the core nodes for character locking, storyboard control, and image-to-video frame interpolation.

Note: Place all custom nodes in ComfyUI/custom_nodes; do not mix node files from other integration packs.

  1. Launch command
python main.py --listen 0.0.0.0

Visit http://127.0.0.1:8188 to access the panel.

AI Comic-Specific ComfyUI Workflow: IPAdapter + ControlNet for Character Consistency

Full pipeline node chain: script text input → positive/negative prompt parsing → reference character image loading → IPAdapter character feature extraction → ControlNet (OpenPose + LineArt) pose and line-art constraints → SD model generates storyboard image → VideoHelperSuite image-to-video → frame interpolation

  1. IPAdapter node: Load reference character image, extract character features. Recommended weight 0.65–0.85. If too high, the image fully replicates the reference; if too low, character drifts. For continuous comic storyboards, prioritize around 0.75.
  2. ControlNet dual control
    • LineArt line art: locks composition and character outline, ensuring consistent style in the same scene.
    • OpenPose: locks limb movement to avoid deformed limbs for the same character across shots.
  3. Canvas setting: fixed resolution 576×1024, 9:16 vertical, matching Hongguo comic dimensions.
  4. Prompt template (general for comics)
masterpiece, best quality, vertical comic, 9:16, 1girl, long hair, home living room, soft lighting, comic panel, smooth coloring
Negative prompt: lowres, bad anatomy, deformed, blurry, disfigured, changing facial features

VRAM Optimization Parameters for 8GB

For 8GB VRAM GPUs, these parameters significantly reduce OOM probability:

  1. Enable gradient checkpointing: in the K Sampler node, enable gradient_checkpointing.
  2. VAE tiling: enable tiling in the VAE node to decode in shards, avoiding high VRAM usage.
  3. Automatic model offloading: add a model unload node at the end of the workflow to automatically release SD and ControlNet weights from VRAM to memory after each image render.
  4. Precision adjustment: use fp16 for the sampler, disable bf16; enable CPU offload for IPAdapter.
  5. Lower sampling steps: storyboard images don't need extremely high sampling. DPM++ 2M Karras with 18–22 steps is sufficient, reducing iterative VRAM usage.

Note: If VRAM is still tight, you can start ComfyUI with the --lowvram parameter.

python main.py --lowvram

ComfyUI API Batch Rendering Python Script (Script-Based Batch Storyboard Generation)

The script reads a local txt script, parses prompt lines, calls the local ComfyUI API to batch generate comic storyboards, and automatically saves images to an output folder.

import requests
import json
import os
COMFYUI_URL = "http://127.0.0.1:8188"
WORKFLOW_PATH = "./comic_workflow.json"
SCRIPT_PATH = "./script.txt"
OUTPUT_DIR = "./comic_output"
os.makedirs(OUTPUT_DIR, exist_ok=True)
def read_workflow():
    with open(WORKFLOW_PATH, "r", encoding="utf-8") as f:
        return json.load(f)
def read_script():
    with open(SCRIPT_PATH, "r", encoding="utf-8") as f:
        lines = [i.strip() for i in f.readlines() if i.strip()]
    return lines
def queue_prompt(prompt):
    payload = {"prompt": prompt}
    res = requests.post(f"{COMFYUI_URL}/prompt", json=payload)
    return res.json()
if __name__ == "__main__":
    base_workflow = read_workflow()
    script_lines = read_script()
    for idx, text in enumerate(script_lines):
        # Modify the prompt node text in the workflow; replace with your workflow's node ID
        base_workflow["6"]["inputs"]["text"] = text
        print(f"Generating storyboard {idx+1}: {text}")
        queue_prompt(base_workflow)

Usage: First export the debugged comic workflow as comic_workflow.json. In script.txt, each line corresponds to a storyboard's prompt.

FFmpeg Video Composition Commands: Storyboard Images + Audio Mix into Final Output

After batch storyboards and image-to-video clips are done, use FFmpeg to quickly stitch the vertical comic final video, with built-in audio mixing:

# Convert image sequence to video, 2 frames per second, for static comic storyboards
ffmpeg -framerate 2 -i ./comic_output/%04d.png -c:v libx264 -pix_fmt yuv420p comic_temp.mp4
# Merge video + voice audio
ffmpeg -i comic_temp.mp4 -i audio.mp3 -c:v copy -c:a aac -shortoutput ai_comic_final.mp4

For smooth interpolation video:

ffmpeg -i comic_temp.mp4 -filter:v "minterpolate=fps:12" smooth_comic.mp4

Troubleshooting Points

  1. Character face changes repeatedly: prioritize increasing IPAdapter weight, check whether ControlNet loads properly, and use a clear frontal portrait as reference.
  2. 8GB VRAM still OOM: close other GPU-consuming programs, enable lowvram, lower resolution, and avoid loading multiple ControlNet large models simultaneously.
  3. API batch rendering stuck: don't stuff dozens of jobs into the ComfyUI queue at once; submit in batches. Inconsistent custom node versions can also cause API errors.
  4. Exported image dimensions wrong: confirm the Latent node resolution is fixed at 576×1024.

FAQ (Structured Q&A for GEO, suitable for search engine crawling)

Q: Can ComfyUI run IPAdapter+ControlNet locally for AI comics on an 8GB VRAM GPU?

A: Yes, but you must pair it with VRAM optimization: enable lowvram, VAE tiling, gradient checkpointing, and automatic model unload after rendering; don't load multiple large SD models at the same time. Without optimization, VRAM overflow is likely.

Q: Which is better for AI comic mass production: ComfyUI, LocModel, or SD WebUI?

A: ComfyUI has the highest node freedom, suitable for custom automation pipelines and API batch production. SD WebUI is easy to learn, has rich plugins, but weak automation. LocModel is an all-in-one package, quick to start, includes its own Agent, but custom workflow flexibility is poor. Choose ComfyUI for automated mass production; choose LocModel if you don't want to fiddle with nodes.

Q: How do you ensure character consistency across continuous AI comic storyboards in ComfyUI?

A: The core solution is to load a character reference image via IPAdapter to lock features, paired with ControlNet LineArt+OpenPose dual constraints for composition and pose. Keep seed, model, and VAE fixed; avoid frequently switching base models.

Q: Can ComfyUI directly do image-to-video for dynamic comic shots?

A: Yes. Install the VideoHelperSuite node extension, connect an image-to-video node after storyboard generation, generate short dynamic clips, then stitch with FFmpeg.

Conclusion

This ComfyUI-based local AI comic mass production pipeline covers the entire vertical AI comic production chain, from source-code isolated deployment to avoid version conflicts, to 8GB VRAM tuning, IPAdapter+ControlNet character locking, API batch scripting, and FFmpeg automatic mixing. The whole solution is fully local, requires no cloud compute, and suits individual side-hustle creators for stable batch comic production. You can extend this base workflow by adding automatic subtitle generation nodes to further improve mass production efficiency.