跪拜 Guibai
← Back to the summary

WorkBuddy Taps Local ComfyUI for Credit-Free Text-to-Image Generation

In a previous article AI小白如何使用并创建自己的skill, I directly created a skill in WorkBuddy to draw illustrations in my own style. After using it for a few days, I found that although this skill can use free models for the text stage, it calls other drawing models for illustrations, and generating images consumes credits; each image costs at least 5 credits, and the daily free credits run out after a few images. I wondered if I could draw without credits. So I thought about connecting local ComfyUI to WorkBuddy. After some troubleshooting, I got it working. Now I can use my own skill to generate illustrations without limit.

Here is the process.

Step 1: Create a text-to-image workflow in local ComfyUI

Install ComfyUI on your computer and create a text-to-image workflow. This step is complicated and I won't demonstrate it here.

在这里插入图片描述

I used the Qwen 2512 text-to-image workflow; other text-to-image workflows also work. After the workflow runs without issues, export the workflow's API.

在这里插入图片描述

Here you must select "Export as API". A directly exported workflow won't work; you must export as an API, because the API workflow includes an external interface for direct calls to ComfyUI. A directly exported workflow has no external interface, so external programs cannot call it.

在这里插入图片描述

After exporting, you can install the workflow in WorkBuddy.

Step 2: Install the skill-creator plugin in WorkBuddy

In WorkBuddy, you can directly enter a command to create a skill, but your own description may not always be comprehensive, so the created skill may not be great. To create a professional skill yourself, you need to first install a skill that creates skills.

Search for 魔搭社区 (ModelScope) in your browser.

在这里插入图片描述

Enter the site and select "skill" at the top.

,

At the very top you'll see this creat-skill.

在这里插入图片描述

Click it to see the detailed introduction of this skill.

在这里插入图片描述

You can see this is a skill specifically for creating, modifying, and optimizing skills. It follows certain standards and steps to create and modify skills.

Now to install this skill into WorkBuddy, copy the skill's installation address on the right side of the page.

在这里插入图片描述

Click the copy button next to "Install via npx", then paste the command directly into WorkBuddy:

Install npx skills add https://www.modelscope.cn/skills/@anthropics/skill-creator skill

在这里插入图片描述

Wait for the installation to complete. After that, WorkBuddy will have a dedicated skill for managing skills.

Step 3: Automatically create a ComfyUI text-to-image skill in WorkBuddy

Next, turn the local ComfyUI text-to-image workflow into a skill.

Based on the ComfyUI text-to-image workflow defined in the file "trae本地部署文生图.json", develop a skill that can interact with a ComfyUI service (address: http://127.0.0.1:8188/). The skill should implement the following features:

  1. Call the specified workflow and dynamically configure three key parameters:
  • Positive prompt: fill it accurately into the "positive prompt" node in the workflow

  • Image dimensions: set the width and height parameters of the "image dimensions" node

  • Image output path: configure the text field parameter of the "save image to local" node

  1. Implement a strict image-size restriction mechanism that only allows the following resolution combinations:

1:1 : 1328x1328, 16:9 : 1664x928, 9:16 : 928x1664, 4:3 : 1472x1104, 3:4 : 1104x1472, 3:2 : 1584x1056, 2:3 : 1056x1584

Verify that the user-supplied dimensions comply with the above rules, and provide clear error messages for invalid input. If the user does not mention dimensions, automatically pick a resolution based on the user's needs.

  1. Establish a safe and reliable parameter-passing mechanism:
  • Ensure that the user's prompt, selected image dimensions, and image output path are accurately mapped and passed to the corresponding nodes in the ComfyUI workflow

  • Implement parameter validation logic to prevent invalid parameter values from failing the workflow

  1. Implement workflow parameter protection:
  • Strictly limit the adjustable range of workflow parameters; apart from the explicitly specified prompt, image dimensions, and image output path, all other parameters in the workflow are forbidden from modification or adjustment

  • Implement parameter locking to ensure unauthorized parameters are not accidentally or maliciously changed

  1. Develop a complete workflow execution flow:
  • Implement a workflow trigger mechanism that correctly starts ComfyUI workflow execution

  • Monitor the image generation process and provide execution-status feedback

  • Handle the result return after the workflow completes

  • After the workflow finishes and saves the generated result, automatically unload the models loaded by the workflow to free VRAM and memory usage.

  1. Implement comprehensive error-handling:
  • Network connection error: give a clear message when the ComfyUI service cannot be reached

  • Parameter validation error: provide specific error information and correction suggestions for invalid inputs

  • Workflow execution error: catch and handle exceptions during execution

  • Image generation failure: implement a retry mechanism or provide a detailed error log

After the skill is developed, run comprehensive tests to verify the correctness and stability of all features, ensuring accurate parameter passing, reliable workflow execution, and effective error handling. Note that skills should be in Chinese.

Drag the previously exported 千问-文生图-API.json workflow into the WorkBuddy dialog, and paste the above text into it as well.

在这里插入图片描述

Wait for WorkBuddy to automatically install the ComfyUI text-to-image skill. WorkBuddy will automatically call the skill-creator plugin to create the ComfyUI text-to-image skill.

Step 4: Use the ComfyUI text-to-image skill in WorkBuddy to generate images for free

Once the ComfyUI text-to-image skill is created, you can directly call the Qwen text-to-image workflow in ComfyUI from WorkBuddy. Because ComfyUI runs locally, image generation is free, and you no longer need credits in WorkBuddy.

During the skill installation, GPU memory usage suddenly increased a lot, indicating that WorkBuddy is calling the ComfyUI workflow.

在这里插入图片描述

When WorkBuddy is creating the ComfyUI text-to-image skill, make sure ComfyUI is running on your computer. Otherwise, WorkBuddy will fail to call ComfyUI and the skill will not be created.

在这里插入图片描述

In this command window of ComfyUI, you can see WorkBuddy calling the workflow to generate images.

在这里插入图片描述

Skill installation successful.

在这里插入图片描述

Now type "/" in the dialog, and you'll see the newly created ComfyUI text-to-image skill in the skill list above.

在这里插入图片描述

Select this skill and enter the prompt for the image you want. After entering, if you think the prompt is not good, you can click the star icon in the bottom right to let AI automatically optimize the prompt.

在这里插入图片描述

AI optimizes based on your prompt.

在这里插入图片描述

After optimization, send the command and wait for the image to be generated.

在这里插入图片描述

Image generation completed, indicating the new ComfyUI text-to-image skill works properly. Move to the next step.

在这里插入图片描述

Step 5: Call the ComfyUI text-to-image skill from the ink-wash sketch skill

Enter the workspace for ink-wash sketches, modify the current skill's execution flow, and change the image generation step to directly call the Qwen text-to-image skill.

在这里插入图片描述

Modification complete.

在这里插入图片描述

Now you can generate illustrations for free.

在这里插入图片描述

Generated successfully, without spending any credits.

在这里插入图片描述

From now on, illustration freedom achieved!