A Local Proxy Gives Text-Only LLMs Inside Claude Code the Ability to See Images

I Gave a Text-Only Model a Pair of "Eyes": Now Pure Text LLMs Can See Images

The Origin: A Maddening 400 Error

Ever since I started using Claude Code, my coding efficiency has indeed taken off. But there's one hurdle I couldn't get past--

For text-only models like GLM and DeepSeek, there's a fatal problem in Claude Code:

You take a screenshot of a bug, paste a design draft, and want to ask it "What's wrong with this error?" or "Replicate this page layout," only to get a direct 400 Bad Request.

The reason is simple: these models don't support image input at all. Claude Code stuffs the image into the request, and the upstream server flat-out rejects it.

There is a workaround: first throw the image to Doubao/GLM-4V yourself to get it recognized, copy the text description back, and then feed it to Claude Code. It works, but doing this every single time will drive you insane.

So I spent a weekend writing image-vision-mcp, an MCP server that gives Claude Code image recognition capabilities. One line of npx to connect, Ctrl+V to paste images directly, and even text-only models can "see and speak."

What It Can Do

The project actually contains two capabilities, which can be used independently or together.

Capability 1: MCP Image Recognition Tool (Default)

Equips Claude Code with a vision_describe_image tool that supports both local file paths and network image URLs as input. Once installed, just tell Claude:

Identify this image: C:/Users/me/screenshot.png
Describe this network image: https://example.com/chart.png
Parse the design spec for restoration: C:/Users/me/design.png
Analyze visible issues in this test screenshot: C:/Users/me/bug.png

Claude will automatically invoke the image recognition tool.

Capability 2: HTTP Image Interception Proxy (Advanced Play, the Key Part)

This is what truly solves the pain point.

You set up a local proxy layer (listening on 127.0.0.1:8787) between Claude Code and the upstream LLM. When a request contains an image, the proxy will:

Intercept the image in the request;
Call a vision model (Doubao, Kimi, GLM-4V, Qwen-VL, your choice) to convert the image into a text description;
Replace the image with the description text, then forward it to the upstream text-only model.

From the upstream model's perspective, it always receives pure text—400 error? Doesn't exist.

┌───────────────────────────────────────────────────────────┐
│ Claude Code                                                │
│   ├─ Call MCP tool vision_describe_image -> Vision Model API│
│   └─ Send message (with image) -> http://127.0.0.1:8787 (this proxy) │
│                            ↓                               │
│                  Proxy intercepts image -> calls vision model to recognize -> replaces with text │
│                            ↓                               │
│                  Forwards text-only request -> Upstream LLM API (e.g., GLM)    │
└───────────────────────────────────────────────────────────┘

Once the proxy is set up, the experience is like this:

In the input box, Ctrl+V paste an image, casually type "What's wrong with this picture?", and hit Enter.

No need to first say "identify the image," no need to worry about keywords—just paste and go.

My Proudest Design: Automatic Classification for Image Recognition

While writing this tool, I discovered one thing: for different types of images, the information you actually want is completely different.

Design drafts -> You want HTML/CSS specs, tokens, and layout that can be directly restored;
Prototypes/Wireframes -> You want an understanding of the page structure and interaction logic;
Bug screenshots -> You want anomaly information and stack trace localization analysis;
Ordinary images -> Just describe the content honestly.

So I built an auto automatic classification mode into vision_describe_image: the vision model first determines what category the image belongs to, then uses the corresponding prompt template to output structured results—specifically optimized so that text-only LLMs can "understand it and get straight to work."

The four modes can be explicitly specified:

mode	Applicable Scenario
`auto` (default)	Automatic judgment, hassle-free
`design_rebuild`	Restore design draft to code specs
`prototype_understanding`	Understand prototype/wireframe structure
`bug_screenshot`	Analyze test/exception screenshots
`general`	Ordinary image description

The same applies in proxy mode: even if you don't write keywords like "restore/prototype/bug," the vision model will first auto-classify, then send the corresponding analysis result along with your question to the upstream.

Integration Takes Just 1 Minute

Step 1: Configure the MCP Server

Edit ~/.claude.json (using Volcengine Ark Doubao as an example; switch to Kimi / Zhipu / Tongyi just by changing three variables):

{
  "mcpServers": {
    "vision-mcp": {
      "command": "npx",
      "args": ["-y", "image-vision-mcp"],
      "env": {
        "VISION_API_KEY": "your-volcengine-ark-api-key",
        "VISION_BASE_URL": "https://ark.cn-beijing.volces.com/api/v3",
        "VISION_MODEL": "doubao-seed-2-1-turbo-260628"
      }
    }
  }
}

At this point, the MCP image recognition tool is already usable. To use the proxy for pasting images, add Step 2.

Step 2: Route Claude Code Through the Proxy

Edit ~/.claude/settings.json:

{
  "env": {
    "UPSTREAM_BASE_URL": "https://your-upstream-llm-api-address",
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:8787",
    "ANTHROPIC_AUTH_TOKEN": "your-upstream-llm-api-token"
  }
}

Restart Claude Code, Ctrl+V to paste an image, and you're done.

Supported Vision Models (any with an OpenAI-compatible /chat/completions endpoint that supports image_url + base64 will work):

Platform	Model Example
Volcengine Ark (Doubao)	`doubao-seed-2-1-turbo-260628`
Moonshot AI Kimi	`kimi-k2.6`
Zhipu AI	`glm-4v-plus` / `glm-4.5v`
Alibaba Bailian	`qwen-vl-max` / `qwen2.5-vl-72b-instruct`
SiliconFlow	`Qwen/Qwen2-VL-72B-Instruct` etc.
OpenAI	`gpt-4o` / `gpt-4o-mini`

Tip: On Volcengine Ark, currently only the Doubao series supports visual input; GLM/DeepSeek on Ark are text-only. To use GLM-4V / Qwen-VL, go through their respective platforms or SiliconFlow.

A Few Engineering Details Worth Mentioning

After finishing, I realized the real effort wasn't in "getting it to run," but in these edge cases:

SSRF Prevention: By default, access to internal/private network image URLs is denied. To allow it, you must explicitly set ALLOW_PRIVATE_NETWORK_IMAGES=1. A local proxy that doesn't guard against this will cause trouble sooner or later.
max_tokens Allocated by Mode + Automatic Truncation Upgrade: Different recognition modes have vastly different output lengths; design draft restoration can be very long. Token budgets are allocated per mode, and if the output is truncated, it automatically retries with an upgraded limit, preventing design draft restoration from being cut off halfway.

My Daily Usage

Checking errors: When a test run fails, take a screenshot and throw it in: "Analyze the visible issues in this test screenshot." Faster than copying logs.
Restoring design drafts: Paste a Figma screenshot directly: "Parse the design spec for restoration." The output is HTML+token ready to be modified.
Reading prototypes: Wireframes provided by the product team: "Understand the structure and interactions of this page."
Reading charts: Data charts from annual reports, monitoring dashboard screenshots: "Analyze the chart data."

Essentially, it transforms Claude Code from "can only read text" to "can also see images," and it doesn't care about the backend model—you continue using domestic text models, and leave the image recognition to a dedicated vision model.

Final Words

The project is open-sourced at github.com/staticdeng/image-vision-mcp, MIT license, npx -y image-vision-mcp to use immediately. The architecture reserves extension points; any OpenAI-compatible vision model can be connected. Issues and PRs are welcome.

If you're also using Claude Code + domestic models and have been tormented by the inability to paste images, give it a try. If you find it useful, a ⭐ is the greatest encouragement to me :)