OpenViking on a 99-Yuan 2-Core 2G Cloud Host: Renewal Math, 128K Embeddings, and a Hidden Query Planner

I recently migrated my AI Coding Agent memory backend to a self-hosted OpenViking. The official docs look intimidating, often recommending high-end servers and local GPU memory, but in practice, as long as you think through your choices, a cheap entry-level cloud server for a few tens of yuan can run it smoothly.

This is not an installation tutorial that teaches you how to type commands; it's just a real account of the insights and trade-offs around cloud hosts, model combinations, and compute costs after going through this whole process.

1. Cloud host: why I chose the 99-yuan 2G instead of a competitor's cheaper 4G?

When buying a cloud server, it's easy to be attracted by competitors' new-customer promotions like "a few tens of yuan for 4G or even 8G". But I ended up choosing Aliyun's 99 yuan/year 2-core 2G (ecs.e-c1m1.large).

The core reason is one: think about the long-term renewal costs and avoid the trap of "low first-year price, steep increase next year."

If you're just setting up a stateless personal blog, it doesn't matter to switch hosts every year. But OpenViking is your memory database - it stores all daily conversation traces, accumulated project architecture, entity graphs, and vector indexes. It's a strongly stateful asset. Many competitors offer 60 yuan for the first year, then jump to 600, 800, or even over a thousand on renewal. A year later, to save money you'd be forced to shut down, copy gigabytes of data across clouds, reconfigure security groups and DNS resolution - the hassle costs far more than the price difference.

The biggest advantage of this Aliyun deal is the guarantee that the next year can still renew at the same 99 yuan price without tricks.

2G memory is tight, but as long as you set up Swap as a safety net, limit container memory, and offload all computation to external APIs, with the machine only doing local storage and retrieval, daily memory usage stays at just a few hundred megabytes. In exchange for a slightly tuned system, you get a stable memory foundation for just 99 yuan a year without frequent migrations - that math works out better no matter how you calculate it.

2. Vector model (Embedding): the big cost center, must be done right the first time

Running the whole system, the ongoing cost center is actually the vector model. But I gave up the idea of "free models" from the start and firmly chose to pay for commercial services, and to "get it right in one step."

Why must it be done right the first time?

VLM, intent analysis, rerank models are all real-time inference. If they don't work well today, you can change a line of config, restart, and switch. But the vector model determines the spatial geometric characteristics of your database's underlying representation. If you later want to change the vector model, even if the output dimensions are identical, all previous vectors are wasted - you must rebuild the entire index (Re-index) from scratch. As memory accumulates, the cost and downtime risk of a second rebuild are extremely high, so you must choose the best vector model from the start.

Why abandon SiliconFlow's free BGE-M3?

Many tutorials recommend SiliconFlow's free BGE-M3. In our actual vector recall tests, we found its recall quality lags a clear generation behind newer models. BGE-M3 is from January 2024, too old, and it's a pure text model - images can only be stored as attachments, not indexed, so "search images by text" is impossible.

More critically, the public free pool has been crushed by wool-party users. During our tests routing through the free model, we frequently encountered packet loss, high latency, and no response. As the foundation for core infrastructure, tying your fate to a free public pool is asking for trouble.

Why choose Volcano Ark's Doubao Vision (0.7 yuan) instead of Aliyun Qwen?

I ultimately used Volcano Ark's doubao-embedding-vision-251215, with official pricing of 0.7 yuan per million tokens for inference input.

Actually, Aliyun Bailian also has a multimodal vector model (qwen3-vl-embedding), priced identically at 0.7 yuan, but there's a fatal difference:

• Aliyun Qwen's context window is only 32K;
• Volcano Doubao's context window is up to 128K.

For agent scenarios, you often need to stuff in long development specs, long-context code snippets, or multiple high-res screenshots. 128K gives 4x the safety margin, so you never worry about overflow from long slices. Plus, OpenViking itself is open-sourced from ByteDance Volcano Engine, so native provider compatibility is the most stable. At the same price, obviously choose the 128K Doubao.

Note: If you have no need for image/screenshot retrieval at all and only do text or code notes, Aliyun Bailian's pure-text vector qwen3.7-text-embedding is a good budget alternative at only 0.5 yuan per million tokens.

3. Intent analysis (Query Planner): the "hidden switch" 90% of people miss

Many people just starting with OpenViking share a common confusion: "Why is this thing so slow to respond? Every time I finish typing a sentence in the IDE or terminal, the cursor stalls for five or six seconds before reacting?" Some even think it's because the machine isn't powerful enough, or the system itself is badly designed.

Actually, it's not a performance problem at all - most people simply haven't read the configuration docs carefully and don't know you can configure a separate intent analysis model.

In the official default example, people usually only fill in vlm and embedding. But if you don't explicitly declare query_planner in the config, OpenViking, when processing each turn of conversation, will by default fall back to calling the main VLM model to analyze whether this turn is small talk, whether to search memory, and how to extract keywords. If your VLM is a large model or one with a thinking chain (like Zhipu's glm-5.3-flash), a single pre-analysis can consume 6-7 seconds, and the front-end interaction naturally stutters like a PPT.

To fix this pre-request stutter, I first tried to find alternatives among mainstream general-purpose LLMs. But after running a round of tests, I was frustrated:

• Aliyun Bailian's official Qwen3.7-Flash has deep thinking enabled by default; one run takes 10 seconds to think;
• SiliconFlow's Qwen/Qwen3.5-9B has slow throughput (usually 30 tok/s), and each query still pauses 4 seconds;
• SiliconFlow's Qwen/Qwen2.5-7B has unstable format, tending to spit out bare arrays;
• SiliconFlow's public free pool models (like Qwen3.5-4B) are crowded by wool scripts, with frequent packet loss and timeouts.

Right when we were stuck at the general-purpose model selection bottleneck, I accidentally found a treasure model in Aliyun Bailian's model plaza - tongyi-xiaomi-analysis-flash (Lingque - dialogue analysis - flash). The official description is plain: a domain model specializing in dialogue information extraction, scene classification, and low-latency analysis.

I immediately pulled it into the test matrix and ran a head-to-head competition with the previous candidate models. It's important to note: for fair comparison, in stress-testing all candidate models, we explicitly turned off thinking mode (enabling non-chain-of-thought mode, pure direct output).

In the actual evaluation, we focused on three core dimensions: response latency, output structure, and planning performance. Under these three strict checks, this accidentally discovered treasure model gave textbook-level performance:

Dimension 1: Response latency (the time budget must be kept under 1 second)

query_planner sits on the critical path between the user pressing Enter and the model's actual answer. Even one extra second of latency is clearly noticeable stutter.

• Aliyun Bailian Qwen3.7-Flash: Even with thinking mode off (enable_thinking: false), a single analysis still takes 1.1-1.4 seconds (70-90 tok/s), a bit heavy as a frequent pre-request gate;
• SiliconFlow Qwen/Qwen3.5-9B: With thinking off, constrained by the provider's 30 tok/s generation throughput ceiling, a multi-turn analysis still takes 3.8-4.2 seconds;
• SiliconFlow free pool (like Qwen3.5-4B): The public free pool is constantly crowded by wool scripts, with severe latency jitter or even timeouts;
• Tongyi Lingque (Flash): First-token latency (TTFT) is only 0.31 seconds, generation throughput reaches 200+ tok/s, and the whole planning round takes about 0.8 seconds - perfectly within the sub-second budget.

Dimension 2: Output structure (strictly following OpenViking's JSON protocol)

OpenViking's underlying intent parser has an explicit contract for output format: the model must return a standard JSON object structure like {"queries": [...]} (directly deserialized by internal code to extract retrieval intents), with zero tolerance for format deviation:

• Common problem of general-purpose LLMs: They like to add polite fluff ("Okay, planning for you as follows"), or wrap output in json ... fences, making the clean output messy and requiring system regex to clean up;
• SiliconFlow Qwen/Qwen2.5-7B's hard flaw: Though speed is compressed to 1.8 seconds, its format is extremely non-compliant, often dropping the top-level key OpenViking requires, directly outputting a bare array [...]. Once the object structure is lost, parsing errors easily trigger retrieval failures;
• Tongyi Lingque (Flash): Its official feature is "custom analysis standard following". In actual tests, it achieved 100% strict compliance with OpenViking's JSON protocol - outputting pure bare JSON, no markdown fences, no extra text, perfectly matching OpenViking's deserialization contract.

Dimension 3: Planning performance evaluation (quality of business intent decisions)

An intent analysis model isn't just about blindly generating search terms; it must have clear business-scenario decision-making ability:

• Small talk rejection rate: For inputs like "hello", "thanks", or "help me write a bubble sort" that don't need long-term memory involvement, Tongyi Lingque can decisively determine no retrieval is needed in 0.4 seconds, outputting an empty array {"queries": []}, avoiding meaningless vector database queries and context pollution;
• Multi-turn coreference resolution and entity extraction: In multi-turn conversations (e.g., the user asks "send me the service port we mentioned earlier"), it can precisely resolve pronouns back to concrete entity terms from historical context (expanding to "OpenViking 1933 port configuration"), extracting highly precise and compact queries without rambling or hallucinating.

The results prove that this accidentally found treasure model didn't win by luck - its vertical fine-tuning focus on "dialogue extraction, scene classification, and format following" aligns textbook-perfectly with OpenViking's pre-request routing engineering needs (and crucially, it's the cheapest, far cheaper than general-purpose LLMs).

4. Rerank and VLM: pragmatic cost accounting and disaster recovery

Finally, the two remaining components:

• Rerank: I still put it on SiliconFlow, using Qwen/Qwen3-Reranker-4B. I use it because it's on a lightweight commercial billing channel (avoiding the free pool queue), and each call costs just a few cents. The key is I have 16 yuan of free credits in my account, so why not use them. In tests, scoring 4 documents took 0.26 seconds, 30 documents took 0.57 seconds, fast and accurate scoring, great value.
• VLM (multimodal semantic extraction): I use glm-5.3-flash from Zhipu's Coding package, with a dual-key primary/backup disaster recovery setup. Here's an extremely important practical experience: in the config, you must explicitly set thinking mode to reasoning_effort: "low". We ran multiple A/B tests. With no parameter or set to high, the model first deeply thinks for dozens of seconds (20-48 seconds, generating thousands of meaningless chain-of-thought tokens), while the final extracted entities and overview content are almost identical to low. Memory extraction is essentially information extraction and structured archival - it doesn't need complex mathematical logic reasoning. Turning on high is just wasting time and compute. Even with low, a single extraction still takes 4-6 seconds (with underlying base thinking and a 20-30 tok/s throughput ceiling), and it's currently the only latency point in the system. But because VLM only does background async conversation archiving and overview generation, it doesn't block front-end question interaction at all. Plus, the Coding package is extremely cheap on an annual basis, and with dual-key disaster recovery it's very stable. For now there's no need to spend money to switch it.

5. Some plain-spoken thoughts at the end

After going through this configuration, my biggest takeaways:

1. For stateful services, check renewal costs before buying a server: For a long-term memory database, it's better to spend time tuning on 2G than to buy a machine with skyrocketing renewal prices just to save a few yuan in the first year. Moving is really tiring.
2. Core assets should be done right the first time; stateless nodes can be optimized anytime: The vector model determines your underlying space. Don't be greedy for free. Multimodal + large context (128K) locks it in directly, avoiding a second rebuild later. VLM and Rerank are pure inference nodes - switch whoever is faster and cheaper.
3. Specialized small models beat general-purpose large models in vertical scenarios: For tasks like intent analysis that require "classification, extraction, strict JSON format, and speed", fine-tuned dialogue analysis small models are far better than general-purpose large models that write long essays and use thinking chains.