Violin: A Zig-Powered Coding Agent That Lays Bare the While-Loop at AI's Core
1. Background
Since the winter of 2024, various coding agents have emerged endlessly—starting with Gemini CLI, trying Claude Code, and now Codex, OpenCode, PI, Qoder, and others are flourishing.
In just over a year, agents have gradually evolved from an appendage of large models into an amplifier of model capabilities, becoming a key carrier for AI engineering. Various business agents on the market—customer service, data analysis, workflow orchestration, etc.—are basically generalized variants of coding agents when traced back to their roots.
Understanding the construction principles of a coding agent means grasping a key to understanding other agents. The author is very interested in the principles of the agents they use every day, so they decided to write one themselves: violin.
2. Effect Preview
Although a bit ugly, the basic functions (multimodal, skills, plugins) are already complete.
3. Overall Architecture
Violin's architecture design deeply draws on Pi's design philosophy. As a TypeScript-implemented AI coding agent, Pi's most prominent feature is its concise and extensible architecture—three-layer separation (model adaptation layer / kernel layer / product layer), EventBus event-driven, tool registry, and plugin system, with each module performing its own duties and loosely coupled.
Even more rare is that Pi's code is completely open source, which not only gives Violin a solid reference blueprint but also makes it an excellent Agent engineering learning model: clear code structure, detailed comments, clear responsibilities for each layer. For those who want to learn Pi's architecture, this project is highly recommended: how-pi-agent-works.
It can be said that understanding Pi's architecture means understanding the core design patterns of modern AI coding agents. While maintaining the essence of this architecture, Violin replaced TypeScript with Zig, making further explorations in memory safety and performance.
Violin agent's architecture is as follows:
Why Zig?
Violin's architecture design deeply draws on Pi's three-layer separation and EventBus solution, but made a completely different choice in implementation language. Since each layer is decoupled through interfaces (or network protocols), it is entirely feasible to implement different layers in different languages—it is not necessary nor possible to "write everything in one language."
Violin's division of labor is: the underlying engine such as Agent Loop, model adaptation, and session management is implemented in Zig, pursuing ultimate performance and memory controllability; while the Client side is implemented in Python, leveraging Python's ecosystem to quickly build terminal interaction UI.
The two communicate via TCP + JSON line protocol, and the Server does not care what language the Client is written in. The author's tech stack happens to be Zig + Python, not knowing JS and not familiar with Rust, but this combination perfectly covers both ends of "engine" and "interface," with clear division of labor, each leveraging its strengths.
Why Each Layer Exists
ai: Flattening Providers
Different model APIs express tool calls, reasoning content, caching, errors, OAuth, and streaming protocols differently. Violin unifies these differences into Message, Tool, AssistantMessageEvent, and streamSimple().
This way, the upper-level Agent Loop does not need to know "this is Anthropic's tool_use, or OpenAI Responses' function call." It only cares about the unified toolCall content block.
agent-core: Only Manages Agent Runtime
product: Turning Agent into a Usable Product
Writing an Agent Loop is not difficult; the difficulty lies in turning it into a development tool that can be used daily. The product layer is responsible for these "troublesome but critical" things:
server: Wrapping Agent Capabilities as a TCP Server
To make the client implementation language-agnostic, we need to wrap the agent capabilities into a communication protocol + a TCPServer. TCPServer's streaming communication + full-duplex is naturally suited for this scenario.
4. Layered Implementation Introduction
Agent Loop (The Core of Everything)
What is an Agent Loop? A while loop that switches back and forth between "asking the model" and "executing tools" until the model gives a final answer. It does not care whether the model is OpenAI or Anthropic, nor whether the tool reads files or runs commands; it only cares about two things: does the model want to call a tool? If yes, continue asking after execution; if no, end.
This is the core code of Agent Loop (simplified):
while (turn < max_turns) : (turn += 1) {
// 1. Call model
const assistant = try model.complete(.{
.messages = messages.items,
.tools = tool_registry.definitions(),
});
// 2. Check if there are tool calls
const has_tool_calls = assistant.toolCalls().len > 0;
// 3. No → end; Yes → execute tools one by one
if (!has_tool_calls) break;
for (assistant.content) |block| {
if (block == .tool_call) {
const result = tool_registry.execute(tc.name, tc.args);
messages.append(result); // Write back to model
}
}
}
Points to note here:
Why max_turns is needed — The model may fall into an infinite tool loop; the upper limit is a safety valve.
Why tool results need to be appended back to messages — The model needs to see the results to decide the next step.
Why tools definitions need to be passed into complete() — The model needs to know in advance what tools are available.
Of course, both LLM calls and tool calls can fail, so we need to add a retry mechanism at the AgentLoop layer. First, we need to distinguish error types, whether it is a retryable error. Violin has done a simple classification:
Agent Loop only cares about the "loop," not where messages come from or where execution results are stored. These "outside the loop" matters—loading and saving session history, retrying after context compression, Arena memory management—are encapsulated by the upper product layer (product/agent.zig).
agent.zig passes historical messages when calling loop.run(), and after the loop returns, it is responsible for persisting new messages to the Session and truncating context for retry on ContextOverflow.
AI Model Adaptation Layer (Who Agent Loop Calls)
The model adaptation layer's responsibility is just one sentence: encapsulate the API differences of different LLM Providers behind a Model.complete() interface. Agent Loop only recognizes this interface, not caring whether the backend is OpenAI or Anthropic.
Simplified core logic (pseudocode):
interface ModelAdapter {
complete(input: CompleteInput) -> AssistantMessage
name() -> string
}
// Each Provider implements this interface:
// OpenAIAdapter — calls /chat/completions
// AnthropicAdapter — calls /v1/messages
struct CompleteInput {
system_prompt: string,
messages: Message[],
tools: ToolDefinition[],
max_tokens: int,
temperature: float,
stream_callback: optional callback, // Streaming output
}
The actual Zig implementation uses a function pointer table (Zig has no traits or virtual functions, using function pointers for polymorphism). Each adapter provides three function pointers (complete, name, deinit), called through a unified interface. The adapter's internal state (such as base_url/api_key) is restored in callbacks through type-erased pointers.
Violin adapts the two most popular LLM protocols currently:
The two adapters have similar code volume (531 vs 558 lines) because they each need to handle JSON serialization, HTTP requests, SSE streaming parsing, and error mapping. The differences are mainly in: Request body format differs (OpenAI uses messages[], Anthropic uses content[]). Tool call structure differs (OpenAI is tool_calls[], Anthropic is tool_use block in content[]). Streaming protocol differs (OpenAI uses data: lines, Anthropic uses event: lines).
LLM generating an answer may take several seconds or even tens of seconds. If you wait for the entire generation to complete before returning, the user can only wait idly. For a smooth client experience, mainstream Agents currently adopt streaming output. The solution to this problem is streaming (SSE).
// Model adapter calls stream_callback when receiving each SSE chunk
// Agent Loop immediately emits message_update event via EventBus after receiving callback
// Client appends to terminal display in real-time after receiving event
pub const StreamCallback = *const fn (ctx, chunk: StreamChunk) bool;
Violin has no built-in models; all model configurations are loaded from ~/.violin/agent/models.json, and the client decides which model to use:
{
"providers": {
"openai": {
"base_url": "https://api.openai.com/v1",
"api": "openai-completions",
"api_key": "$OPENAI_API_KEY",
"models": [
{ "id": "gpt-4o", "name": "GPT-4o", "contextWindow": 128000 }
]
}
}
}
Tool System (Agent Loop's Hands and Feet)
The model adaptation layer allows Agent Loop to call any model, and the tool system allows Agent Loop to do anything.
Tool definition:
pub const Tool = struct {
name: []const u8,
description: []const u8,
parameters: []const u8, // JSON Schema
execute: ToolExecuteFn,
};
Violin has 6 built-in basic tools, consistent with Pi's design:
To facilitate managing these tools, Violin implements a tool registry (pseudocode): The actual Zig implementation uses HashMap to store tools, quickly looking up by name. definitions() encodes the tool list into JSON Schema format recognizable by the model.
// Simplified tool registry logic:
classToolRegistry:
tools: Map<string, Tool>
funcregister(name, description, execute_fn):
tools[name] = Tool(name, description, execute_fn)
func get(name) -> Tool:
return tools[name]
func definitions() -> ToolDefinition[]:
// Return all tool definitions, send to model
return [t.definition() for t in tools.values()]
func execute(name, args) -> ToolResult:
tool = tools[name]
return tool.execute(args)
Product Layer (Things Outside the Loop)
agent.zig — The Glue Layer, but the Most Critical
product/agent.zig is only 123 lines, but it is the layer that "glues" the entire project together. What it does is simple:
- Load historical messages from Session.
- Pass to loop.run() for execution.
- After loop returns, save all new messages to Session.
- If loop throws ContextOverflow, call compaction to compress and retry.
Show core logic (simplified):
// Simplified agent.run() logic:
func run(config, user_input) -> AgentResult:
history = load_history(config.session)
while True:
loop_result = loop.run(
model=config.model,
initial_messages=history,
input=user_input
)
if loop_result == ContextOverflow:
compact_session() // Compress history, keep key info
continue // Retry after compression
// Save new messages to session
for msg in loop_result.new_messages:
session.appendMessage(msg)
return AgentResult(text=loop_result.final_text)
session.zig — The "Memory" of Conversation
Without Session, the Agent is "amnesiac" for each conversation. Storage format: JSONL, each line an independent JSON object. The first line is the session header (id, created_at, cwd), and subsequent lines are each a message.
{"id":"sess_001","created_at":1717234567,"cwd":"/project","model":"gpt-4o"}
{"id":1,"parent_id":null,"timestamp":1,"role":"user","content":"Help me read README.md"}
{"id":2,"parent_id":1,"timestamp":2,"role":"assistant","content":"I'll help you read..."}
To manage Sessions, violin adds a SessionStore data structure:
// Simplified SessionStore data structure:
classSessionStore:
file_path: string // JSONL file path
entries: Map<id, Entry> // Message index, supports random access
leaf_id: int | null // Current leaf node (latest message)
next_id: int // Auto-increment ID, used to generate new message IDs
header: SessionHeader // Session header info
The actual Zig implementation uses ArenaAllocator to uniformly manage memory (all allocations come from arena, released at once on deinit), saving the trouble of freeing each one individually.
Core methods are as follows:
Message persistence writeEntry core logic (pseudocode):
// Simplified message persistence logic:
func writeEntry(entry):
json_line = serialize_to_json(entry) + "\n"
// Write to temp file, then append to session file
// Using temp file method avoids trouble handling special characters
write_temp_file(json_line)
append_to_session_file(temp_file)
Append message (pseudocode):
// Simplified append message logic:
func appendMessage(message) -> entry_id:
id = next_id++
entry = Entry(
id: id,
parent_id: current_leaf_id, // Attach after current message
timestamp: now(),
message: message
)
entries[id] = entry
leaf_id = id // New message becomes latest message
writeEntry(entry) // Persist to JSONL file
return id
Session recovery from file (pseudocode):
// Simplified session recovery logic:
func load():
if not file_exists(file_path):
createSession() // File doesn't exist, create new session
return
for each line in read_lines(file_path):
if is_first_line:
header = parse_header(line) // First line = session header
is_first_line = false
else:
entry = parse_entry(line) // Subsequent lines = messages
if entry is valid:
entries[entry.id] = entry
else:
log_warn("Skipping corrupted message") // Skip corrupted lines, don't crash
Comparison with Pi: Pi's Session is also JSONL + tree structure, supporting tree-shaped conversation structure, supporting branch fork and rollback. Violin inherits these capabilities.
compaction.zig — Conversation "Brain Capacity Management"
When tokens exceed the threshold, compress old messages into a summary, keeping the most recent N messages. The Agent layer's retry loop checks the compression result and re-executes the loop.
LLMs have context window limits. A Coding Agent's conversation may last dozens of rounds, accumulating thousands of tokens. If not handled, early messages will be truncated by the window, and the model "forgets" the previous context. When tokens exceed the threshold, compress old messages into a summary, keeping recent messages unchanged.
Before compression:
[Message1] [Message2] [Message3] ... [MessageN-10] [MessageN-9] ... [MessageN]
After compression:
[Summary: Key points discussed earlier] [MessageN-9] [MessageN-8] ... [MessageN]
Core design:
Token Budget
// No tokenizer introduced, approximate with character count/4. Good enough—compression decisions don't need single-token precision.
pub fn estimateTokens(text: []const u8) usize {
return text.len / 4;
}
Trigger Condition
func needsCompaction(messages, config) -> bool:
return estimateMessagesTokens(messages) > config.max_tokens
Default threshold 100K tokens, keep most recent 10 messages, summary target length 500 tokens.
Summary Generation
Concatenate old messages into text, call model to generate summary:
[user]: Help me write an HTTP server
[assistant]: You can use Zig's std.http...
[tool_call]: write_file("server.zig", ...)
[tool_result]: File written
--->
User requested writing an HTTP server, assistant used Zig standard library to create server.zig, implementing basic request handling.
Violin's compression system is divided into 2 layers:
resources.zig — Resource Loading "Injecting Soul into Agent"
Resources loads project rules (AGENTS.md) and skills (SKILL.md) from the file system, parses frontmatter, formats them as system prompt injected into the LLM, letting the model know what tools and capabilities it has available. ResourceLoader is responsible for loading three types of resources from the file system, formatting them as system prompt for LLM use:
- Project Rules — AGENTS.md/ CLAUDE.md.
- Skills — SKILL.md files under .agent/skills/.
Data structure (pseudocode) is as follows:
// Simplified data structure:
struct Skill {
name: string // Skill name
description: string // Skill description
filePath: string // SKILL.md file path
source: enum // "global" or "project"
content: string // SKILL.md original text
}
struct Resources {
rules: ProjectRules
skills: Skill[]
cwd: string
}
Resource search paths are as follows:
Project rules (priority from high to low):
{cwd}/AGENTS.md
{cwd}/CLAUDE.md
~/.violin/agent/AGENTS.md
~/.violin/agent/CLAUDE.md
Skills (project first, global later, project wins on name conflict):
{cwd}/.agent/skills/*/SKILL.md
{cwd}/.agents/skills/*/SKILL.md
~/.violin/agent/skills/*/SKILL.md
loadAll()
├─ loadProjectRules()
│ ├─ Try {cwd}/AGENTS.md
│ ├─ Try {cwd}/CLAUDE.md
│ ├─ Try ~/.violin/agent/AGENTS.md
│ └─ Try ~/.violin/agent/CLAUDE.md
│
└─ loadSkills()
├─ Scan {cwd}/.agent/skills/*/SKILL.md
├─ Scan {cwd}/.agents/skills/*/SKILL.md
└─ Scan ~/.violin/agent/skills/*/SKILL.md
└─ Report diagnostic on name conflict, skip
SKILL.md files all carry a YAML frontmatter. Violin parses this content and builds it into XML data, which is ultimately injected into the system prompt:
<available_skills>
<skill>
<name>basedpyright</name>
<description>Python static type checking</description>
<location>/path/to/skill/SKILL.md</location>
</skill>
</available_skills>
Event System (Foundation for Plugin Implementation)
Agent Loop is running, but how does the outside world know what step it has reached? — The event system is the answer. Every time Agent Loop does something (starts a turn, generates a token, calls a tool), it sends an event to EventBus. Whoever cares about this event registers a callback.
The advantage of plugins is plug-and-play, dynamic loading. Initially, the following solutions were considered:
Ultimately, Lua was chosen. Not because Lua is the best language, but because it is the smallest right one. 500KB runtime, 20 years of proven embedding scenarios (from WoW to Redis to Nginx), callable with one C function—for a Zig project, this is the shortest path choice.
Plugin system architecture is as follows:
The core mechanism is very simple: EventBus has three callback slots (agent/session/compaction). install() saves the original callback and replaces it with its own dispatch wrapper function. It first executes the original callback (writing socket streaming back to the client), then iterates through all registered Lua plugins, calling the corresponding hook function for each.
What does a plugin look like?
-- ~/.violin/plugins/bash-guard.lua
return {
name = "bash-guard",
version = "0.2.0",
description = "Intercept dangerous bash commands, auto-add safety prefix",
-- tool_start: Intercept before tool execution
on_tool_start = function(event)if event.tool_name == "bash" then-- Block dangerous commandsif event.arguments:find("rm -rf", 1, true) thenreturn { action = "block", reason = "Dangerous command blocked" }
end-- Modify command (add safety prefix)return { action = "modify", arguments = "set -e; " .. event.arguments }
end-- No return = allowend,
-- tool_end: Modify tool execution result
on_tool_end = function(event)if event.tool_name == "bash" and event.is_error thenreturn { action = "modify", content = "Error auto-handled", is_error = false }
endend,
-- context: Inject system instruction before LLM call
on_context = function(event)return { action = "modify", inject_text = "Be careful when using bash", inject_role = "system" }
end,
-- agent_start: Block agent startup
on_agent_start = function(event)-- Block agent startup under certain conditions-- return { action = "block", reason = "Execution not allowed currently" }end,
-- session_before_compact: Block manual compaction
on_session_before_compact = function(event)if event.reason == "manual" thenreturn { action = "block", reason = "Manual compaction prohibited" }
endend,
}
Network Layer (Foundation for Client Implementation)
The network layer defines how the Violin client and server communicate—the client sends a message, and the server pushes the thinking process, tool calls, and final answer to the client one by one, like watching a live broadcast of someone thinking, doing, and speaking simultaneously.
Why C/S architecture? Most Coding Agents are monolithic—the Agent engine is directly embedded in the editor or CLI tool, with startup, model loading, tool execution, and session management all completed in one process.
Violin chose front-end and back-end separation:
Monolithic: [agent + UI + session] — One process, runs locally
Violin: [Zig Server (daemon)] ←TCP/JSON-lines→ [Python TUI Client]
Can also connect other clients
This design choice was also inspired by the ACP protocol. ACP protocol has higher complexity, and Zig doesn't have good ACP implementation dependencies. As a toy project, violin chose the smallest and easiest way—TCP + JSON LINES.
A detailed communication protocol description is maintained in the project. If users want to implement clients in other languages, they just need to hand the communication protocol to AI to vibe out a new client.
Limited by space, only a few core protocols are briefly introduced:
Handshake
Client → Server:
{"type":"handshake","cwd":"/home/user/project/violin"}
Server → Client:
{"type":"models_result","models":[...],"default":"deepseek-v4-flash"}
{"type":"skills_result","global_skills":[...],"project_skills":[...]}
cwd is used to load project skills {cwd}/.agent/skills/ and inject system prompt.
Chat Request
Client → Server:
{"type":"chat","content":"List files in directory","model":"deepseek-v4-flash"}
Optional fields: session_id, temperature, max_tokens, system_prompt, images.
Event Stream
When the server processes a chat request, it pushes events in a streaming manner. Violin designed 8 event types to fulfill conversation needs:
Complete event stream example:
// Client sends
→ {"type":"chat","content":"Help me read README.md"}// Server starts streaming back
← {"type":"turn_start","session_id":"sess_001","model":{"id":"gpt-4o","provider":"openai","label":"GPT-4o"},"context":{"tokens":5400,"window":128000,"usage_pct":4.2}}
← {"type":"delta","text":"I'll"}
← {"type":"delta","text":"help"}
← {"type":"delta","text":"you read"}
← {"type":"tool_start","name":"read_file","args":{"path":"README.md"}}// ... Waiting for tool execution ...
← {"type":"tool_end","name":"read_file","ok":true,"output":"# Violin\n..."}
← {"type":"delta","text":"The file"}
← {"type":"delta","text":"content is:"}
← {"type":"delta","text":"Violin is an AI assistant"}
← {"type":"turn_end","turn":0,"usage":{"input_tokens":5400,"output_tokens":18,"cost_usd":0.0115}}
← {"type":"result","text":"I'll help you read...","turns":1,"aborted":false,"session_id":"sess_001","usage_total":{"input_tokens":5400,"output_tokens":18,"cost_usd":0.0115}}
Python Client Implementation
With the previous section's network protocol as a foundation, we can write a simple Violin Python client. The core of the Python Client is an async TCP connection + event dispatcher:
classViolinClient:
"""Async Violin protocol client (no UI, just I/O)."""
def__init__(self, host="127.0.0.1", port=9877):
self.reader = None
self.writer = None
self.models = []
self.session_id = ""
# Callbacks—UI layer registers these, achieving separation from display
self.on_delta = None
self.on_tool_start = None
self.on_tool_end = None
self.on_result = None
...
asyncdefconnect(self, retries=3):
"""TCP connection → handshake → receive models_result + skills_result"""
for i inrange(retries):
try:
r, w = await asyncio.open_connection(*self.addr)
self.reader = r
self.writer = w
await self._send({"type": "handshake", "cwd": os.getcwd()})
msg = await self._read_msg()
if msg and msg.get("type") == "models_result":
self.models = msg.get("models", [])
returnTrue
except ConnectionRefusedError:
await asyncio.sleep(0.5 * (2 ** i))
returnFalse
asyncdefhandle_messages(self, state):
"""Event dispatch—read socket → call corresponding callback by type"""
whileTrue:
msg = await self._read_msg()
mt = msg.get("type", "")
if mt == "delta":
state["full_text"] += msg.get("text", "")
if self.on_delta:
self.on_delta(msg["text"])
elif mt == "tool_start":
if self.on_tool_start:
self.on_tool_start(msg["name"], msg["args"])
elif mt == "result":
self.session_id = msg.get("session_id", self.session_id)
if self.on_result:
self.on_result(msg["text"], msg.get("usage_total", {}))
return msg["text"]
elif mt == "error":
if self.on_error:
self.on_error(msg["msg"])
return""
elif mt == "ping":
await self._send({"type": "pong"})
return state["full_text"]
4. Summary
The biggest takeaway from the entire project is: there is no magic at the core of a coding agent. Stripping away all the fancy UI and features, the bottom layer is just a while loop—ask the model, get the result, whether to call a tool, ask again after calling. Each layer of Violin is patching this loop:
- Model adaptation layer solves "who to ask."
- Tool system solves "what can be done."
- Resource layer solves "can it remember, where is the SKILL."
- Plugin solves "capability expansion."
This toy is still a pile of unfilled holes from a mature Coding Agent: the tools parameter in buildJson hasn't been serialized yet, so the model can't receive tool definitions at all. Plugins have no permission isolation; Lua can do anything. The ACP protocol hasn't been connected either; for now, it can only play with itself. But as a toy agent built from scratch, the purpose of this project was never to deliver a commercial product, but to verify a judgment: understanding the construction principles of a coding agent means grasping a key to understanding other agents.
This judgment was continuously verified during the construction process—unified model adaptation, tool registration and scheduling, session persistence and recovery, context compression and retention, plugin injection and interception. These seemingly unrelated problems all converge at the bottom layer to the same loop: ask the model, call the tool, ask again. Customer service agent session management, data analysis agent tool chain orchestration, workflow agent state machine design—tracing back to the roots, these are all variations of this loop in different scenarios.
The remaining unfilled holes are both the current boundaries of the project and the starting point for the next exploration. Patching a toy project all the way to actual deployment—the process itself is the best way to learn.
Previous Reviews
Text / Jiumi
Follow Dewu Technology, technical干货 updated every Thursday
If you find the article helpful, welcome to comment, forward, and like～
Reproduction without permission from Dewu Technology is strictly prohibited, otherwise legal liability will be pursued according to law.