跪拜 Guibai
← All articles
AIGC

SoduWork's Acceptance Checks a Fixed Manifest, Not the Working Directory

By 一只牛博 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

SoduWork's acceptance model exposes a core truth for anyone automating deliverables with AI agents: an agent's claim that files exist is not proof of delivery. Versioned manifests and per-round evidence give teams an audit trail, but they also add a step where things can silently fail if the agent doesn't include its output in the current version.

Summary

Running SoduWork from source on a Mac, one user set up a local Hub, connected the desktop client, and created a requirement to validate an orders CSV against acceptance criteria. The first execution generated the script and test files, and unittest reported 13 passing cases, but automatic acceptance returned "needs modification": the fixed delivery manifest only listed orders.csv, with no script, test, or evidence. The working directory is not the deliverable; acceptance compares against a pinned version of the manifest.

The rework cycle required updating the current version's manifest and evidence, not rewriting the code. Later rounds listed SHA-256 hashes, CLI output, and unittest logs tied to each acceptance criterion. The final pinned version reached v8 with 7 main deliverables and an "accepted" status, with a ZIP download.

The report also documents an environment mismatch (system Python 3.9.7 vs required 3.12+, Node 22.12.0 vs 22.19+) and a macOS Keychain credential error in the source client, which the user worked around by using the downloaded desktop client against the same local Hub. The author notes that ambiguous instructions can cause the agent to prioritize acceptance criteria over task text.

Takeaways
— SoduWork is an open-source, enterprise-grade AI-native engine; its source is at github.com/mdiri-ai/SoduWork.
— Running from source requires Python 3.12+, Node.js 22.19+, and Git; the test machine had system Python 3.9.7 and Node 22.12.0, and worked around by using python3.12 and nvm.
— The source-launched desktop client on macOS hit a "cannot access device credentials" error from safeStorage.decryptString() failures; the downloaded client connected to the same local Hub instead.
— The CSV validation requirement specified acceptance criteria: three fields, empty values, duplicate order numbers, amounts greater than 0, valid YYYY-MM-DD dates, and test files with raw execution output.
— The first delivery had validate_orders.py and test_validate_orders.py in the working directory and 13 unittest cases passing, but the fixed manifest only listed orders.csv, so acceptance failed.
— Rework required adding the script and test files to the current version's manifest and evidence, not just regenerating them.
— The final version v8 had 7 main deliverables, was marked "accepted", and offered a ZIP download.
— The user's own instructions were contradictory: the task text said not to create tests, but acceptance criteria required them; the agent prioritized acceptance criteria.
Conclusions

The acceptance loop separates the agent's local file system from the versioned deliverable, which means an agent that writes files and passes tests can still fail acceptance if it doesn't update the manifest. This is a design choice that forces agents to reason about what constitutes delivery, not just about creating output.

The user's own conflicting instructions show that requirement quality is a bottleneck: the agent resolved the contradiction by favoring the acceptance criteria, but this behavior is unpredictable if the criteria themselves are ambiguous. AI agents need unambiguous, versioned specifications.

The source client's Keychain error and the workaround via the downloaded client suggest that local agent infrastructure still has rough edges on macOS, and that the error message from safeStorage gives no detail, making debugging a shotgun approach. This is a reminder that agent frameworks are still young on the desktop.

Concepts & terms
Fixed delivery version (固定交付版本)
A snapshot of the deliverables that automatic acceptance checks against, independent of the working directory. In SoduWork, each requirement round generates a pinned manifest, and only files in that manifest count for acceptance.
Hub (本机 Hub)
The local server instance run from source, at http://127.0.0.1:18989, which both the browser admin and the desktop client connect to. It manages models, requirements, and executions.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗