SoduWork's Acceptance Checks a Fixed Manifest, Not the Working Directory
SoduWork's acceptance model exposes a core truth for anyone automating deliverables with AI agents: an agent's claim that files exist is not proof of delivery. Versioned manifests and per-round evidence give teams an audit trail, but they also add a step where things can silently fail if the agent doesn't include its output in the current version.
Running SoduWork from source on a Mac, one user set up a local Hub, connected the desktop client, and created a requirement to validate an orders CSV against acceptance criteria. The first execution generated the script and test files, and unittest reported 13 passing cases, but automatic acceptance returned "needs modification": the fixed delivery manifest only listed orders.csv, with no script, test, or evidence. The working directory is not the deliverable; acceptance compares against a pinned version of the manifest.
The rework cycle required updating the current version's manifest and evidence, not rewriting the code. Later rounds listed SHA-256 hashes, CLI output, and unittest logs tied to each acceptance criterion. The final pinned version reached v8 with 7 main deliverables and an "accepted" status, with a ZIP download.
The report also documents an environment mismatch (system Python 3.9.7 vs required 3.12+, Node 22.12.0 vs 22.19+) and a macOS Keychain credential error in the source client, which the user worked around by using the downloaded desktop client against the same local Hub. The author notes that ambiguous instructions can cause the agent to prioritize acceptance criteria over task text.
The acceptance loop separates the agent's local file system from the versioned deliverable, which means an agent that writes files and passes tests can still fail acceptance if it doesn't update the manifest. This is a design choice that forces agents to reason about what constitutes delivery, not just about creating output.
The user's own conflicting instructions show that requirement quality is a bottleneck: the agent resolved the contradiction by favoring the acceptance criteria, but this behavior is unpredictable if the criteria themselves are ambiguous. AI agents need unambiguous, versioned specifications.
The source client's Keychain error and the workaround via the downloaded client suggest that local agent infrastructure still has rough edges on macOS, and that the error message from safeStorage gives no detail, making debugging a shotgun approach. This is a reminder that agent frameworks are still young on the desktop.