api-scenario-runner Executes Full Business-Chain API Tests with Three-Level Assertions and CI-Ready Compilation
In the previously released SKILL, we had already run the AI pipeline for interface testing, but there was always a gap: login, add-to-cart, and order placement each tested fine individually, but when combined they didn't necessarily work — individual interfaces could all be green, yet the business could still get stuck.
These two new Skills fill that gap: treat a complete business chain like "register→login→search→add-to-cart→order→payment→query order" as a single test object.
🔹 api-scenario-modeler (scenario modeling) 🔹 api-scenario-runner (chain execution engine)
One Skill manages "what to test", the other "how to run it". They connect seamlessly with the execution, diagnosis, cleanup, and reporting skills from previous chapters. From now on, single-interface and business-scenario lines run in the same system.
In the previous article, we already introduced api-scenario-modeler (scenario modeling). Today's article breaks down the api-scenario-runner (chain execution engine) Skill.
Preparation for the hands-on project
The hands-on case in this tutorial continues to use the shop-lab full-stack e-commerce project. This business-scenario chain Skill spans 5 business modules of the project:
| Module | Role in the chain |
|---|---|
| auth (authentication) | Register, login, produce the token used throughout |
| search (search) | Find the target product, produce productId |
| product (product) | View details, obtain the stock baseline stockBefore |
| cart (shopping cart) | Add to cart, produce cartItemId |
| order (order) | Place order, pay, query order; the main battlefield for assertions |
First make sure the backend service is running and Swagger UI is accessible: http://localhost:8080/swagger-ui.html,
Business-scenario interface test execution engine: api-scenario-runner
The role of the api-scenario-runner Skill is to actually execute the scenario_xxx.yaml orchestration files produced by api-scenario-modeler (or manually): send requests in order, auto-extract and inject variables, inject auth headers, perform three-level assertions (step assertions + DB final-state assertions), keep a snapshot of every step, stop on failure and preserve the scene, and also support compiling the orchestration files into pytest or any specified framework chain script project.
Applicable scenarios
- Execute scenario orchestration files, run chains, end-to-end flow verification
- Compile scenario YAML into pytest chain scripts, integrate into api-test-executor for unified scheduling
- Preserve the full scene after a failure and hand it to api-failure-diagnoser for deep diagnosis
Usage
Provide the orchestration file path in Claude Code, or enter:
/api-scenario-runner
Two execution modes
| Mode | Description |
|---|---|
| run (default) | Execute directly: structure validation → sequential requests → three-level assertions → report + snapshot |
| compile | Compile into a pytest project: testcases/test_scenario_<id>.py + data/scenario_<id>.yaml, reusing the tested project's utils/config |
Core capabilities
- Scenario context:
${var}auto-extraction and injection (supports entire-string replacement and concatenation),auth: truesteps auto-inject Bearer token - Three-level assertions: step assertions (status_code → business code → jsonpath) → final-state DB assertions (expect_value direct comparison / expect_expr variable expression evaluation)
- Stop on failure + scene preservation: each step's snapshot is written to
steps_snapshot.json(the actual injected request, response, variable view, with sensitive values masked) - Initial attribution classification into five types: VARIABLE_MISSING / ASSERTION_FAILED / API_ERROR / CONNECTION_ERROR / DB_ERROR
- Safety mechanisms: DB assertions only allow SELECT, SQL parameterized execution, orchestration files forbid hardcoding connection info, config missing triggers fail-fast
Output: scenario_report.md (execution report) + steps_snapshot.json (diagnostic snapshot) + compile artifacts (pytest project)
Hands-on demo
Enter the following prompt, as shown in the figure:
During the Skill run, it will compare and correct the field names of each interface automatically.
It also verifies the consistency of MySQL database and Redis cache data.
After execution, SC-order-001 passes through the entire chain (the full chain includes 14 interface call steps).
At the same time, scenario_shopping_flow.yaml and modeling_notes.md have been updated to the latest version from actual tests:
Finally, an MD-format scenario test execution report is generated automatically (by default, stored in the interface test project directory /shop-lab-api-test/scenario_run_results/xxx/scenario_report.md).
Integration into the existing system (compile + pipeline):
/api-scenario-runner compile scenario orchestration file /Users/xxx/scenarios/scenario_order_flow.yaml into pytest scripts, output to the shop-lab-api-test project
Check two things in the compiled artifacts: the directory structure conforms to the layered specification from article 007 (testcases/ + data/). As shown in the figure, under testcases/, a pytest chain script test_scenario_order_001.py is generated.
Under the data/ directory, the business-scenario related script test data scenario_order_001.yaml is generated.
Everything after that is familiar: use the api-test-executor from article 009 to schedule by label (scene:business-flow), or hand it to api-pipeline-scheduler to include in the "execution → cleanup → report" pipeline.
For example, enter the following prompt:
Call the api-test-executor skill to run ~/ai_project/shop-lab-api-test/testcases/test_scenario_order_001.py, this business-scenario test script
After execution, the chain's 14 steps + 4 terminal DB checks all pass, and a compiler bug was fixed along the way.
After execution, call the api-report-generator skill to generate test reports.
It automatically generates two test reports: one is based on Allure's native report (online service).
The other is a self-developed customized HTML test report (single file, can be opened independently).
Idempotency and repeated execution
The fourth round addresses the "works yesterday, fails today" problem: without any modifications, run the same chain again immediately.
The passing criteria are very clear: the second run must also be all green. Because username is dynamically generated using a timestamp strategy, the second run registers a new user, avoiding the duplicate registration conflict of newuser_001 from article 009. Chain data recycling follows the cleanup configuration and cooperates with api-testdata-cleaner, so the environment is clean between the two runs. If the second run fails, it likely exposes one of two issues: a uniqueness field missing the dynamic strategy, or a cleanup configuration missing a data object. You can fix the orchestration file accordingly.
This round directly determines whether the chain can enter the CI pipeline and run daily. It is the most indispensable of the five rounds.
At this point, business-scenario testing is fully integrated into the automation system established in previous chapters. Single-interface and business-scenario two lines now run in the same system.
With this, the Agent Skill system for interface automation testing has completed the last piece of the puzzle. The complete picture is as follows:
How to get the Skills
All skills created during hands-on exercises will be uploaded to GitHub uniformly, and you can take what you need.
At the same time, all Skills have been uploaded to the Skill store on the website testfather.cn.
The two new Skills in this article are installed the same way as before: copy or clone the Skill directory to your Claude Code or WorkBuddy skills directory and use them.
## Online install
npx skills add xxx/api-scenario-modeler
npx skills add xxx/api-scenario-runner
# Clone repository
git clone [email protected]:xxx/skills.git
# Copy to Claude Code skills directory
cp -r skills/api-scenario-modeler ~/.claude/skills/
cp -r skills/api-scenario-runner ~/.claude/skills/
# Or copy to WorkBuddy skills directory
cp -r skills/api-scenario-modeler /Users/xxx/.workbuddy/skills/
cp -r skills/api-scenario-runner /Users/xxx/.workbuddy/skills/