跪拜 Guibai
← All articles
Frontend · Artificial Intelligence · Testing

A 5-Agent Pipeline That Replaces the Flakiest Parts of UI Automation

By 狂师 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

UI test suites die because maintenance costs outpace writing speed. Splitting the problem into five single-purpose agents—each producing a verifiable artifact—lets teams automate the repetitive 80% without surrendering quality control. The agents are independently callable, so a team can drop one into an existing Playwright or Selenium stack without rebuilding everything.

Summary

UI automation breaks constantly because every step—element grabbing, script writing, CI hardening, visual checks, and maintenance—is manual and fragile. A single monolithic AI skill can't fix this. Instead, a chain of five narrowly-scoped agents, each with a single responsibility, forms a closed loop from page parsing to self-healing maintenance. The pipeline starts with a BFS crawler that outputs a standardized pages.yaml, feeds it to a generator that produces POM classes, test cases, and data in one shot, then passes the scripts through an enhancer that adds smart waits, popup handling, and captcha solving. A visual assertion agent adds pixel-level screenshot comparison across browsers and viewports, and an optional maintainer detects DOM changes and auto-repairs broken locators.

The architecture's core rule is that AI does the grunt work from 0 to 80, and a human verifies the final 20. Each agent can run independently, so teams can adopt only the pieces they need. The approach compresses weeks of manual scripting into minutes, but the output still requires a human to check locator strategies, data coverage, and baseline accuracy against the real application.

Takeaways
A single "universal" AI skill for UI testing becomes bloated and brittle; five specialized agents, each handling one phase, form a maintainable pipeline.
The ui-page-parser agent crawls a site via BFS from one entry URL, reuses login state through CDP, and outputs a standardized pages.yaml with multi-level locator strategies.
ui-testscript-generator merges data construction and script generation because test data and page operations are tightly coupled in UI tests—one input produces POM classes, test cases, and data files.
ui-testscript-enhancer replaces hardcoded sleeps with intelligent waits, adds popup blocking, iframe switching, exception retries, failure screenshots, and captcha recognition.
ui-visual-assert goes beyond DOM checks with pixel-level screenshot comparison, responsive-viewport testing, and separate baselines per browser engine to avoid cross-browser false positives.
An optional ui-auto-maintainer detects DOM changes via diffing, self-heals broken locators using visual similarity, and updates visual baselines after intentional redesigns.
Every agent's output must be verified by a human—checking traversal completeness, locator correctness, data coverage, and baseline accuracy—before it reaches production.
Conclusions

Merging test-data generation with script generation is a pragmatic design choice that reflects how tightly form fields and page actions are coupled in UI tests; splitting them would create an awkward handoff.

The insistence on prohibiting long XPaths and preferring data-testid then semantic locators is a concrete, enforceable rule that an agent can apply more consistently than a human team following a style guide.

Positioning the maintainer as optional acknowledges that many teams abandon UI automation before maintenance becomes the bottleneck; it's a realistic adoption curve rather than an all-or-nothing architecture pitch.

Concepts & terms
BFS crawler for page discovery
A breadth-first search algorithm that starts from a single entry URL, follows links to discover all pages on a site, and feeds them into the parsing agent without requiring a manually maintained URL list.
CDP (Chrome DevTools Protocol) login-state reuse
A technique where the agent launches a Chrome instance and attaches via the DevTools Protocol to reuse an existing authenticated session, allowing it to parse pages behind login walls without scripting a separate login flow.
Locator strategy priority chain
A deterministic fallback order for element locators: data-testid attributes first, then semantic locators like getByRole/getByLabel, then CSS selectors, with long XPaths explicitly forbidden because they break on the smallest DOM change.
Visual baseline per browser and viewport
Maintaining separate screenshot baselines for each combination of browser engine (Chromium, Firefox, WebKit) and viewport size, so that legitimate rendering differences between browsers don't trigger false-positive visual diff failures.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗