Modern Web Guidance: A Searchable Browser API Spec for Coding Agents
The browser platform moves faster than model training data, so agents need a reliable way to look up current APIs and per-project compatibility rules. This tool gives them that without forcing a full vector database or a live LLM call, and its evaluation harness makes 'guidance works' a testable claim.
Coding agents routinely produce hand-rolled modal dialogs with custom overlays, Esc listeners, and inert tricks when the browser's native <dialog> element already covers most of that behavior. Modern Web Guidance, a Google Chrome project now on GitHub, treats the mismatch between browser platform changes and model training data as a retrieval problem. It ships a CLI and Agent skills that search short English queries against a curated set of Web platform guides, retrieve relevant sections, and apply per-project compatibility gates via .mwgrc.
The implementation keeps retrieval cheap and self-contained: static Int8 word embeddings for queries, BM25 for literal matches, and Reciprocal Rank Fusion to merge both lists. Feature gating blocks preview or Origin Trial capabilities unless a project explicitly opts in. A separate legacy-scan entry finds existing patterns that match modernization opportunities. The repo also includes a harness that compares guided versus unguided agents on real tasks, with Playwright assertions against behavior expectations, so guide authors can measure whether their advice actually changes generated code.
The deliberate choice to keep query embeddings static and lexical means retrieval is cheap and self-contained, but it also caps expressiveness; the tool is tuned for a bounded set of Web guides, not general semantic search.
Hard feature gating in retrieve is a policy gate, not a security boundary; an agent can still write code from its own knowledge, so compliance depends on the calling context.
The evaluation harness measures success on specific tasks and assertions, not overall code quality; a guide that improves a dialog task may not transfer to other scenarios.
Search and retrieval are deliberately separated: ranking signals relevance, while retrieval checks compatibility, preventing high-ranking but unsafe recommendations from being treated as production-ready.