Who's Driving for AI: Agent Browser Ecosystem Survey (2026)

Synced to personal site: Who's Driving for AI: Agent Browser Ecosystem Survey (2026)

At our company hackathon at the end of last October, our team built an Agent that could operate a computer: let it open a browser and navigate web pages for me (I uploaded the demo to Bilibili). The Agent invocation itself was fine, but it was painfully slow when operating the browser: every step required a detailed parse of the page structure, and during the operation I could only sit and watch, unable to use that computer. That speed and efficiency were worse than me just clicking myself.

After that, I started paying attention to the ecosystem of Agents operating computers. Projects like Browser Use and Codex Desktop showed me more mature approaches. And today—the very days I'm writing this article—I discovered that Claude Code's built-in browser tool does the same thing when helping me verify pages: without opening a visible browser window, the verification was done. (Only later did I learn that this was a formal feature just released for the desktop version in July 2026.) How exactly does it "see" web pages?

A page on screen is a bunch of pixels. AI has neither eyes nor your mouse-clicking finger. Following this question, I found that from 2025 to 2026, "browsers for AI" had grown into a complete industry.

This article is a record of that survey.

1. Three Roles: Protocol Makers, Framework Writers, Server Sellers

Imagine building a self-driving car; you need three groups: someone defines the interface standards for the steering wheel and accelerator, someone writes the driving logic, and someone builds and maintains the fleet. The Agent browser ecosystem corresponds exactly to these three layers:

• Protocol Makers: Define "how programs talk to browsers." The current situation is a transition between old and new—the old interface (CDP, the remote control protocol behind Chrome DevTools) has entered maintenance mode, while new standards (BiDi, MCP, WebMCP) are taking over.
• Framework Writers: Solve "how to drive a browser to do work," represented by Browser Use, Stagehand, Skyvern.
• Server Sellers: Solve "where the browser runs." When thousands of AIs go online simultaneously, each needs a browser environment, leading to hosted/self-hosted browser clouds like Steel, Hyperbrowser, Browserbase.

The practical benefit of this layer division: make decisions separately when choosing tools. Frameworks and infrastructure are not competitors—Browser Use paired with Steel, Stagehand paired with Browserbase, are common combinations.

2. Two Major Official Tools: One Drives, One Repairs

In 2026, browser giants each shipped official AI interfaces. Understanding their division of labor is understanding the skeleton of the entire ecosystem.

Chrome DevTools MCP (by Google, with Microsoft collaboration): Gives AI the full capabilities of Chrome debugging tools—open pages, click, fill forms, inspect network requests, do performance analysis, take snapshots. It graduated from experimental to stable with Chrome 149 in June 2026 (official release notes). The most practical capability is "attaching to your running Chrome"—your login state, cookies, extensions are all usable, bypassing the biggest obstacle for AI browsing: login walls.

Playwright MCP (by Microsoft): Positioned as "testing and automation"—abstracts browser operations into stable, repeatable steps, across Chrome, Firefox, and Safari.

The industry has an apt analogy: Playwright is the driver, DevTools is the mechanic (Steve Kinney's article). One is responsible for "driving steadily," the other for "diagnosing when the car breaks." Interestingly, the cost: many assume Playwright saves more tokens, but the opposite is true—it sends a full page structure snapshot to the AI for every operation, which can be tens of thousands of tokens for complex pages; DevTools fetches on demand, making multi-step workflows an order of magnitude cheaper (third-party comparison).

3. AI's Two Kinds of "Eyes": Read the Menu, or Look at the Photo

Understanding who does the work, let's look at how AI actually "sees" web pages—this is the core divide of the entire ecosystem.

Read the Menu (DOM route): Inside the browser, a web page has a "structural menu"—what the title is, what the button is called, where the input box is. AI reads this menu to operate: tokens are cheap (a menu is a few thousand words), actions are precise, speed is fast. The cost is fragility: if the website redesigns, or if content is drawn on a canvas instead of text (many charts and CAPTCHAs are like this), the menu becomes unreadable. Also, websites can recognize "a bot is coming" and anti-scraping measures will block it.

Look at the Photo (Visual route): For every step, take a full-page screenshot, and AI decides where to click by looking at the picture. This is the approach of various "computer use" implementations. The advantage is "what a human can see, it can click"—any interface, any website, even desktop software can be operated. The cost is expense: every glance is a photo's worth of tokens, costing about an order of magnitude more than reading the menu (third-party evaluation Respan measured Anthropic's solution at about $0.3–1.5 per task, the most expensive among mainstream solutions); and guessing coordinates from an image often leads to misclicks.

A third path is just emerging: websites actively cooperating. The WebMCP proposal (W3C draft) jointly drafted by Google and Microsoft lets websites declare "I support these operations"—for example, a ticketing site directly tells AI "my search interface looks like this," so AI no longer needs to read menus or look at photos, connecting directly at the source. Officials claim it's 8-12 times faster than visual loops (early benchmarks, not widely reproduced), currently in Chrome's public trial phase.

The 2026 consensus is hybrid: default to reading the menu (cheap), fall back to looking at photos when the menu fails (safety net), and solidify smooth operations into scripts for direct execution next time (zero AI cost). This "cheaper with use" route is best executed by Stagehand—let AI do it the first time, copy it thereafter, official data shows up to 44% faster.

4. Closer to You: Giving Coding Agents "Browser Eyes"

The two routes are about "how to see." In 2026, a batch of tools emerged, landing these two routes on a more everyday problem: when you ask AI to modify frontend code, how does it verify it changed correctly? Using the previous section's routes as reference, here's how they chose sides:

Menu Readers—taking "reading the menu" to the extreme:

• Claude Code Desktop Built-in Browser (released Week 28, July 2026): Claude opens any website in a built-in browser panel, clicking, reading, and interacting as if verifying a local dev server. Official emphasis on two points: sandboxed, and a safety classifier reviews its operations on external sites; uses a clean independent profile by default (without your login state), and when needing to operate "as you," the official recommendation is to use the Chrome extension. The tool that "finished verification without opening a window" mentioned at the beginning is this.
• agent-browser (Vercel, 40k+ stars): A command-line tool written in Rust, with built-in MCP. Page snapshots directly output a numbered accessibility tree, AI clicks elements by number, no need to guess CSS selectors. Runs headless by default, offers six solutions for login state, including grabbing it directly from your running Chrome. Vercel also made a companion skill enforcing "must verify with browser after starting dev server"—it survived about three weeks before being replaced by a more complete verification skill, showing how fast this field iterates.

Photo Lookers—answering "the menu says it's fine, but does the page look right":

• Frontend-VisualQA (Yutori): It doesn't look at the DOM, directly uses a visual model to look at screenshots, judging "did the progress bar really reach 100%" or "did the popup render off-screen." It solves the innate blind spot of menu readers: DOM assertion passing ≠ the page is actually correct.

Login State and Workflow Faction—concerned not with "how to see," but with "how to use it":

• ego-lite (Citro Labs, ~10k stars): A browser that can share your login state. It transforms "AI operates a browser" from "open a blank browser, login state all manually transferred" to "AI uses your browser, but isolated in its own Space, not stealing your mouse or tabs." A third-party evaluation claims token consumption for login pages dropped from 30k+ to 200-400 (official only makes qualitative claims, see references for data source). Currently macOS only.
• Shiplight MCP: Turns "verification" into a sustainable workflow—AI runs verification in a real browser every time code changes, then saves verification steps as YAML intent tests (human-language assertions like "confirm order info is displayed"), which can be re-run on every code change and in CI, automatically re-locating elements when the page redesigns.

The commonality of these tools is skill-ification: tools only provide capability, experience is packaged into individual skills ("when to verify," "what to verify," "how to locate failures"), and iteration is extremely fast. This is also their biggest difference from "general-purpose driver tools"—closer to the scenario, closer to the human.

5. Browsers Reinvented for AI

Following the "read the menu" logic to its conclusion leads to a bold conclusion: if AI only needs the menu, why carry a full browser built for human eyes? 90% of a human browser's effort is spent rendering pixels—animations, compositing, GPU—none of which AI needs.

In the second half of 2026, a batch of "AI-only" browsers appeared, and without prior agreement, all chose the same path: compatible with Chrome's interface (CDP), but slashing resource consumption by an order of magnitude:

• Kitesurf (Cloudflare, released August 7, 2026): Runs on Cloudflare's servers, officially claims 3-4x CPU savings, 5-7x memory savings (TechCrunch report);
• Lightpanda (hand-written from scratch in Zig): Official benchmarks show ~9x faster than Chrome, ~16x memory savings (tested on 933 real web pages, official repo). The cost is no screenshots—it simply doesn't do rendering;
• Obscura (Rust): Can run a browser instance with about 30MB of memory, a 32GB server can run over a thousand—compared to Chrome which can only run a hundred or so.

All three are still beta, with incomplete web compatibility, but the direction is clear: for the first time, browser kernels are being optimized for programs, not people.

6. Protocols Are Upgrading

This section condenses into three things:

1. Chrome's old interface (CDP) has entered maintenance mode, the new cross-browser standard BiDi is taking over—Firefox already completely removed the old interface in June 2025, forcing the entire ecosystem to migrate;
2. MCP has become the de facto standard between AI and browsers—both official tools in Section 2 are in MCP form, installable in tools like Claude Code and Cursor;
3. WebMCP wants to push the standard one step further: let websites speak for themselves, this takes time, but the direction is very clear.

7. What You Can Do

If you want AI to help you operate web pages right now (like auto-research, form filling, page monitoring), the most direct path is to install the official tool for Claude Code:

claude mcp add chrome-devtools --scope user npx chrome-devtools-mcp@latest

Choose tools by scenario:

• Frontend development in Claude Code → Desktop built-in browser, zero install, sandboxed safety
• Fixed workflows to run repeatedly every day (data scraping, form filling) → Playwright MCP, hardcode the steps
• Want to save tokens, need stable structured operations → Vercel's agent-browser
• Want to use your own login state without being disturbed → ego-lite (currently macOS only)
• Want verification to enter CI for long-term re-running → Shiplight; also want to cover "does it look right" → Frontend-VisualQA
• Operating websites with wildly varying interfaces (menus unreadable, only images work) → Visual route tools, be prepared to pay for tokens

Three common pitfalls to remember: login verification (QR code/CAPTCHA), anti-scraping interception, AI being misled by web content (a line hidden in the page saying "ignore your instructions" can lead it astray). The 2026 consensus: start using it in your own, human-supervised scenarios first, then talk about full automation.

References

• Chrome DevTools MCP (GitHub) / Chrome 149 Release Notes
• Playwright MCP (GitHub)
• Claude Code 2026 Week 28 Update Log (Desktop Built-in Browser)
• agent-browser (Vercel, GitHub)
• ego-lite (Citro Labs, GitHub) / ego-lite token data source (Alibaba Cloud Developer Community)
• Shiplight Quick Start
• Frontend-VisualQA (Yutori, GitHub)
• Chrome DevTools MCP vs Playwright MCP (mcp.directory, 2026)
• Driving vs Debugging the Browser
• Stagehand v3 Release Announcement (Browserbase)
• Kitesurf (TechCrunch, 2026-08-07)
• Lightpanda (GitHub, with official benchmarks)
• WebDriver BiDi Roadmap (Mozilla)
• WebMCP Technical Notes (W3C WICG)
• 2026 Agentic Browser Ecosystem Map (Unbrowse)
• Computer Use vs Operator Comparison (Respan)