ARTEMIS: Google's Open-Source Agent That Operates Android Phones Like a Human
This gives Western developers a working alternative to scripted mobile UI automation: describe a task in natural language and let the agent handle the taps, swipes, and verification on real hardware. The MCP integration means it drops directly into existing AI coding workflows, and the self-drawn UI fallback covers the blind spots that have long frustrated uiautomator-based testing.
ARTEMIS is Google's open-source AI agent that automates real Android devices from a single plain-language instruction. It finds UI controls through a three-layer strategy—accessibility hierarchy, coordinates, and OCR plus vision for self-drawn pages like Compose or Flutter—so it works even on interfaces that traditionally break scripted locators. The first run installs a local accessibility helper that never sends data off-device.
Two execution profiles handle different workloads: Flash (3–5 seconds per step) for deterministic tasks like installation or login, and Pro (15–40 seconds per step) which plans, verifies each action against the current UI tree and pixels, and does a final check against the original goal. History compression keeps long tasks from blowing up the context window. For developer workflows, a built-in MCP server connects to Claude Code and other AI IDEs with one command, and a Web console offers live screen casting with a task queue and replay.
Setup is two commands: clone and run `./start.sh`, which pulls ADB, scrcpy, FFmpeg, and Python dependencies. A Python SDK with Pydantic-typed results integrates into pytest and CI. Caveats: Android only for now, each step is a model call so token costs pile up, and apps without accessibility support fall back to less reliable coordinates and vision.
The benchmark number is officially self-scored: Google Research wrote the questions and Google submitted the answer, so treat the 99% as indicative rather than an independent third-party evaluation.
The three-layer control-discovery pattern is a practical answer to the long-standing pain point of self-drawn UIs, which have always been a blind spot for uiautomator-based tools.
The token cost per step makes plain-script regression suites the better choice for thousands of test cases; this agent is targeted at interactive and exploratory tasks, not bulk automation.