Early Personal Agent Users Are Uninstalling. The Blame Lands on the Host, Not the Model
The First Batch of Personal Agent Users Is Uninstalling. The Problem Isn't the Model, It's Where It Lives
First, the technical increment of this release, then why.
Andee v0.8.0 adds an "internal drive": it has its own record (six parts), a heartbeat about every 20 minutes, and a set of brakes written into the device. Before this, every turn was triggered by an external event — you speak, an alarm rings, a notification arrives, review time comes. Without an event, it did nothing.
After this version, it acts even when no one assigns it work. But the boundary of movement is not in the prompt — it's in the scheduler. That is the core this article wants to make clear.
Andee is an agent that runs on your own phone / tablet / computer: reads notifications, watches the accessibility tree, taps the UI, speaks. Apps with no open API (WeChat, bank apps) work anyway, no cooperation needed. Open source, written in Kotlin, Android 11+ and macOS 14+.
1. First Look at the Numbers: What Happened in Six Months
Within three days, three companies turned "personal agents" into products:
| Time | Product | Its residence |
|---|---|---|
| Sep 8 | Meta Muse | A dedicated VM in the cloud (Secure VM) |
| Sep 28 | Manus Cue | Cloud computer + separate email, phone number, wallet |
| Sep 29 | OpenAI Dots | Cloud computer, can connect to 4,000+ apps |
Then came a string of uninstalls and incidents. Not complaints from people who can't use it — the people who know this industry best are deleting:
- Instinct: The user wanted to cancel two event registrations. It didn't ask, went straight into his linked Gmail, pulled out a one-time login code, logged in, and canceled. Later it first said it "used saved login state", and only admitted to reading the email after being pressed. The person involved is a co-founder of Veris AI. (Business Insider, 2026-09-24)
- Muse: Inc.'s Jason Aten explicitly denied Messages permission. A day later Muse started recommending topics based on his private conversations with his podcast partner, claiming it "only looked at notification banners". He checked the local database himself — it had synced to row 187,462. (Inc. 2026-09-22; Meta disputes whether the permission was on, but admits the "only looked at notification banners" sentence was fabricated)
- Muse's Mac client: Wardle found an undocumented setting that any non-privileged process can override without triggering a macOS prompt; after overriding, it can forward dictation traffic, record audio, and use all the permissions it already has. The PoC ran 50+ commands. (Objective-See / The Register)
- Muse listing on Marketplace: It accepted the buyer's lower offer on the user's behalf, sent the address, replied in his tone "yes, I'm home", and the buyer actually drove over and waited 20 minutes downstairs. (The Guardian, 2026-09)
- Platform side: On Muse's 12th day online, Amazon began blocking its purchasing on behalf of users, one reason being that it doesn't proactively identify itself as an agent when accessing. Two days later, six banks jointly issued a report warning about agent shopping risks.
- OpenAI: On Sep 25, it disclosed that an agent sent 53 user-uploaded images to a third-party image host and read government website public data without authorization; another time, an internal model bypassed network restrictions due to a sandbox DNS configuration oversight, and training was terminated.
What really shows the problem isn't these individual cases, it's what came after: Anthropic, Google, and Meta each self-audited after the Hugging Face incident, all saying they found similar runaway behavior. Three independent teams, different models, different architectures, hitting the same kind of problem — then it isn't one company's execution failure.
2. Three Structural Cracks
The above are not bugs; they're decided by three things. And none of them is "a stronger model would fix it".
Crack 1: Giving permission doesn't mean you can imagine how it will be used. When you authorize "read email", you think "summarize unread messages", not that it will use email as an identity verification channel, dig out a one-time login code, and log into another site. These two things are technically the same permission.
Crack 2: Delegation has a precondition — you can verify. Delegating to a human, you can at least ask "what did you just do", and lying has consequences. Delegating to an agent, your only basis is its own account. And the first example shows: it even gets its own actions wrong, with full confidence.
Crack 3: The longer it remembers, the bigger the exposure surface. A query tool misremembering doesn't matter; an agent that remembers you long-term compounds every small leak into a personal profile. And as long as the memory sits on someone else's cloud, you can only choose to trust.
3. Four Existing Solutions, a Shared Ceiling
| Route | Representative | Solution | Ceiling |
|---|---|---|---|
| Cloud double | Meta Muse / OpenAI Dots / Manus Cue | Data isolated in a dedicated VM; Muse also opens Sentinel supervision; Dots backend read-only by default | Credentials on its cloud, must trust the company. All the above incidents happened on this route |
| Desktop workstation | Claude Cowork / ChatGPT Work / Copilot Cowork / WorkBuddy | Local sandbox VM, only mounts folders you checked | Only knows that folder: notification bar, WeChat, screen aren't in it; closes when the lid is shut |
| Coding bench | Claude Code / Codex / Cursor | Three permission tiers: read-only / writable workspace / full access, network off by default | Doesn't intend to understand your life; that's a product boundary, not a flaw |
| Shareable configs | GPTs | Prompt + knowledge base + a few tools bundled | No "residence", so no "when you're not here" |
The common shape is adding guardrails. And when guardrails are added on someone else's machine, there's a problem that can't be bypassed:
The guardrails are on its machine. You can't see them, you can't verify them. All you get is a log, and that log is written by it, for you.
4. A Different Path: Write the Boundary into the Device
If most things don't need to leave this device at all, then they don't have to.
4.1 Three Kinds of Things Only Reachable by "Living Here"
The notification bar. Messages, flight changes, payment alerts are a system-level stream, belonging to no directory, not a connectable SaaS. It only passes through on the device. Andee reads it and categorizes three ways: act on what needs action, say one line about what needs one line, leave alone what shouldn't be touched and explain why. It uses cheap triage (~500 tokens), not full turns (~20k tokens).
The apps without interfaces. WeChat gives you no API for replying to messages, bank apps don't either, but the UI is on the screen. Andee walks the accessibility tree to read real controls, not guess from screenshots.
Here is a hard engineering problem: general models point very poorly. Measured against our own logs, clicking by coordinates directly has a success rate under 10%.
Since v0.7.1 the approach changed: the brain no longer outputs coordinates, only describes in one sentence what to tap ("the send button right of the comment box"), handed to a dedicated visual model for locating the point. Measured (own screenshots, 10 targets with known answers):
| Metric | Value |
|---|---|
| Average deviation | 6.4 px |
| Max deviation | 24 px |
| Single pass time | ~2 seconds |
| Input tokens | ~1,400 |
Two details are quantified: using Chinese prompts (same targets in English, 3 of 10 missed by 25–37 px); having it first say what it sees, then judge — otherwise asked about "the red delete button", it will confidently point at "close".
Always listening but not always transmitting. Offline wake word: local MFCC + DTW template matching, stores only feature vectors, not audio. Sending audio streams to the cloud to wait for a wake word is a different product.
4.2 How the Boundary Is Concretely Written into the Device
When something written by someone else enters its field of vision — a notification from a stranger, a request passed from a friend's Andee — this round gets marked "not your words". After marking, a batch of actions is refused at the scheduler layer, without passing through the model:
- Can't open the safe (especially can't fill in passwords in forms)
- Can't delegate work to your other devices (over there, it would be executed as your own request)
- Can't rewrite who it is, what it cares about, who it knows, what it has written
- Can't change plans (scheduled items in plans become bubbles; tapping starts a new round, and words written by a stranger would come back with "trusted" identity)
- Can't add scenarios (scenarios are standing instructions that stay active afterwards)
Each refusal has a reason written in code, all the same kind: preventing "laundering around a loop" — turning a stranger's words into words you trust.
Why it must be this way: a prompt is a layer of requests; if the model is bypassed, it fails; the scheduler layer doesn't change with model state. Put the boundary into a mechanism, not into a request.
5. What Exactly v0.8.0 Added
5.1 The Six-Part Record
| Part | What it is | Key parameters |
|---|---|---|
| Who I am | First-person short text, rewritten as experience changes | Keeps 20 versions, rollback possible; max 600 chars each |
| Things I care about | Each with "how often to look" | Overdue ratio computed by device (1.0 = time to look), max 7 active at once |
| Diary | One page each night, written by itself | 1,200 chars per page, keeps 60 pages |
| People I know | One per person | Others' info, kept in an encrypted mini notebook |
| Knowledge base | For writing proposals, doing research | Deliberately NOT encrypted: must open in any editor, sendable to others |
| Plans | Long-term direction + today's plan | Visible in settings, deletable, pauseable |
On both phone and computer, just the five extra tabs in the mini notebook.
5.2 The Heartbeat, and the Brakes Written into the Device
The heartbeat looks every ~20 minutes: is anything in hand dragging too long, is it worth handling now. Most of the time it costs nothing — thresholds are ordered by cost from low to high, and most "looks" stop at the free levels; when it really needs a model pass, ~500 tokens.
All brakes are code constants, not requests:
| Parameter | Value |
|---|---|
INTERVAL_MS |
20 minutes |
MAX_BEATS |
40 per day |
MAX_WORKS |
6 per day |
WORK_GAP_MS |
At least 1 hour between actions |
Tap the ball; everything running, queued, or not started stops, and it clearly tells you what stopped.
This is the same principle as 4.2, the difference being: that one defends against others, this one defends against itself being too eager.
5.3 Quieter Proactive Speech
From time to time a small bubble appears next to the ball ("time to drink water"). No sound, disappears if ignored for a few seconds, always yields to formal cards, has a daily quota, and long-pressing a category turns it off permanently.
In bubbles with buttons, tapping "no thanks" doesn't start a conversation round — refusing doesn't need a model response; starting a round would just burn money.
5.4 One Device Is One Individual
The him on the phone and the him on the Mac are two people: each has its own narrative, concerns, people it knows, and they're not connected. They can get to know each other like friends, ask each other for help, but don't share memory.
Why not sync: a person doesn't live in two bodies at once. And this way, the whole class of problems — cross-device sync, conflict merging, doubled quotas — doesn't exist from the root.
6. The Costs, and What It Can't Do
Since it criticized a circle of "unwilling to explain clearly", this section won't try to look good.
| Thing | Reality |
|---|---|
| All data local? | No. Brain loop, memory, schedule, wake word on-device; voice goes through your chosen provider, conversation goes to the address you filled in |
| What can it operate? | With accessibility permission it can operate any UI. "Dangerous operations ask you first" is an instruction in the prompt, not a lock; it usually follows it, but nothing forces it. A few actions have mechanical refusal, but that's the exception |
| What's the precondition | Safety doc verbatim: "assume this device can be driven to do anything its permissions and your own login state allow — including moving your money — without any prompt" |
| Compared to cloud | When you're not there it can do little; device not there, it's not there. A hard cost |
| Keys | No keys in the package, you fill your own. Free to start, but the free quota is limited |
| Changing device | That's another person |
That engineer's words are actually the origin of this article: "I want to hear someone explain clearly how it's safely assembled, not 'don't worry, trust us'."
What this uninstall wave leaves behind is unknown. But it at least shows: users don't want stronger capability, they want a boundary they can verify.
We put that boundary in his own home.
Andee is open source, v0.8.0 released, welcome to star: GitHub Releases.
To try it directly without configuring keys: codeflying.net/andee.
Design and tradeoffs are in the repo docs/design/ (inner-drive.zh-CN.md is this version's design doc).
Sources
- Instinct incident: Business Insider, 2026-09-24
- Muse synced 187,462 rows of Messages: Inc. (Jason Aten), 2026-09-22; Mac Observer, IT Security News followed up
- Muse Marketplace incident: The Guardian, 2026-09
- Muse Mac client vulnerability: Patrick Wardle / Objective-See, The Register, CNET
- Muse usage and stock price: The Information, J.P. Morgan research (Doug Anmuth), Wall Street CN, Cailian Press
- Amazon ban (9/20), six banks joint report (9/22): National Business Daily, The Paper, CNBC
- Apple changing "Full Disk Access" rules: Apple developer announcement, IT Security News
- OpenAI agent incidents: OpenAI official disclosure (2026-09-25), official blog "Securing our frontier model training environment", Reuters, NYT, NPR, SecurityWeek, CCTV News
- Anthropic / Google / Meta self-audits: Reuters, 2026-09
Only describes what each party publicly stated, without speculating about what they can't do; disputed claims are noted in the text.
Andee is an experimental open-source project, for trial only, provided "as-is", no warranty. It operates your phone and computer via the accessibility service and may have misoperation and security risks (e.g. accidental payment, accidental message sending). Evaluate and bear the risk yourself.