Unattended Turns on Android: Only Two Kinds of Tools, and Better to Come Back Empty-Handed

Andee is an agent running on Android, with its brain on the device. It uses accessibility services to read elements and tap buttons to operate. Besides the two kinds of turns initiated by the user speaking and by an alarm going off, there is a third: after the conversation has been silent for a while, the device sends itself a (to yourself …) message, for it to review the conversation alone and write what should be kept into a local notebook.

Background autonomous turns are not a new concept. The tricky part is the way it fails — there is no user present, so there is no error popup, no "that's not right", and no one to press retry for you. The first thing 0.5.1 fixed was here: the review died together with the process, it never came back after restart, and when it did recover it would read "0 items to review" and then do nothing. I split this round into three independent problems: the permission surface, retry semantics, and the bill. Each has its own approach.

Conclusion first: this round's usable tool set converged into a single if, empty results are defined as a legitimate completion, and the daily budget only counts the half that can be turned off. Below are the evidence, implementation, and costs.

1. Trigger: Four Invariants, Not a Timer

Scheduler.armSweepIn(context, delayMs) is the only trigger entry, setAndAllowWhileIdle, one-shot, default 20 minutes (configurable 5–240). Around this entry there are four invariants:

Invariant	Implementation
Won't trigger while continuously speaking	Re-arm at the end of each turn, last write wins
Even when time is up, may not need to run	onSweepTimer rechecks now − lastActivityAt ≥ quiet, if not, arm the remainder
Can't run infinitely in a day	sweepAllowed(cap), default 12 times/day (configurable 0–100, 0 = off)
Process death must not leak	At startup, catchUpSweep compares lastSweepAt with lastActivityAt to distinguish "already reviewed" from "died before reviewing"

The fourth row is the only piece with real design weight here. lastSweepAt is only written by Notebook.noteSweep, which is only called when a review round actually finishes. So this field is not "last attempt", it's "last success". Compared with the last activity time, the two cases separate: already reviewed (do nothing), and died half-way (run one make-up round). Without this field, a post-restart review would treat itself as never having run, and would forever re-run the same conversation.

Also, this is not an exact alarm. The code comment makes it clear: checking one quiet period isn't worth applying for an exact alarm; the punctuality that permission buys has to be paid for with a dialog that must be explained to the user. So this round may arrive late.

2. Permission Surface: One If, With Two Exceptions

The biggest risk of an autonomous turn is not that it thinks wrong, but that it wants to do something. The constraint is placed at the scheduling entry, using booleans passed down per turn type, and doesn't rely on the model cooperating:

// core/.../net/TurnGuards.kt:39
if (silent && !method.startsWith("note.") && method !in GEAR_SILENT_OK) {
    throw IllegalStateException(
        "A silent turn may only use the notebook (note.*); it must not touch the device: $method",
    )
}

private val GEAR_SILENT_OK = setOf("gear.note", "gear.list")

Three decisions worth mentioning:

First, why a whitelist with prefix, not a blacklist. A blacklist is necessarily wrong here — the tool table changes with every version, and no one will remember to add a new screen.* or app.* entry to it. The prefix note. means "only read and write local notes", and its semantics are closed; the whitelist has only two entries, adding one requires changing code, and changing code requires a review.

Second, why the gate is in the dispatcher, not in the schema. The schema is static; one definition serves local turns, hub turns, and unattended turns alike, and it can't tell who is calling at the moment. The same reasoning appeared once in noteFold: folding is only allowed for review turns, a normal turn calling it gets a refusal with a reason — the decision is in the dispatcher, because the prompt is only a request.

Third, why "read" stays in the whitelist. Giving only gear.note and not gear.list means whatever is written can't be read back next time, effectively re-reading the screen every time; giving read admits he has a list of "what's installed on this machine". I chose to give read. This decision wasn't made on safety grounds, but on usability; I'm writing it here because it's an exception, and exceptions should have names.

3. Empty Results: A Valid Completion, Not a Failure

The sentence in the prompt is the entire basis for this design: if nothing new was produced, don't record it; ending empty-handed is allowed; recording for the sake of recording only dirties the notebook.

The code has to cooperate, otherwise that sentence is empty talk:

• There is no branch that does "zero output → re-run". A review is done when it's done, and onSweepSpent still counts the run.
• If nothing is written, folded, or recorded, the turn still ends, and history only keeps the monologue itself, no more than 900 characters.
• If indeed nothing remains, the only change on the user's side is that the counter in settings goes up by one.

Conversely, the pitfall this section stepped on is exactly proof of how dangerous "retry" is. The one harvest review of the same name that failed when context was nearly full appended one of its scaffolding messages to history on each attempt; and this time's anti-duplicate criterion was "is history length the same as last trigger?" — each failure changed the length, so the criterion was never true. On-device it logged 218 attempts, about 5 seconds apart, until the app was reinstalled. The fix was to delete that failed round's scaffolding (dropUnfinishedHarvest) so the length goes back to where it was.

So the order is opposite to intuition: first define "doing nothing" as a valid result, then talk about anti-duplication. A system that treats empty results as failures naturally grows retry loops, and in unattended turns those loops are invisible.

4. The Bill: Only Counting the Half That Can Be Turned Off

There are two kinds of unattended turns: sweep, triggered by silence, and harvest, forced when context is nearly full. Both cost money, but only the former is counted against the user's daily budget:

Only TurnKind.SWEEP is reported: a harvest is not optional, so counting it against a daily budget the user set for reviews would be misleading.

This means the meaning of a budget is "how many times I'm willing to let it run by itself", not "how much money I'm willing to spend". If the non-optional half were counted in, the number 12 would lose explanatory power — a user hitting 12 wouldn't know whether they were too diligent or their context had ballooned.

The cost is an incomplete bill: the "today" line in settings shows counts and tokens, but only for sweep. I accept this, because the alternative is to let a variable the user can't change consume a budget the user can change.

5. Where It Doesn't Work Right Now: The List of Costs

• Only the local brain has this round. startBrain returns early in the hub branch, so the alarm is never armed. To use this feature, the model address must be filled in locally.
• Fewer than 6 messages won't run. Short conversations never enter review.
• Maximum 3 topics folded at once, and folding is irreversible on the model side. So all gates are written as refusals with reasons, not "retry on failure" — when a review is refused without a reason, the next time it slams the same input into the same wall.
• Empty-handed still costs money. No speech, no subtitles, no chat history, but the money is spent and you don't see the process.
• It can be late. It's not an exact alarm; in Doze it can be delayed by over ten minutes.
• It reads the conversation in memory. If the process dies, this round has to wait for the next startup to rebuild up to 40 messages from the disk record.

6. If You Want to Add an Unattended Turn to Your Own Project

Four points, in order of importance:

1. Define the permission surface first, then write the prompt. An unattended turn's tool set should be a prefix plus a whitelist, and that decision must live in the dispatcher, not in the schema — the schema doesn't know who is calling.
2. Write "empty result" into the contract. Without this, the system grows its own retries; retries in unattended turns show up as bills and battery, not errors.
3. Separate the "attempt" timestamp from the "success" timestamp. With only one timestamp, crash recovery either re-runs what was already done or never makes up what wasn't.
4. Only budget the half that can be turned off. If the non-optional part counts into the same quota, the quota number becomes meaningless to anyone.

The project is now at 0.5.3. The above numbers — 20 minutes, 12 times, 6 messages, 3 topics — will change; what shouldn't change is the position of the three things: the trigger in time, permissions in the dispatcher, and output allowed to be zero.

Repository: https://github.com/kuafuai/andee