Running Claude Code at Home, Watching from My Phone: Pits I Fell Into Building an Open Source Tool

Tags: AI, Rust, AI Programming, Open Source Category: AI

Why

The most annoying thing about running larger tasks with Claude Code, Codex, is not that it's slow, but that it stops halfway and asks you a question: Should I execute this command? Choose plan A or B? If you're not at the computer, it just waits. You go out for a meal and come back, only to find it's been stuck there for twenty minutes.

SSH + tmux + networking can solve that, but typing SSH on your phone is painful: there's no Esc key, Ctrl is awkward, and Claude Code's interface jumps all over the screen when it redraws.

So I made Looklook: a background service installed on your own computer that puts the terminal and AI sessions on that computer into a web page. Open it in a mobile browser to see progress, reply with one line, or paste a screenshot. The client is fully open source (AGPL-3.0) with end-to-end encryption and all data stored locally: https://github.com/imythu/looklook

First, how to use it

curl -fsSL https://looklook.dev/install.sh | sh

After installation, open http://127.0.0.1:1234 on your local machine, no registration needed. Enable LAN access in "Access Control" and a phone on the same Wi-Fi can open it too. There are desktop versions for Windows and macOS.

It works at the terminal layer: Claude Code, Codex, Gemini CLI, OpenCode, Kimi, Qwen Code, or htop, vim, anything that runs in a terminal. It doesn't touch AI assistant logins; subscriptions, API keys, domestic Coding Plans all work as usual.

Things you can do on your phone:

• Full terminal with button bar for Esc / Tab / Ctrl / arrow keys
• Claude Code and Codex output organized into conversation cards, with buttons to tap when confirmation is needed
• When the AI stops to wait for you, push notifications to Feishu, WeChat Work, DingTalk, Bark, ntfy
• Paste screenshots directly to send to the AI
• The terminal runs inside a built-in tmux; closing the webpage, disconnecting, restarting the app, the session stays alive

Now some interesting pitfalls I fell into.

Pit 1: ttyd's built-in frontend flashes when Claude Code redraws

The terminal web page initially used ttyd's own page directly. The problem is ttyd's upstream is still on xterm.js 5.5, which doesn't support synchronized output (DEC 2026). TUI apps like Claude Code and Codex do "clear screen + redraw" on every repaint. Without synchronized output, the browser draws intermediate states, so the screen flickers. DOM renderer is also slow on phones.

Finally I ported ttyd's frontend and upgraded to xterm.js 6:

• Enable WebGL rendering, fall back to DOM renderer automatically if the WebGL context is lost
• Enable terminal-features xterm*:sync in tmux config so synchronized output passes through tmux
• Also support OSC 9;4 progress bars and OSC 8 hyperlinks

Another small pit: xterm 6's .xterm-viewport background is hardcoded to #000, which leaves a black border around the terminal in light themes, so I use MutationObserver to sync the inner background color.

Pit 2: tmux swallows ZMODEM, so rz/sz doesn't work

Many people are used to transferring files with rz / sz. But once sessions go into tmux, ZMODEM's control character \x18 gets filtered by tmux and the transfer never starts.

Solution: skip ZMODEM. The frontend detects when rz / sz prompts appear in the terminal and pops up its own upload/download panel. Files go over HTTP to the terminal's current directory. Uploads are chunked by 8 MiB to avoid timeout and single-request size limits. Also bundled trzsz (trz / tsz), which is designed to pass through tmux.

Pit 3: Remote connection uses WebRTC, but libjuice doesn't support ICE restart

When accessing from outside, a WebRTC encrypted channel is created between the browser and your computer: direct connection when possible, otherwise TURN relay; the relay only sees ciphertext. The client uses libdatachannel's Rust bindings.

Phone networks are very unstable: switching Wi-Fi / 4G, locking screen, switching apps will drop the connection. Standard practice is ICE restart, but libjuice under libdatachannel doesn't support ICE restart (source code says "ICE restart is not supported").

So I changed approach, doing resume at the application layer:

• When connection drops, the client suspends the terminal session for 60 seconds, without closing it
• The browser re-establishes a new WebRTC connection with a session key and "how many bytes I've received"
• The client sends the missing output from that position, and to the user it just looks like a brief lag

Two more network pitfalls:

• Some networks block all UDP, so WebRTC can't connect; I started a local UDP-to-TCP forwarder and use TURN over TCP
• libjuice's ICE role conflict handling has a bug that prevents browser connections through TURN/TCP; I patched it

Pit 4: Turning AI's TUI into conversation cards

Looking at the terminal on a phone is still tiring, so I made a conversation interface: it organizes Claude Code and Codex replies into cards, and gives you buttons when confirmation is needed.

The hard part is determining "what state the AI is in": working? waiting for input? popped a selection box? Initially I used rules like "no more than 4 lines below a bottom line", but newer Claude Code versions list background agents below the input box, so with 3 or more agents it misjudged.

Now:

• Only recognize the input box shape: top line, ❯-prefixed input line, bottom line; ignore whatever below
• Don't guess when the UI is unreadable: if the screen stays unchanged for 10 seconds, only give a light hint, not a "waiting for your choice" push
• Codex's history may also contain › 1. …; only recognize the last line to avoid false selection boxes

There's also a timing pit: Codex's app-server writes the user message into the record about 10 seconds after receiving it (waiting for MCP startup etc.). So after you send a message, the interface shows nothing for 10 seconds, making you think it wasn't sent. Finally the frontend displays the message first, and replaces it when the actual record arrives.

The client code is all in the repo, and the install packages are built by GitHub Actions in the repo with SHA256 checksums.

Finally

This project is basically written by me plus Claude Code: over the month since open-sourcing, ninety-plus versions have been released, and many features were added the same day users requested them in the group.

If you also often let AI run tasks but you have to go out, feel free to try it. File issues directly, or join the QQ group 1029871265 for discussion. If you find it useful, give it a star; that's the main way for others to discover it.

GitHub: https://github.com/imythu/looklook