A Private AI Wrong-Question Notebook Closes the Loop from Photo to Practice Paper
The engineering around the model, not the model itself, determines whether an AI tool survives daily use. The quality chain, human-in-the-loop review, and data-as-files approach in this project transfer directly to any AI application handling messy real-world input; the token-saving shift to on-demand vector graphics is a concrete lesson for cost-sensitive pipelines.
A private AI system for a middle-schooler's math errors turns a photo of a worksheet into a structured, human-verified question bank. The workflow runs: photograph multiple questions on a page, a vision model transcribes them into LaTeX (handwriting is only marked, not transcribed), then a human reviews each card against the original image, selects which to store, tags the error cause (careless, doesn't know, other), and assigns a knowledge point from a textbook-derived tree. After submission, the question goes through a quality chain: dedup (stem normalization, ≥92% similarity means duplicate), independent solving by a second model to verify the printed answer, and chapter/difficulty/knowledge-point analysis. The bank feeds two practice modes: a single-question loop that auto-grades and recommends archiving after 3 consecutive correct answers, and a full-paper mode that batch-grades a selection and aggregates wrong-question knowledge points.
Three design decisions stand out. Data lives as human-readable JSON files with atomic writes and git versioning, which turned out to be a life-saver when a test accident overwrote the question bank. SVG vector graphics generation, initially the default for geometry, became on-demand because it consumed the most tokens and time (1–3 LLM calls per question plus retries); photo crops are now the default at 0 tokens. And human review is treated as the product's core, not a transitional step, because photo transcription is too unreliable for full automation. The builder went through 12 iterations driven by real usage over 3 days of initial development, and the system is now used daily at home.
Human review stays the product's core because photo transcription is too unreliable for full automation. The 80/20 split of AI grunt work to human judgment is a realistic division for many consumer AI tools.
Storing the question bank as versioned files rather than SQLite turned an accident into a recovery story. For personal or small-scale data, git becomes an audit trail and rollback mechanism that a database doesn't naturally offer.
Switching SVG generation from default to on-demand shows that 'smart' processing should be opt-in when it costs tokens and latency. The measured 645s→16s improvement after turning off DeepSeek's thinking chain is the same principle applied to reasoning.
The three-consecutive-correct archive rule implicitly builds spaced repetition, and periodic review reminders—planned as the next step—would complete that loop on top of the existing practice modes.