Self-Hosted Flight Fare Monitor Pulls Real Data From 6 Platforms and 10 Channels
The project is a working template for aggregating many scraped sources behind one schema while being honest about reliability gaps. The same channel-matrix and three-way alignment pattern applies to any monitoring task where sources disagree, sessions expire, and you need forensic evidence of what the scraper actually saw.
Self-hosted fare monitoring has a new reference implementation: ticket-monitoring aggregates 6 Chinese booking platforms (Ctrip, Qunar, Fliggy, Tongcheng, Tuniu, Zhixing) through 5 browser and 5 App channels, normalizing flight price, number, times, stops, duration, terminal, cabin, aircraft, punctuality, and baggage into a unified schema stored in SQLite. A console page shows side-by-side comparisons of the same flight across sources, a price calendar and K-line view, and alert channels for DingTalk, Windows notifications, ntfy, Bark, and Alibaba Cloud calls or SMS.
The project grew out of a concrete annoyance: comparing a direct flight with a connecting one on a Shanghai-to-Tokyo route showed ¥975 direct vs ¥1414 connecting, but the direct flight had no checked baggage while the connecting one included 23kg, so the headline difference was misleading. Worse, calendar prices, detail prices, and per-source data disagreed. The tool keeps those differences visible and excludes sparse calendar rows from alert logic.
Technically, each browser channel picks its most stable path (H5, SSR, or HTTP), App channels run on a MuMu Android emulator via adb and uiautomator2/OCR for canvas pages, and login states are snapshotted and restored. A 10-by-7 channel capability matrix labels each cell as supported, unsupported, or unverified, and unverified is never treated as supported. Deployment runs from a prebuilt binary, source, or Docker with built-in Chromium and Chinese fonts.
Keeping raw and parsed rows with three-way alignment turns the scraper into a forensic tool; when a source page changes, you can see exactly where parsing diverged from the mirror.
The honest three-state capability matrix (supported/unsupported/unverified) is a rare counterweight to the usual claim-everything scraper readme; it sets expectations that unknown endpoints are not confirmed working.
Treating calendar prices and detailed flight rows as separate data classes prevents false alerts from sparse data, a design choice many aggregators skip.