Zero-Loss Offline Continuity: A Modbus-MQTT Gateway Backfills 86,400 Records After a 24-Hour Outage
For facilities where RS485 meters sit behind flaky 4G links, this design closes the data-loss gap that turns monthly energy reconciliation into guesswork, and it scales to thousands of devices without blocking collection.
On an RS485 bus in a Pearl River Delta industrial park, nearly 2,000 three-phase rail meters feed an edge gateway that publishes MQTT to the cloud for energy analysis. Metal workshops and underground cable trenches degrade 4G signal, so the gateway-to-cloud link drops several times a week. The first gateway version had no local cache: a network outage meant the collection thread simply discarded the current round of data, leaving a hole in the cloud curve. Monthly reconciliation then had to estimate those missing kilowatt-hours, which finance wouldn't accept.
The fix splits collection speed from storage stability into two independent paths. Netty 4.1.94's async event loop runs non-blocking Modbus requests: it sends writes and handles responses asynchronously, cutting per-device latency from 320ms to 45ms and dropping CPU usage from 94% to 18% at full load. Any failed write or MQTT publish goes straight to a local SQLite 3.42.0 database in WAL mode, which writes at roughly 3,000 TPS and survives sudden power loss via periodic checkpoints. A backfill task polls the cache every 5 seconds for batches of 500 records and publishes with QoS1; a sequenceId embedded in each payload lets the cloud de-duplicate with Redis SETNX for 24 hours.
The field test simulated a 24-hour outage with one record per second, 86,400 total. The gateway backfilled all of them in about 142 seconds, no duplicate and no gap on the cloud. The design fits RS485 buses with many meters and intermittent network; it's overkill for loss-tolerant public network direct connections.
The design separates 'collect fast' from 'store stable' into independent paths, letting the gateway fail over to local persistence without slowing collection; that pattern generalizes to any asynchronous ingestion pipeline.
The performance swing from serial polling to event-driven collection (45ms vs 320ms latency, 18% vs 94% CPU) suggests many industrial Modbus gateways are still bottlenecked by blocking I/O, not by the fieldbus itself.
The cache-backfill approach deliberately targets intermittent connectivity, not loss-tolerant public networks; teams needing zero-loss accounting in variable environments get a concrete reference, while those with stable links can skip the complexity.