跪拜 Guibai
← Back to the summary

How Real-Time Bus Arrival Works: GPS Drift, ETA Accuracy, and 100K QPS Reads

The phrase "2 stops, about 5 minutes" on a bus app - how is it actually implemented?

This is also a high-frequency scenario design question for backend interviews at big companies: GPS drift, message out-of-order, hot read amplification, inaccurate ETA - every one is a pitfall. This article walks through requirements alignment, architecture design, data model, and core code. You might want to bookmark it before reading.

1. Don't rush to draw the architecture diagram - align requirements and scale first

The first mistake people make with scenario design questions is jumping straight to an architecture diagram. The first step of an experienced developer is always: clarify requirements and estimate scale.

1.1 Functional boundaries: what to do, and more importantly what NOT to do

The system needs to provide:

The display looks like this:

Route 302 going to the railway station
YueB12345: 2 stops away, about 5 minutes
YueB67890: 4 stops away, about 11 minutes

Equally important is explicitly defining what not to do - actively constraining the boundaries shows you have thought it through:

1.2 Key metrics: all design revolves around numbers

Metric Target Description
GPS reporting frequency Once every 3-10 seconds Determined by onboard device capability
End-to-end latency User-visible < 3-5 seconds Experience red line
Query response P99 < 100ms Read path must be extremely lightweight
ETA error Peak < 1-2 minutes Algorithm determines experience
Availability 99.9% It's allowed to be inaccurate, but not unavailable

1.3 Capacity estimation: design without QPS numbers is irresponsible

Different scale assumptions lead to entirely different designs. Align the magnitude first, then discuss the solution. Three common tiers:

Scale Number of vehicles Write TPS Read QPS Read/Write ratio
Single city, small 10,000 ≈3300, burst 10,000 peak 100,000 ≈30:1
First-tier city 30,000 3000-6000 peak 10,000+ ≈10:1
National 200,000-1,000,000 peak 50,000-100,000 peak 100,000-1,000,000 -

Estimation process for a first-tier city:

At a national scale with millions of vehicles, peak writes around 330,000 records/second. Ten Kafka partitions plus Flink with hundreds of parallel tasks can handle it; the approach remains the same.

One-line characterization: writes are high-frequency small packets, reads are bursty hot streams, the whole chain allows eventual consistency, not strong consistency. All subsequent design revolves around this characteristic.

Remember: quantify first, then design.

2. Overall architecture: one main thread through everything

With clear requirements, the architecture is actually straightforward. First, the panoramic view:

image.png

Data flow in one line:

GPS report → MQTT access → Kafka to absorb spikes → Flink corrects and computes ETA → Redis hot data → API instant response + WebSocket push

2.1 Layered responsibilities, each minding its own business

2.2 Technology selection list

Layer Technology Reason
Gateway Nginx + Spring Cloud Gateway High performance, supports rate limiting/circuit breaking, horizontal scaling
IoT access MQTT + EMQX IoT standard protocol, supports millions of long connections, reconnection
Message queue Kafka High throughput, persistent, suitable for massive GPS reports
Real-time computation Flink Native stream processing, Exactly-Once semantics, strong windowing
Hot cache Redis Cluster Millisecond response, native GEO and ZSET support
Historical storage TDengine / ClickHouse / HBase Time-series write, high compression, analytical queries
Relational storage MySQL Line, stop, vehicle base data
Search Elasticsearch Fuzzy stop search, geolocation search
Real-time push WebSocket + Netty Long connections, proactive arrival reminders

2.3 Core flow sequence diagram

image.png

Drawing the architecture diagram is fun, but honestly, anyone can draw such a "block diagram". What really separates you is the detail in each following chapter - how to clean GPS dirty data, how to make ETA stable, how to handle hot queries. Let's break it down one by one.

3. Data model: one dist_from_start field decides everything

3.1 First, correct a misconception: this is not a "nearby vehicles" problem

Many people, when they see "real-time bus", immediately think of Redis GEO to find nearby vehicles - one GEORADIUS and done, right?

Wrong. That's the biggest misunderstanding in this problem.

"People nearby" or "delivery riders" ask for straight-line distance, but buses run along fixed routes. Users care about "how far along the route to the stop". A bus could be 200 meters away in straight-line distance, but with a river in between, the actual detour is 2 kilometers. Straight-line distance is meaningless.

Technical highlight: the core of this system is not Redis GEO, but a "distance along route model". This is the first highlight you can raise in an interview.

3.2 MySQL base tables

-- Route table (low-frequency changes, can be fully cached)
CREATE TABLE bus_line (
    line_id BIGINT PRIMARY KEY,
    line_name VARCHAR(50),
    start_station VARCHAR(50),
    end_station VARCHAR(50)
);

-- Stop table
CREATE TABLE bus_station (
    station_id BIGINT PRIMARY KEY,
    station_name VARCHAR(50),
    longitude DECIMAL(10,7),
    latitude DECIMAL(10,7),
    geohash VARCHAR(12),
    line_ids VARCHAR(255)
);

-- Route-stop relationship table (core)
CREATE TABLE bus_route_station (
    line_id         VARCHAR(32),
    direction       TINYINT COMMENT '0 outbound 1 inbound',
    station_seq     INT COMMENT 'stop sequence',
    station_id      VARCHAR(32),
    station_name    VARCHAR(64),
    lon             DECIMAL(10,7),
    lat             DECIMAL(10,7),
    dist_from_start INT COMMENT 'distance from route start in meters',
    PRIMARY KEY (line_id, direction, station_seq)
);

-- Vehicle location history table (fallback; real-time data goes to Redis)
CREATE TABLE bus_location (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    vehicle_id VARCHAR(20),
    line_id BIGINT,
    longitude DECIMAL(10,7),
    latitude DECIMAL(10,7),
    speed INT,
    direction INT COMMENT '0 outbound 1 inbound',
    report_time DATETIME,
    INDEX idx_vehicle_line (vehicle_id, line_id)
);

The dist_from_start field in bus_route_station is the soul. Example:

Route 302 outbound:
Stop A: 0m
Stop B: 800m
Stop C: 1500m
Stop D: 2600m

The vehicle is projected onto the route at 1300m from start. A user queries stop C (1500m):

Remaining distance = 1500 - 1300 = 200m

One subtraction gives the foundation for both "how many stops away" and "how many minutes".

3.3 Redis data structure design

Almost all real-time data lives in Redis. Key design directly determines query efficiency:

Data Structure Key example Description
Vehicle real-time state Hash bus:live:{vehicleId} Longitude, latitude, speed, direction, current stop sequence; TTL 60s-10min; expiration means "lost"
Line vehicle progress ZSET line:{lineId}:{direction}:vehicles score = distance along route or stop sequence; one command gets all vehicles that haven't arrived
Line vehicle coordinates GEO line:{lineId}:geo For drawing vehicle positions on the map
Stop coordinates GEO bus:stations:geo Nearby stop queries
Arrival prediction results ZSET route:{lineId}:arrivals score = predicted arrival timestamp; naturally supports "next few buses" sorting
Stop arrival cache String/Hash arr:{lineId}:{stopId} Precomputed results, short TTL 3-10 seconds

Two design highlights here:

Highlight 1: ZSET score = vehicle distance along route. When querying vehicles arriving at a stop, use ZREVRANGEBYSCORE to pull all "not yet passed" vehicles sorted by distance. Time complexity O(logN+M), no full scan.

Highlight 2: Hash tag under Redis Cluster. Put keys like bus:{line:302:1}:* into the same slot so Lua scripts can operate on them together. Under Cluster mode, cross-slot Lua access causes an error - many people step on this landmine.

Two common ZSET score designs, each with its own use case:

Score design Advantage Scenario
Distance along route (meters) Precise remaining distance Pairs with Lua for batch queries, precise ETA
Current stop sequence currentSeq Quickly determines "how many stops away" Filter vehicles by stop range

The vehicle real-time state in Redis looks like:

{
  "vehicleId": "YueB12345",
  "lineId": "302",
  "direction": 1,
  "lon": 113.95,
  "lat": 22.54,
  "distFromStart": 1300,
  "speed": 8.5,
  "ts": 1730000000000
}

With the data model ready, let's look at the write path - raw GPS data is dirty, and cleaning it is as involved as it is important.

4. Write path: GPS data cleaning, correction, and idempotency

Raw GPS data is very dirty. Computing ETA directly without cleaning produces garbage like "the bus teleported 500 meters". This chapter covers four things: wild point filtering, deduplication, coordinate correction, and out-of-order idempotency.

4.1 Report ingestion: don't write to DB directly, send to Kafka first

Onboard devices report via MQTT / HTTP. The access layer does basic validation and immediately sends to Kafka, returning instantly:

@PostMapping("/location")
public Result<Void> reportLocation(@RequestBody BusLocationDTO dto) {
    if (dto.getVehicleId() == null || dto.getLineId() == null) {
        return Result.error("Invalid parameters");
    }
    kafkaTemplate.send("bus-location-topic", JSON.toJSONString(dto));
    return Result.success();
}

Why not write to DB directly? With 10,000 vehicles reporting every 3 seconds, that's 3300+ TPS, and bursts multiply by 3. Writing to DB directly would blow it up. Kafka absorbs the spike; downstream consumes in batch. The reporting API always returns in milliseconds. This is the standard asynchronous decoupling.

4.2 Kafka partition strategy: ordering is the first line of defense

Two common partition key choices, a design tradeoff:

Partition key Effect Scenario
vehicleId Messages for the same vehicle are absolutely ordered Most common, consumers process per vehicle
lineId Data for the same line is grouped Flink aggregates per line

Ordering messages for the same vehicle is a hard requirement (out-of-order means old points overwriting new ones). Hashing by vehicleId is the most stable; if the computation layer aggregates by line, lineId also works, with consumer-side timestamp validation as a fallback.

4.3 Wild point filtering + coordinate correction

GPS drift is the most damaging data quality problem in this scenario: tunnels, elevated roads, building canyons - the position can jump across the river. Wild point judgment usually uses two hard rules:

@Component
public class GpsCleanProcessor {

    private static final double MAX_SPEED_KMH = 120.0;
    private static final double MAX_JUMP_METERS = 500.0;

    @KafkaListener(topics = "gps_raw", groupId = "clean-group")
    public void onMessage(GpsReport report, Acknowledgment ack) {
        GpsReport last = RedisUtil.get("gps:last:" + report.getBusId());

        if (last != null && isWildPoint(last, report)) {
            metrics.increment("gps.wild.point");
            ack.acknowledge();
            return; // Discard wild point, prevent ETA pollution
        }

        // Coordinate system correction: domestic maps use GCJ-02, GPS chips output WGS84
        double[] wgs84 = CoordTransform.gcj02ToWgs84(report.getLng(), report.getLat());
        report.setLng(wgs84[0]).setLat(wgs84[1]);

        kafkaTemplate.send("gps_clean", report.getBusId(), report);
        RedisUtil.setex("gps:last:" + report.getBusId(), 300, report);
        ack.acknowledge();
    }

    private boolean isWildPoint(GpsReport last, GpsReport cur) {
        long dtSec = Math.max(1, (cur.getTs() - last.getTs()) / 1000);
        double dist = GeoUtils.distance(last.getLng(), last.getLat(),
                                        cur.getLng(), cur.getLat());
        double speedKmh = dist / dtSec * 3.6;
        return speedKmh > MAX_SPEED_KMH
                || (dist > MAX_JUMP_METERS && dtSec < 5);
    }
}

Speed jump > 120km/h, or a jump > 500m within 5 seconds, marks a wild point. Buses can't fly; physics helps us filter dirty data. Before and after cleaning: wild point rate drops from 3% to 0.1%.

Lesson learned: coordinate system correction is mandatory. Domestic map services use GCJ-02, while GPS chips output WGS84 - the difference can be hundreds of meters. I once stepped on this mine; the whole fleet appeared "shifted by one street", and it took ages to discover it was a coordinate system issue.

4.4 Deduplication and idempotency: preventing data from "going back in time"

Duplicate reports and out-of-order messages are the norm. Three layers of defense:

Layer 1: Consumer-side idempotent deduplication. Use Redis SETNX as a dedup key; duplicate reports within a short window are ignored:

@KafkaListener(topics = "bus-gps", concurrency = "8")
public void onGps(GpsMsg msg) {
    // 1. Idempotency, prevent duplicate reports
    String dedupKey = "gps:dedup:" + msg.vehicleId() + ":" + msg.ts();
    Boolean first = redis.opsForValue()
            .setIfAbsent(dedupKey, "1", Duration.ofMinutes(2));
    if (!Boolean.TRUE.equals(first)) return;

    // 2. Map matching: project GPS onto the route, compute current station sequence
    VehicleState state = mapMatcher.match(msg);

    // 3. Update latest state, expire in 10 minutes to avoid zombie vehicles
    String stateKey = "bus:state:" + msg.vehicleId();
    redis.opsForHash().putAll(stateKey, state.toMap());
    redis.expire(stateKey, Duration.ofMinutes(10));

    // 4. ZSET for same line and direction, for "which vehicles haven't arrived"
    String zKey = "line:" + state.lineId() + ":" + state.direction() + ":vehicles";
    redis.opsForZSet().add(zKey, msg.vehicleId(), state.currentSeq());
}

Layer 2: Expired data is discarded directly. If the message timestamp is more than 30 seconds behind current time, drop it. Real-time systems only care about "now"; late historical data has no value for real-time state and pollutes it.

Layer 3: ts version number optimistic lock (important). On weak 4G networks, retransmissions cause "new packet first, old packet later" - vehicles appear to move backward. Use the timestamp as a version number with atomic Lua writes:

-- KEYS[1]=vehicle:{id}  ARGV=[ts,lng,lat,speed]
local ts = tonumber(ARGV[1])
local old = tonumber(redis.call('HGET', KEYS[1], 'ts') or '0')
if ts <= old then return 0 end          -- Old packet, discard
redis.call('HMSET', KEYS[1], 'ts', ts, 'lng', ARGV[2],
           'lat', ARGV[3], 'speed', ARGV[4])
redis.call('EXPIRE', KEYS[1], 300)     -- Offline vehicles auto-expire
return 1

Technical highlight: using ts version number as optimistic lock is 10x faster than distributed lock. No lock/unlock, no lock timeout renewal. One Lua script handles three things at once: out-of-order discard, atomic update, and automatic expiry for offline vehicles. This pattern works in all high-frequency write scenarios.

4.5 Writing real-time state to Redis: hash tag + ZSET

After cleaning and matching, data is written to Redis. Here's how the hash tag and ZSET design come together:

@Service
@RequiredArgsConstructor
public class BusGpsConsumer {

    private final RouteMatcher routeMatcher;
    private final SpeedSmoother speedSmoother;
    private final StringRedisTemplate redis;

    @KafkaListener(topics = "bus.gps.cleaned", groupId = "eta-writer")
    public void onGps(BusGps gps) {

        // Discard expired positions to avoid old data polluting real-time state
        if (System.currentTimeMillis() - gps.getTs() > 30_000) {
            return;
        }

        // Hash tag: ensure same line and direction data lands in the same slot
        String hashTag = "{line:%s:%d}".formatted(gps.getLineId(), gps.getDirection());
        String lineKey = "bus:%s:zset".formatted(hashTag);
        String vehicleKey = "bus:%s:veh:%s".formatted(hashTag, gps.getVehicleId());

        // Map matching: project GPS onto route, get distance along route
        RouteProjection projection = routeMatcher.project(gps);

        // Speed smoothing: avoid GPS jitter causing ETA to fluctuate wildly
        double smoothSpeed = speedSmoother.update(
                gps.getVehicleId(), gps.getSpeed(), gps.getTs());

        Map<String, String> pos = Map.of(
                "dist", String.valueOf(projection.getDistFromStart()),
                "speed", String.valueOf(smoothSpeed),
                "ts", String.valueOf(gps.getTs()),
                "seg", String.valueOf(projection.getSegmentId())
        );

        // ZSet score = distance along route; query with ZREVRANGEBYSCORE to find vehicles that haven't passed
        redis.opsForHash().putAll(vehicleKey, pos);
        redis.opsForZSet().add(lineKey, gps.getVehicleId(), projection.getDistFromStart());

        redis.expire(vehicleKey, Duration.ofSeconds(90));
        redis.expire(lineKey, Duration.ofMinutes(30));
    }
}

4.6 Batch writing historical trajectories to DB

Real-time state goes to Redis; historical trajectories must be persisted - they feed segment duration baselines later. Batch inserts replace row-by-row writes:

@Async
public void batchSave(List<BusLocationDTO> batch) {
    if (batch.isEmpty()) return;
    SqlSession sqlSession = sqlSessionFactory.openSession(ExecutorType.BATCH);
    try {
        BusLocationMapper mapper = sqlSession.getMapper(BusLocationMapper.class);
        for (BusLocationDTO dto : batch) {
            mapper.insert(convert(dto));
        }
        sqlSession.commit();
    } finally {
        sqlSession.close();
    }
}

At larger scales, this step goes straight to a time-series DB (TDengine / ClickHouse), not MySQL. The storage chapter covers this in detail.

Once data is cleaned, we need to "snap" it to the route, then determine whether the vehicle has arrived. If these two steps are done poorly, all previous effort is wasted.

5. Map matching and arrival judgment: nailing the "drifting" GPS to the route

5.1 Map matching: GPS projection onto the route

Cleaned GPS points are still "in the wild". They must be projected onto the route polyline to get the mileage along the route:

double mileage = RouteProjector.project(polyline, p);        // projected mileage in meters
int nextIdx = upperBound(line.stationMileages(), mileage);   // binary search for next station

The principle isn't complex: for each polyline segment, find the nearest projection point to the point, take the best, and accumulate previous segment lengths to get the mileage. A more refined production approach uses Hidden Markov Model (HMM) map matching, which combines road network topology to "snap" GPS points onto roads. It handles parallel roads and elevated bridges better.

For point-to-point distance, the Haversine formula is basic but essential:

/** Haversine formula: spherical distance between two lat/lon points in meters */
static double dist(double la1, double lo1, double la2, double lo2) {
    final double R = 6_371_000, rad = Math.PI / 180;
    double dLat = (la2 - la1) * rad, dLon = (lo2 - lo1) * rad;
    double a = Math.pow(Math.sin(dLat / 2), 2)
             + Math.cos(la1 * rad) * Math.cos(la2 * rad) * Math.pow(Math.sin(dLon / 2), 2);
    return 2 * R * Math.asin(Math.sqrt(a));
}

Why not straight-line distance? Buses follow fixed routes; on curvy roads, straight-line distance error can be several times. ETA must use arc-length along the route, which is the second highlight you can raise in an interview.

5.2 Arrival judgment: hysteresis dual-threshold state machine

There's a subtle pitfall here: GPS drift plus stopping at red lights makes vehicles jitter at the edge of stop fences. If you only use "within 50m of the stop counts as arrived", the state will flicker between "arrived/not arrived" like a disco.

Solution: hysteresis dual-threshold (Hysteresis) + state machine. The arrival threshold is strict (40m + low speed), the departure threshold is loose (80m + high speed), with a hysteresis zone in between that keeps the current state:

image.png

public enum StopState { APPROACHING, ARRIVED, DEPARTED }

/** Different thresholds for arrival/departure, naturally immune to fence-edge jitter */
static StopState transit(StopState cur, GpsPoint p, Station s) {
    double d = dist(p.lat(), p.lng(), s.lat(), s.lng());
    if (cur != StopState.DEPARTED && d < 40 && p.speed() < 2.0) return StopState.ARRIVED;
    if (cur == StopState.ARRIVED  && d > 80 && p.speed() > 5.0) return StopState.DEPARTED;
    return cur; // hysteresis zone: keep current state
}

This design is common in industrial control. Two asymmetric thresholds are inherently immune to edge jitter, more stable than any filter.

5.3 Speed smoothing: don't let instantaneous speed ruin ETA

Raw GPS instantaneous speed can be extreme: 10 m/s one moment, 0.5 m/s the next, 25 m/s after that. Using it directly for ETA causes wild jumps. Common approaches:

Method Principle Cost
Sliding window average Average of last N velocities Low, most common
Exponential moving average new×0.3 + old×0.7 Low, memory efficient
Kalman filter State estimation + noise model Medium, best effect
Clamp Clamp anomalous speed to a reasonable range Low, mandatory as fallback

Flink often uses exponential weighting - one line: smoothSpeed = lastSmooth * 0.7 + instantSpeed * 0.3.

In high-frequency write scenarios, a lock-free ring buffer can store the velocity window to avoid lock contention:

/** GPS writes every 5s, 30,000 vehicles concurrently → use lock-free RingBuffer to reduce contention */
public class SpeedRingBuffer {
    private final double[] buf = new double[60];   // 5-minute window
    private final AtomicInteger idx = new AtomicInteger();

    public void offer(double speed) {
        buf[idx.getAndIncrement() % buf.length] = speed;
    }

    public double avg() {
        return Arrays.stream(buf).filter(v -> v > 0).average().orElse(0);
    }
}

Another detail: when the window speed is too low (severe congestion), fall back to the historical speed for that road segment at the same time. Crawling speed is meaningless:

double speed = bus.speedWindow().avg();
if (speed < MIN_SPEED) {
    speed = historyService.getSpeedByHour(bus.getSegmentId(), currentHour());
}

Data cleaning, matching, and smoothing done, we finally reach the core of the problem: ETA prediction. This directly determines user experience and is also the most-pushed-back-on part in interviews.

6. ETA prediction algorithm: the soul of this system

6.1 Why "distance ÷ speed" is a trap

The simplest idea: ETA = remaining distance / current speed. Looks fine at first glance, but collapses in practice:

So the right approach for ETA is: segmented baseline + real-time correction. Here's the full evolution path:

image.png

6.2 V2: segment prediction + historical baseline

Core idea: don't calculate the whole stretch at once; calculate segment by segment. Locate the segment the vehicle is currently in, then accumulate the time for each segment from the current position to the target stop, using historical statistics for that time period:

ETA = current segment remaining time
    + Σ(median historical duration for intermediate segments)
    + stop count × dwell time
    × road condition correction factor

/**
 * Predicts seconds for the vehicle to reach the target stop.
 * Key points: Haversine distance + segment accumulation + dynamic road factor
 */
public static int predictArrivalSeconds(BusLocation bus, List<Station> lineStations,
                                        Long targetStationId, double avgSpeed) {
    // 1. Locate the vehicle's current station interval
    int curIdx = findCurrentStationIndex(bus, lineStations);
    int targetIdx = findStationIndexById(lineStations, targetStationId);
    if (targetIdx <= curIdx) return 0; // already passed

    // 2. Accumulate all segment distances from current to target
    double totalDistance = 0;
    Station nextStation = lineStations.get(curIdx + 1);
    totalDistance += calcDistance(bus.getLat(), bus.getLon(),
            nextStation.getLat(), nextStation.getLon());
    for (int i = curIdx + 1; i < targetIdx; i++) {
        Station s1 = lineStations.get(i);
        Station s2 = lineStations.get(i + 1);
        totalDistance += calcDistance(s1.getLat(), s1.getLon(), s2.getLat(), s2.getLon());
    }

    // 3. Real-time road condition factor correction (congestion factor >1, smooth =1)
    double roadFactor = RoadConditionService.getRoadFactor(bus.getLineId());
    return (int) (totalDistance / avgSpeed * roadFactor);
}

Historical baselines are usually computed offline from trajectory data, stored per "time slot × road segment" in Redis. The key is bus:avg_speed:{lineId}:{direction}:{hour}, with a default of 15km/h if not found - cold start doesn't guess; run on the baseline first.

When using real-time speed for correction, clamp the correction factor to a reasonable range to prevent instant jitter from blowing up the result:

// Historical median + real-time speed correction, more stable than raw distance / speed
private int segmentTime(String vehicleId, String line, int dir, int from, int to) {
    int base = historyCache.get(line, dir, from, to, timeSlot(now()));
    double factor = clamp(avgSpeed(vehicleId) / baseSpeed(line, dir, from, to), 0.7, 1.5);
    return (int) (base / factor);
}

A simpler implementation is dual-weight averaging: historical average speed 0.6, real-time speed 0.4, multiplied by a congestion factor. The idea is the same - history as the base, real-time as fine-tuning.

6.3 V3: three-layer weighted speed model

Going further, each road segment's speed is determined by three weighted layers: real-time measurement, historical same-period, and whole-network road condition:

@Service
public class EtaCalculator {

    // Weights: real-time 0.5 / history 0.3 / network 0.2 (can be tuned online)
    private static final double W_REAL = 0.5, W_HISTORY = 0.3, W_GLOBAL = 0.2;
    private static final double SMOOTH = 0.3; // exponential smoothing coefficient

    public EtaResult predict(BusLiveStatus bus, Station target) {
        List<Segment> segments = routeService.getRemainingSegments(
                bus.getRouteId(), bus.getNextStationId(), target.getStationId());

        long totalSec = 0;
        for (Segment seg : segments) {
            double vReal    = bus.getSpeed();                          // current measured speed
            double vHistory = historySpeedDao.avgSpeed(                // historical same-period (past 4 weeks, same weekday, same time)
                    seg.getId(), bus.getTimestamp());
            double vGlobal  = congestionCache.getRoadSpeed(seg.getRoadId());

            double vPredict = W_REAL * vReal + W_HISTORY * vHistory + W_GLOBAL * vGlobal;
            vPredict = Math.max(vPredict, 3.0); // min 3km/h, avoid division by zero / crawling

            totalSec += (long) (seg.getLengthKm() / vPredict * 3600);
            totalSec += seg.getStopDwellSec();  // stop dwell time (historical statistics)
        }

        // Exponential smoothing: avoid ETA numbers bouncing, more stable UX
        Long prev = etaCache.get(bus.getBusId() + ":" + target.getStationId());
        if (prev != null) {
            totalSec = (long) (SMOOTH * totalSec + (1 - SMOOTH) * prev);
        }
        etaCache.put(bus.getBusId() + ":" + target.getStationId(), totalSec);
        return EtaResult.of(bus.getBusId(), totalSec);
    }
}

Congestion factors can start with a simple time-period strategy - low cost, fast effect:

private double getCongestionFactor(Date reportTime) {
    Calendar cal = Calendar.getInstance();
    cal.setTime(reportTime);
    int hour = cal.get(Calendar.HOUR_OF_DAY);

    if ((hour >= 7 && hour <= 9) || (hour >= 17 && hour <= 19)) {
        return 1.5; // morning/evening peak 1.5x slower
    } else if (hour >= 11 && hour <= 13) {
        return 1.2; // midday mild congestion
    }
    return 1.0;     // off-peak no factor
}

Traffic light waiting also has empirical values: count the intersections along the route, add an average of 15 seconds per intersection.

The effect of this approach is striking:

Error distribution (left is better)
Pure distance/speed     ████████████████████████░░░░   average error 180s
With historical speed    ██████████████░░░░░░░░░░░░░░   average error  75s
Three-layer weighted+smooth   █████████░░░░░░░░░░░░░░░░░   average error  28s

From 180 seconds to 28 seconds - that's the value of the algorithm.

6.4 V3+: Kalman increment correction (the highlight design)

The three-layer weighting solves "accuracy", but there's another subtle UX issue: every new GPS arrives triggers a full recalculation, so the number jumps. Users stare at "5 minutes", suddenly it jumps to "2 minutes", then back to "4 minutes". Terrible experience.

A more elegant approach: don't do a global recalculation; only do incremental correction. Use a Kalman filter to estimate a "duration deviation coefficient" k per vehicle (1.0 means same as usual). After each segment is completed, use "actual duration ÷ baseline duration" to correct k:

/**
 * ETA predictor: historical segment baseline × real-time deviation coefficient
 * Highlights: ① Cold start uses baseline directly, no guessing  ② Only corrects remaining segments, converges and stabilizes as it approaches
 */
public final class EtaPredictor {

    /** One-dimensional Kalman: estimates the vehicle's "duration deviation coefficient" k */
    private double k = 1.0;   // state estimate
    private double p = 1.0;   // estimate covariance
    private static final double Q = 0.02;  // process noise: how fast road conditions change
    private static final double R = 0.35;  // observation noise: GPS/dwell jitter

    /** After each segment completes, correct with "actual duration / baseline duration" */
    public void update(double observed) {
        p += Q;
        double gain = p / (p + R);          // Kalman gain
        k += gain * (observed - k);
        p *= (1 - gain);
        k = Math.max(0.4, Math.min(3.0, k)); // prevent extreme values
    }

    /** Target stop ETA(seconds) = Σ remaining segment baseline × k + Σ dwell time */
    public long predict(List<Segment> remain) {
        double runSec  = remain.stream().mapToDouble(Segment::baselineSec).sum();
        double stopSec = remain.stream().filter(Segment::isStation)
                               .mapToDouble(Segment::dwellSec).sum();
        return (long) (runSec * k + stopSec);
    }

    public record Segment(long id, double baselineSec, double dwellSec, boolean isStation) {}
}

This design has two advantages:

① Cold start is not a panic: new vehicles or new lines without real-time data use k=1.0 and the historical baseline directly; no guessing. ② Only corrects "remaining segments": the closer the bus gets, the more corrections, and the more the result converges. The user sees a smooth converging curve, not jumping numbers.

Here's a golden interview phrase: "Users watch the convergence trend of 'how many minutes left'; jumping from 5 minutes to 2 is far worse than staying at 4."

Combined with the front-end display strategy: monotonically decreasing display (number only goes down, resets at arrival), experience complaints drop by 70%.

6.5 Flink implementation: stateful programming + smooth speed

In Flink, this ETA logic is typically implemented with a KeyedProcessFunction, partitioned by vehicle ID, with state storing the last GPS and current station index:

public class ArrivalTimePredictFunction
        extends KeyedProcessFunction<String, GpsReport, ArrivalPrediction> {

    private transient ValueState<GpsReport> lastGpsState;
    private transient ValueState<Integer> currentStationIndexState;

    @Override
    public void open(Configuration parameters) {
        lastGpsState = getRuntimeContext().getState(
                new ValueStateDescriptor<>("last-gps", GpsReport.class));
        currentStationIndexState = getRuntimeContext().getState(
                new ValueStateDescriptor<>("station-idx", Integer.class));
    }

    @Override
    public void processElement(GpsReport gps, Context ctx, Collector<ArrivalPrediction> out)
            throws Exception {

        GpsReport lastGps = lastGpsState.value();
        Integer currentIdx = currentStationIndexState.value();
        if (currentIdx == null) currentIdx = 0;

        // 1. Compute current speed (distance/time diff) + moving average filter
        double speedKmh;
        if (lastGps != null) {
            double distanceMeter = GeoUtils.calculateDistance(
                    lastGps.getLat(), lastGps.getLng(), gps.getLat(), gps.getLng());
            long timeDiffSec = (gps.getTimestamp() - lastGps.getTimestamp()) / 1000;
            double instantSpeed = timeDiffSec > 0 ? (distanceMeter / timeDiffSec) * 3.6 : 0;
            speedKmh = lastGps.getSmoothSpeed() * 0.7 + instantSpeed * 0.3;
        } else {
            speedKmh = gps.getSpeed();
        }
        gps.setSmoothSpeed(speedKmh);

        // 2. Compute distance and estimated time to the next N stops
        BusLine line = LineCache.getLine(gps.getLineId());
        List<BusStation> stations = line.getStations();
        for (int i = currentIdx; i < Math.min(currentIdx + 10, stations.size()); i++) {
            BusStation station = stations.get(i);
            double distToStation = GeoUtils.calculateDistance(
                    gps.getLat(), gps.getLng(), station.getLat(), station.getLng());

            // Consider traffic light waiting, add 15 seconds per intersection empirical value
            int trafficLightCount = countTrafficLights(currentIdx, i, line);
            double predictSeconds = (distToStation / 1000.0) / Math.max(speedKmh, 5) * 3600
                    + trafficLightCount * 15;

            ArrivalPrediction prediction = new ArrivalPrediction();
            prediction.setLineId(gps.getLineId());
            prediction.setBusId(gps.getBusId());
            prediction.setStationId(station.getStationId());
            prediction.setStationsAway(i - currentIdx);
            prediction.setArrivalSeconds((long) predictSeconds);
            out.collect(prediction);
        }

        lastGpsState.update(gps);
        currentStationIndexState.update(currentIdx);
    }
}

Note one detail: only predict the next 3-10 stops, not all of them. Precomputing for all stations and all vehicles would blow up write amplification. Limiting the precomputation range is the cost/performance balance.

6.6 How to quantify accuracy?

When the interviewer asks "how do you know ETA is accurate", don't panic; give the methodology:

Have a baseline before optimizing. Don't jump to deep learning; get the rule-based model working and let the data speak.

With ETA computed, results are written to Redis. But during morning/evening peaks, queries hit 100K QPS. How do users get millisecond responses? Next is the battle of the read path.

7. Query path: multi-level caching for 100K QPS

7.1 Overall idea: trade computation for queries, caching is king

The read path design principle in one sentence: keep it as light as possible, block at every layer.

image.png

Layer Technology TTL Role
L1 Caffeine local cache 1-3s Catches 80%-90% of hot duplicate queries, P99 < 1ms
L2 Redis 3-10s Catches remaining traffic, DB nearly zero pressure
L3 Timetable/historical fallback - Returns fallback message when all caches fail

You might ask: with a local 1-second cache, isn't the data too old? Remember, ETA itself refreshes every 5-10 seconds; a 1-3 second cache delay is imperceptible. Trading an extremely short TTL for 90% traffic interception is a great deal. Static resources (route maps, station lists) can also have an extra Nginx front cache.

7.2 Lua batch query: one script beats N requests

The core query is "find all vehicles on this line that haven't passed the target stop, and get their distance, speed, timestamp". A naive approach first does a ZSET query for vehicle IDs, then loops HMGET for each vehicle's details - N+1 requests, RT goes through the roof.

One Lua script solves it:

-- KEYS[1]: line vehicle ZSet, score = vehicle distance along route
-- KEYS[2]: vehicle Hash prefix, e.g. bus:{line:302:1}:veh:
-- ARGV[1]: target stop's distance from route start
-- ARGV[2]: max number of vehicles to return

local buses = redis.call(
    'ZREVRANGEBYSCORE',
    KEYS[1],
    ARGV[1],
    '-inf',
    'LIMIT', 0, tonumber(ARGV[2])
)

local res = {}
for i, bus in ipairs(buses) do
    local vals = redis.call('HMGET', KEYS[2] .. bus, 'dist', 'speed', 'ts')
    if vals[1] then
        res[#res + 1] = bus .. '|' .. vals[1] .. '|' .. vals[2] .. '|' .. vals[3]
    end
end
return res

Why ZREVRANGEBYSCORE? For example, target stop is 1500m from start:

Vehicle A: 1300m  → not passed, closest
Vehicle B: 900m   → not passed
Vehicle C: 200m   → not passed
Vehicle D: 1700m  → passed, score > 1500 automatically excluded

Query condition score <= 1500, reverse order, take the nearest few. Vehicles that already passed are naturally filtered. One command, clean and efficient. This is the dividend of the "score = distance along route" design.

7.3 Complete query service implementation

@Service
@RequiredArgsConstructor
public class ArrivalQueryService {

    private final StringRedisTemplate redis;
    private final RouteMetaService routeMeta;

    // Don't underestimate this 1-second local cache; hot stops have very high duplicate queries
    private final Cache<String, List<ArrivalVO>> localCache = Caffeine.newBuilder()
            .maximumSize(20_000)
            .expireAfterWrite(Duration.ofSeconds(1))
            .build();

    private static final DefaultRedisScript<List> ARRIVAL_SCRIPT =
            new DefaultRedisScript<>(LUA_SCRIPT, List.class);

    public List<ArrivalVO> query(String lineId, int direction, String stationId) {
        String cacheKey = lineId + ":" + direction + ":" + stationId;
        return localCache.get(cacheKey, k -> load(lineId, direction, stationId));
    }

    private List<ArrivalVO> load(String lineId, int direction, String stationId) {
        double stationDist = routeMeta.distFromStart(lineId, direction, stationId);

        String hashTag = "{line:%s:%d}".formatted(lineId, direction);
        String lineKey = "bus:%s:zset".formatted(hashTag);
        String vehiclePrefix = "bus:%s:veh:".formatted(hashTag);

        List<String> raw = redis.execute(ARRIVAL_SCRIPT,
                List.of(lineKey, vehiclePrefix),
                String.valueOf(stationDist), "5");

        if (raw == null || raw.isEmpty()) return List.of();

        return raw.stream()
                .map(item -> parse(lineId, direction, item, stationDist))
                .filter(this::fresh)
                .toList();
    }

    private ArrivalVO parse(String lineId, int direction, String raw, double stationDist) {
        String[] arr = raw.split("\\|");
        double dist = Double.parseDouble(arr[1]);
        double speed = Double.parseDouble(arr[2]);
        long ts = Long.parseLong(arr[3]);

        double remainDistance = stationDist - dist;

        // Lower speed limit protection: congestion, entering station, red lights can make ETA explode
        speed = Math.max(speed, 3.0);

        int remainStations = routeMeta.countStationsBetween(lineId, direction, dist, stationDist);
        // Per stop dwell, entering/exiting, red light loss estimated at 20 seconds
        double etaSeconds = remainDistance / speed + remainStations * 20;
        int etaMinutes = Math.max(1, (int) Math.ceil(etaSeconds / 60));

        return new ArrivalVO(arr[0], remainDistance, remainStations, etaMinutes, ts);
    }

    private boolean fresh(ArrivalVO vo) {
        // Positions not updated for over 20 seconds are considered stale, show degraded message
        return System.currentTimeMillis() - vo.ts() < 20_000;
    }

    public record ArrivalVO(String vehicleId, double remainDistanceM,
                            int remainStations, int etaMinutes, long ts) {}
}

Several details are bonus points: lower speed limit, data freshness filter, dwell compensation, minimum display of 1 minute.

7.4 Another implementation: station sequence ZSET + only the nearest 3 vehicles

If the ZSET score uses the station sequence currentSeq, the query can be written as - take the nearest 20 vehicles, calculate ETA, sort, return only the top 3:

public List<Eta> query(String lineId, int direction, int targetSeq) {
    String cacheKey = "eta:" + lineId + ":" + direction + ":" + targetSeq;

    return caffeine.get(cacheKey, k -> {
        String zKey = "line:" + lineId + ":" + direction + ":vehicles";

        // Take the nearest 20 vehicles that haven't reached the target stop
        Set<String> ids = redis.opsForZSet()
                .rangeByScore(zKey, targetSeq - 20, targetSeq - 0.01);

        return ids.stream()
                .map(id -> calcEta(id, targetSeq))
                .filter(e -> e.seconds() >= 0)
                .sorted(Comparator.comparingInt(Eta::seconds))
                .limit(3)
                .toList();
    });
}

private Eta calcEta(String vehicleId, int targetSeq) {
    VehicleState s = getState(vehicleId);
    int eta = 0;
    for (int seq = s.currentSeq(); seq < targetSeq; seq++) {
        eta += segmentTime(vehicleId, s.lineId(), s.direction(), seq, seq + 1);
    }
    eta += (targetSeq - s.currentSeq()) * STOP_SECONDS; // dwell time
    return new Eta(vehicleId, eta);
}

Both implementations are essentially the same: ZSET to circle vehicles, cache to absorb queries, segment computation for ETA. The only difference is the score design. Choose based on team preference in practice.

7.5 Cache penetration protection: SingleFlight + logical expiration

When a hot station's cache expires at that instant, tens of thousands of requests simultaneously try to rebuild - that's cache breakdown. Two weapons.

Weapon 1: SingleFlight request merging. Only one request per key is allowed to rebuild; others wait for the result:

@RestController
@RequestMapping("/api/bus")
public class BusArrivalController {

    // Local Caffeine cache: catches 90% of hot duplicate requests
    private final LoadingCache<String, List<ArrivalVO>> localCache = Caffeine.newBuilder()
            .maximumSize(10_000)
            .expireAfterWrite(2, TimeUnit.SECONDS)
            .build(this::loadFromRedis);

    // SingleFlight: merge requests for the same key to avoid cache breakdown
    private final ConcurrentHashMap<String, CompletableFuture<List<ArrivalVO>>> inFlightRequests
            = new ConcurrentHashMap<>();

    @GetMapping("/arrival")
    @SentinelResource(value = "bus-arrival", fallback = "arrivalFallback")
    public Result<List<ArrivalVO>> getArrivalInfo(@RequestParam String stationId) {
        return Result.success(localCache.get(stationId));
    }

    private List<ArrivalVO> loadFromRedis(String stationId) {
        CompletableFuture<List<ArrivalVO>> future = inFlightRequests.computeIfAbsent(
                stationId, k -> CompletableFuture.supplyAsync(() -> {
                    try {
                        // Key structure: bus:arrival:{stationId} -> field:lineId, value: vehicle list JSON
                        Map<Object, Object> lineData = redisTemplate.opsForHash()
                                .entries("bus:arrival:" + stationId);
                        List<ArrivalVO> result = new ArrayList<>();
                        for (Map.Entry<Object, Object> entry : lineData.entrySet()) {
                            ArrivalVO vo = new ArrivalVO();
                            vo.setLineId((String) entry.getKey());
                            vo.setBusList(JSON.parseArray((String) entry.getValue(),
                                    BusPrediction.class));
                            result.add(vo);
                        }
                        return result;
                    } finally {
                        inFlightRequests.remove(stationId);
                    }
                }));
        try {
            return future.get(300, TimeUnit.MILLISECONDS);
        } catch (Exception e) {
            return Collections.emptyList();
        }
    }

    /** Degradation logic: Redis down returns fallback data */
    public Result<List<ArrivalVO>> arrivalFallback(String stationId, Throwable ex) {
        log.warn("Query degraded, stationId={}", stationId, ex);
        return Result.success(Collections.emptyList(), "Data updating, please refresh later");
    }
}

Weapon 2: Logical expiration. The cache doesn't physically expire; instead, a "logical expiration" mark is stored in the value. When reading expired data, return the old value first, then asynchronously spawn a thread to rebuild:

public ArrivalVO query(long stationId) {
    String key = "arrival:" + stationId;
    CacheObj v = localCache.get(key);            // L1: Caffeine
    if (v == null) v = redis.get(key);           // L2: Redis
    if (v == null) return timetableFallback(stationId); // L3: timetable fallback

    // Logical expiration: only one thread fetches the source, the rest return old values, never break the DB
    if (v.logicExpired() && tryLock(key)) {
        taskPool.submit(() -> {
            try { redis.setex(key, 10, rebuild(stationId)); }
            finally { unlock(key); }
        });
    }
    return v.data();
}

The idea of logical expiration is worth pondering: for a real-time system, "slightly stale data" is far better than "no data" or "queued rebuilding". You can also use Redisson distributed lock: tryLock(50, 10, TimeUnit.MILLISECONDS). If the lock isn't acquired, don't block - return fallback data (estimate from the previous bus), same effect.

Complete defense for the three cache problems:

image.png

Problem Solution
Avalanche (many keys expire at once) Add random offset to TTL
Penetration (querying non-existent stop) Empty-value cache + Bloom filter
Breakdown (hot key expires) SingleFlight / logical expiration / hot key never expires + async background update
Cache-DB consistency Update DB first, then delete cache (delayed double-delete) + short TTL fallback

Random offset to prevent avalanche, one line:

// Base TTL + random offset to avoid mass expiration
long baseTTL = 300;
long randomOffset = ThreadLocalRandom.current().nextLong(60);
redisTemplate.opsForValue().set(key, value, baseTTL + randomOffset, TimeUnit.SECONDS);

Empty-value cache to prevent penetration (also cache empty results to prevent malicious queries for non-existent routes):

public List<VehicleLocationVO> getRealtimeLocations(Long lineId) {
    String cacheKey = "bus:location:" + lineId;
    String cached = redisTemplate.opsForValue().get(cacheKey);
    if (cached != null) {
        return JSON.parseArray(cached, VehicleLocationVO.class);
    }

    List<VehicleLocationVO> locations = locationMapper.selectRealtimeByLineId(lineId);
    if (locations != null && !locations.isEmpty()) {
        redisTemplate.opsForValue().set(cacheKey, JSON.toJSONString(locations),
                LOCATION_TTL, TimeUnit.SECONDS);
    } else {
        // Empty-value cache to prevent penetration
        redisTemplate.opsForValue().set(cacheKey, "[]", 60, TimeUnit.SECONDS);
    }
    return locations;
}

7.6 Rate limiting: the final brake

Use Sentinel for rate limiting at the entrance (single user 10 QPS), and Guava token bucket at the interface level, per-line:

// Each line allows 100 queries per second
private final Map<Long, RateLimiter> rateLimiters = new ConcurrentHashMap<>();

@GetMapping("/realtime")
public Result<List<VehicleLocationVO>> getRealtime(@RequestParam Long lineId) {
    RateLimiter limiter = rateLimiters.computeIfAbsent(lineId, k -> RateLimiter.create(100));
    if (!limiter.tryAcquire()) {
        return Result.error("Request too frequent, please retry later");
    }
    return Result.success(busLocationService.getRealtimeLocations(lineId));
}

7.7 GEO easter egg: nearby stop query

When a user opens the app, they usually first want "which stops are near me". This is where Redis GEO shines:

@Service
public class NearbyStationService {

    @Autowired
    private StringRedisTemplate redisTemplate;

    private static final String GEO_KEY = "bus:stations:geo";

    /** Initialize all station coordinates into Redis GEO */
    public void initStationsToGeo(List<BusStation> stations) {
        GeoOperations<String, String> geoOps = redisTemplate.opsForGeo();
        stations.forEach(station -> geoOps.add(GEO_KEY,
                new Point(station.getLongitude(), station.getLatitude()),
                station.getStationId()));
    }

    /** Query all stops within a radius, sorted by distance ascending */
    public List<StationDistanceVO> queryNearbyStations(double lon, double lat, double radius) {
        GeoOperations<String, String> geoOps = redisTemplate.opsForGeo();
        Circle circle = new Circle(new Point(lon, lat), new Distance(radius, Metrics.METERS));
        GeoResults<GeoLocation<String>> results = geoOps.radius(GEO_KEY, circle,
                RedisGeoCommands.GeoRadiusCommandArgs.newGeoRadiusArgs()
                        .includeDistance()
                        .includeCoordinates()
                        .sortAscending()
                        .limit(20));

        if (results == null) return Collections.emptyList();
        return results.getContent().stream().map(result -> {
            StationDistanceVO vo = new StationDistanceVO();
            vo.setStationId(result.getContent().getName());
            vo.setDistance(result.getDistance().getValue()); // distance in meters
            return vo;
        }).collect(Collectors.toList());
    }
}

Redis GEO is built on GeoHash + ZSET, radius query complexity O(logN), far outperforming MySQL's ST_Distance_Sphere full-table computation. For stop matching, when data volume is small, MySQL spatial indexes can work; at scale, GEO is the recommendation.

Using the same pattern, GEOADD vehicle coordinates per line to line:{lineId}:geo, and on the map draw with GEOSEARCH: with coordinates, sorted by distance ascending, take the nearest 10 vehicles - one command.

Query chapter in one sentence: 99% of requests are intercepted at the cache layer, 1% of rebuilds are merged into one, and extreme cases have fallbacks. That's the full depth defense for high-concurrency reads.

After query, there's still a UX problem: users watch the page while waiting. How does data auto-refresh? Polling? Too low-level.

8. Real-time push: pull-push combo, guarding against push storms

8.1 Why not let the app poll?

Many people's first instinct is front-end polling every 3 seconds. Let's do the math: at 10,000 QPS of users, each polling every 3 seconds, actual request volume can hit 100,000 QPS - 90% of that is wasted polling (vehicle positions haven't changed).

Polling amplifies wasted traffic. The right approach is pull-push combo:

Phase Method Description
First entry HTTP pull Pull once in full, display immediately
While on page WebSocket push Push only when state actually changes, traffic drops by an order of magnitude
Low-requirement scenarios 5-second short polling Don't over-engineer early on

Bus scenarios don't need millisecond accuracy like ride-hailing; users accept 5-10 second refresh. So start with 5-second polling for rapid launch, and switch to WebSocket when experience requirements rise - this evolution path is the most pragmatic.

8.2 Aggregated subscription: compress push volume from "number of users" to "number of combinations"

If each user gets a separate push, a million connections is a push storm. Key optimization: merge users subscribing to the same "line + stop" into one channel:

// Push side: merge N users of the same line+stop into one channel, compute once and broadcast to all
String channel = "line:" + lineId + ":stop:" + stopId;
channelSessions.computeIfAbsent(channel, k -> new CopyOnWriteArraySet<>()).add(session);
// Timer every 5s: batch read Redis → broadcast per channel → push volume drops from "users" to "combinations"

Push volume becomes "line × stop combinations" instead of "users", compressing millions to thousands. This is the most valuable optimization in the push path.

Subscription relationships are maintained locally on the gateway; a timer batch-reads Redis and broadcasts:

@Component
public class BusArrivalWebSocketHandler extends TextWebSocketHandler {

    private final RedisTemplate<String, Object> redisTemplate;
    // stop -> session set subscription relationship
    private final ConcurrentHashMap<String, Set<WebSocketSession>> subscribeMap
            = new ConcurrentHashMap<>();

    public BusArrivalWebSocketHandler(RedisTemplate<String, Object> redisTemplate) {
        this.redisTemplate = redisTemplate;
    }

    @Override
    protected void handleTextMessage(WebSocketSession session, TextMessage message) {
        // subscription message format: subscribe:stationId
        String payload = message.getPayload();
        if (payload.startsWith("subscribe:")) {
            String stationId = payload.split(":")[1];
            subscribeMap.computeIfAbsent(stationId, k -> ConcurrentHashMap.newKeySet())
                        .add(session);
        }
    }

    /** Push arrival info every 3 seconds */
    @Scheduled(fixedDelay = 3000)
    public void pushArrivalInfo() {
        subscribeMap.forEach((stationId, sessions) -> {
            if (sessions.isEmpty()) return;
            Object arrivalInfo = redisTemplate.opsForValue().get("bus:arrival:" + stationId);
            if (arrivalInfo == null) return;

            TextMessage msg = new TextMessage(JSON.toJSONString(arrivalInfo));
            sessions.forEach(session -> {
                try {
                    if (session.isOpen()) session.sendMessage(msg);
                } catch (IOException e) {
                    sessions.remove(session); // remove dead connection
                }
            });
        });
    }
}

8.3 Cluster push: MQ broadcast + local fan-out

In a clustered deployment, subscription relationships are scattered across gateway nodes. Use MQ (Redis Pub/Sub or Kafka) to broadcast change events; each node only pushes the connections on its own machine:

/** Gateway maintains stationId -> Sessions locally; MQ broadcasts events, each node pushes only its own connections */
static final Map<Long, Set<Session>> TOPIC = new ConcurrentHashMap<>();

static void fanout(long stationId, String payload) {
    for (Session s : TOPIC.getOrDefault(stationId, Set.of())) {
        if (s.isOpen()) s.getAsyncRemote().sendText(payload); // async, non-blocking Netty thread
    }
}

If the push events are computed by Flink, you can connect a Sink directly to the push gateway:

@Component
public class ArrivalPushSink implements SinkFunction<ArrivalEvent> {
    @Override
    public void invoke(ArrivalEvent value, Context ctx) {
        wsGateway.broadcast("station:" + value.getStationId(),
                JsonUtil.toJson(ArrivalPushVO.from(value)));
    }
}

8.4 Push path engineering details checklist

9. Storage and hot/cold separation: how to store billions of trajectories without going bankrupt

9.1 Data characteristics determine the choice

Trajectory data has three characteristics: write-only, almost never updated, and queries always include a time range. This is the home turf of time-series databases; putting it in MySQL is burning money.

Common options compared:

Product Features Suitable for
TDengine Super table design, high compression, storage cost down 70% Vehicle trajectory storage, cost-sensitive
InfluxDB Mature ecosystem, low learning curve Small to medium time-series data
ClickHouse Columnar storage, strong analytical queries Historical trajectory analysis, segment duration statistics
HBase Massive KV, mature horizontal scaling Billion-scale trajectory archiving, high-throughput random reads

My recommendation: real-time computation in Redis, trajectory in TDengine / ClickHouse, billion-scale archives in HBase + object storage. Model tables sharded by "line + time" to improve query efficiency further.

9.2 Three-level hot/warm/cold separation

Tier Storage Retention Purpose
Hot Redis seconds to minutes real-time queries
Warm TDengine / ClickHouse 7 days recent trajectory replay, baseline training
Cold OSS / S3 compressed archive forever offline analysis, compliance retention

Regularly archive expired data to object storage; overall storage cost drops 60%-70%. This historical data is not waste - the segment duration baselines for ETA are trained from exactly this data. The data loop is closed.

10. Degradation and disaster recovery: the system can be inaccurate, but not unavailable

A real-time system has one iron rule: users can accept "data delay", but not "white screen".

Design a three-level degradation chain:

image.png

Scenario table:

Failure scenario Degradation plan
Redis down Local Caffeine holds for 60s + return "no real-time data, refer to timetable"
Flink / Kafka failure Return historical timetable ETA, front-end flags "data delayed"
Query timeout Only return the nearest vehicle, no sorting
Push failure Degrade to in-app message / show on next query
Tunnel GPS loss Last valid position + time decay, mark "weak signal"
City-wide GPS outage Three-level degradation: real-time → last position estimate → timetable
Position not updated for 30s Don't force computation, return delayed prompt message

Consistency strategy: eventual consistency model + timestamp version comparison + TTL fallback expiry, accepting up to 3 seconds of delay.

Degradation copy is also part of the product experience:

{
  "lineId": "302",
  "stationId": "1001",
  "status": "DELAY",
  "message": "Vehicle data may be delayed; please refer to actual arrival"
}

This is more reliable than returning a confidently wrong ETA. Better to show "about 5 minutes (timetable)" than a white screen; better to be inaccurate than unavailable.

High availability also needs a set of infrastructure: Redis Sentinel / Cluster, Kafka cluster + message persistence, MySQL master-slave + read-write separation, service health checks + auto-restart, Sentinel rate limiting and circuit breaking. Target availability 99.95%.

11. Difficulty summary table + interview follow-up handling

11.1 Difficulties and solutions summary table

This table condenses all the pitfalls of this system; bookmark it:

# Difficulty Concrete manifestation Solution
1 GPS drift / wild points Position jumps across the river Speed mutation filter + jump detection + Kalman smoothing + map matching constraint, wild point rate 3% to 0.1%
2 Tunnel GPS loss No reports for minutes Last valid position + time decay, mark "weak signal"
3 Coordinate system confusion Fleet shifts by hundreds of meters GCJ-02 / WGS84 unified correction
4 Message out-of-order / duplicate 4G retransmission causes vehicle to "go backward" Kafka partition ordering + ts version Lua idempotency + monotonic timestamp discard
5 Arrival state jitter Fence-edge flicker Hysteresis dual threshold 40m/80m + state machine
6 ETA inaccurate Pure distance/speed error 180s Segmented baseline + three-layer weighting + Kalman correction, error to 28s
7 Prediction jump 3min→1min→4min Exponential smoothing + front-end monotonic decreasing display, complaints down 70%
8 Hot read amplification Thousands of QPS on hot stops at peak Caffeine + Redis multi-level cache + SingleFlight + logical expiration + hot key splitting, P99 < 20ms
9 Cache three brothers breakdown / avalanche / penetration TTL random jitter + empty-value cache + Bloom filter + hot key never expires
10 Write flood 100K TPS blows up MySQL Kafka spike absorption + Flink stream consumption + Redis incremental writes + batch inserts, write latency < 200ms
11 Push storm million long connections Aggregate subscription by line+stop + 5s merge + consistent hash sharding, gateway sustains 500K connections
12 Direction misjudgment Two-way platforms easily confused Continuous displacement direction + station sequence order validation + GeoHash direction features
13 Route change consistency Cache and DB inconsistent Binlog CDC broadcast + version number + short TTL fallback, route data versioning + gray update
14 Historical data bloat Billion-scale trajectory storage cost Hot/warm/cold three-tier separation, storage cost down 60%-70%

11.2 Follow-up predictions, plant hooks in advance

Q: Why not query the DB directly to compute ETA?

Real-time queries must return within 100ms; a single DB query takes tens of milliseconds and can't handle 100K QPS. So computation is moved forward into the Flink stream and computed asynchronously; queries only "read results" - this is the classic trading computation for queries approach.

Q: What if a user queries all lines passing a stop?

Use a station → lines mapping table. Interface layer uses concurrent queries + timeout control + partial failure degradation; don't let one slow line drag down the whole interface:

List<CompletableFuture<List<ArrivalVO>>> futures = lineIds.stream()
    .map(lineId -> CompletableFuture.supplyAsync(
        () -> arrivalQueryService.query(lineId, direction, stationId),
        executor))
    .toList();

Q: How to guarantee last-bus reminders?

Use arrival events + timetable last-bus rules as triggers, using delayed messages (RocketMQ scheduled messages).

Q: What if Redis is completely down?

Local Caffeine holds for 60 seconds + timetable static page. Better to show "about 5 minutes (timetable)" than a white screen.

Q: How to determine direction? Same road has two-way stops.

Three-layer validation: displacement direction from two consecutive GPS points + matching station sequence order (after station A, next should be B) + line GeoHash direction features. A wrong direction shows vehicles going the other way; this pit must be plugged.

11.3 What the interviewer is actually testing

This question appears to test technology, but actually tests four layers of ability:

Interview tip: draw the architecture diagram while talking; pick 2-3 highlights and explain them thoroughly instead of covering everything shallowly.

12. Advanced evolution directions

The first version is not the end; this system has a clear evolution path.

1. Read-time computation → write-time precomputation

Initial phase: compute on query for rapid launch; when QPS rises, move computation forward - when a vehicle position updates, also update its predicted arrival times for the next 3-5 stops; user queries directly read the stop cache:

// station:arrival:1001:302:1
[
  {"vehicleId": "YueB12345", "remainStations": 2, "etaMinutes": 5, "ts": 1730000000000},
  {"vehicleId": "YueB67890", "remainStations": 4, "etaMinutes": 11, "ts": 1730000000000}
]
Dimension Read-time computation Write-time precomputation
Query path Needs real-time computation Reads cache directly
Write pressure Small Write amplification (limited to next 3-5 stops)
Applicable scenario Early stage, small/medium scale High QPS, peak pressure
Complexity Low Must handle expired vehicles, route change cache invalidation

2. Rule model → machine learning

After enough data is collected, ETA models can upgrade to XGBoost / LSTM, adding features like weather, day of week, holidays, real-time road conditions, and using A/B testing to continuously validate.

3. More imagination space

This article is long; it covers all angles of answering this problem. Later we will continue breaking down high-frequency system design questions. If you don't want to miss out, hit follow.

Closing

After all this, let me summarize the system in one paragraph:

The vehicle GPS stream, after cleaning and map matching, is converted into distance along route; then based on station distances and segment baselines, ETA is computed with Kalman incremental correction; the query side uses multi-level caching, Lua scripts, and degradation strategies to handle high concurrency; the push side uses aggregated subscriptions to prevent storms.

Four phrases as a mantra:

  1. Quantify first, then design - system design without QPS numbers is irresponsible
  2. Trade computation for queries - move computation to the write path, keep the read path light
  3. Distance along route, not straight-line distance - the first cognitive hurdle of this problem
  4. Degradation determines the floor - the system can be inaccurate, but not unavailable

Reviewing the technical highlights: ZSET score stores distance along route, one command gets vehicles that haven't passed; ts version number Lua idempotency, 10x faster than distributed lock; hysteresis dual-threshold state machine, immune to edge jitter; Kalman incremental correction, letting ETA converge without jumping; aggregate push, compressing millions of push messages to thousands.

Interaction question: If you were the interviewer, what would you ask after hearing this answer? Or have you done ETA prediction in a real project? What clever solutions did you use? Comment below; I'll pick the 3 most interesting and break them down in a follow-up.

If this helped, like, share, and forward it to friends who are interviewing. Your support is the biggest motivation for me to keep updating. See you next time.

Comments

Top 1 from juejin.cn, machine-translated. The original thread is authoritative.

PasteCode

What's more reasonable is to not have the onboard terminal at all [grin][grin][grin]