跪拜 Guibai
← All articles
Java · Architecture

How a Like Button Survives 100K QPS: Redis Lua, Kafka Batching, and Nightly Reconciliation

By 吃饱了得干活 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

A like counter is the simplest social feature, but at scale it exposes every distributed-systems problem: race conditions, write amplification, cache drift, and abuse. The Lua-plus-Kafka-plus-nightly-reconciliation pattern shown here is the same template used for views, votes, reactions, and any high-write, read-heavy counter, making it directly portable to most consumer apps.

Summary

A like button that handles tens of millions of daily active users runs on a read/write-split architecture. Redis holds the live relationship and count data; a single Lua script atomically checks, toggles, and increments or decrements the count, eliminating race conditions. The service layer immediately returns success to the client while firing an asynchronous Kafka message for eventual persistence.

On the consumer side, messages are pulled in batches and written to MySQL via a bulk Upsert that leverages a unique index on user, target, and type. This collapses thousands of individual writes into a single database round-trip. A sliding-window rate limiter built on Redis ZSets caps per-user like frequency, and a scheduled 3 a.m. reconciliation job overwrites Redis counts with database aggregates to correct any drift.

For extreme hotspots, the design extends to local Caffeine caches, automatic hotspot detection with HeavyKeeper, and sharding or TiDB when MySQL alone cannot absorb the write volume. The entire pipeline is framed around eventual consistency, idempotency at every stage, and layered fallbacks so that no single component failure corrupts the count.

Takeaways
Redis Lua scripts execute the check-toggle-count sequence atomically, preventing the double-increment race that occurs when separate commands are used.
Even idempotent operations re-send a Kafka message to realign the database with the cache in case of prior inconsistency.
Batch consumption with a bulk Upsert (ON DUPLICATE KEY UPDATE) reduces database IO from O(N) individual writes to O(1) per batch.
A sliding-window rate limiter using Redis ZSets caps per-user like frequency more smoothly than fixed-window counters.
A nightly cron job queries the database for true counts and overwrites Redis, correcting any drift from lost messages or consumer lag.
Hotspot mitigation layers a local Caffeine cache and a HeavyKeeper-based detector in front of Redis to absorb celebrity-traffic spikes.
Conclusions

Redis Lua is not just a performance optimization here; it is the correctness primitive. Without it, the system would need distributed locks or a compare-and-swap loop, both of which add latency and complexity.

Sending a Kafka message even when the Lua script returns 'no change' is a deliberate over-correction strategy: it treats the message queue as a repair mechanism, not just a transport.

The reconciliation job runs at 3 a.m. and overwrites Redis with database aggregates, which means the database is always the source of truth. This flips the usual cache-aside pattern and accepts that Redis is disposable.

The architecture deliberately avoids distributed transactions. Instead, it chains atomic Redis operations, at-least-once Kafka delivery, idempotent Upserts, and a periodic full-scan reconciliation to achieve eventual consistency without two-phase commit.

Scaling the write path beyond a single MySQL instance is addressed with sharding by target_id, which aligns the partition key with the natural access pattern: all writes for a single post land on the same shard, making the batch Upsert efficient.

Concepts & terms
Redis Lua atomicity
Redis executes Lua scripts as a single, serialized unit. While a script runs, no other Redis command can interleave, so a multi-step operation like check-then-increment becomes atomic without distributed locks.
Batch Upsert (ON DUPLICATE KEY UPDATE)
A MySQL insert statement that, when a unique key conflict occurs, updates the existing row instead of failing. It allows thousands of insert-or-update operations to be sent in a single round-trip, drastically reducing database load.
Sliding-window rate limiting with ZSet
Each request is added to a sorted set with a timestamp score. Expired entries are trimmed, and the remaining count is checked against a limit. This produces a smooth rate curve compared to fixed-window counters that reset abruptly.
HeavyKeeper algorithm
A probabilistic data structure for finding heavy hitters (frequent items) in a stream with low memory overhead. It can detect suddenly popular keys in real time, triggering local caching or throttling before the hotspot saturates Redis.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗