← Back to the summary

How a Like Button Survives 100K QPS: Redis Lua, Kafka Batching, and Nightly Reconciliation

The like feature seems simple, but with tens of millions of daily active users, how do you ensure data consistency, prevent like fraud, and handle hotspot traffic? This article will take you through building a high-concurrency like system from scratch.

1. Business Scenarios and Challenges

Liking is a standard feature on almost all social/content platforms. When your product reaches tens of millions of daily active users, a simple like button hides quite a few technical challenges:

High-concurrency writes: Hot content can instantly attract hundreds of thousands of like requests
Data consistency: Like status must be accurate, and counts cannot be wrong
Anti-fraud governance: There are always "enthusiastic users" who want to flood your API
Hotspot penetration: When a celebrity makes an announcement, the QPS for a single piece of content can exceed one million

Core design philosophy: Read/write separation, asynchronous decoupling, eventual consistency

2. Overall Architecture Design

First, look at an architectural overview diagram (described in text):

┌─────────┐    ┌─────────┐     ┌─────────────────┐    ┌─────────┐
│ Client  │───▶│ Gateway │───▶│   Like Service  │───▶│  Redis  │
│(Debounce)│    │(Rate Lim)│    │(Business Logic+ │    │ (Cache) │
└─────────┘    └─────────┘     │      Lua)       │    └─────────┘
                               └────────┬────────┘
                                        │
                                        ▼ (Async Message)
                                ┌───────────────┐
                                │   Kafka/MQ    │
                                └───────┬───────┘
                                        ▼ (Batch Consumption)
                                ┌───────────────┐
                                │    MySQL      │
                                │ (Final Store) │
                                └───────────────┘

Responsibilities of each layer:

Layer	Technology Choice	Core Responsibility
Client Layer	Frontend/SDK	Debounce handling, optimistic UI updates
Gateway Layer	Spring Cloud Gateway	Rate limiting, circuit breaking, authentication
Service Layer	Spring Boot	Business orchestration, Lua script execution
Cache Layer	Redis Cluster	Handling real-time reads/writes, storing like relationships
Message Layer	Kafka/Pulsar	Asynchronous decoupling, traffic peak shaving
Storage Layer	MySQL/TiDB	Final data persistence

3. Core Data Structure Design

3.1 Redis Storage Design

This is the key to the system. We use three data structures, each with its own role:

Business Scenario	Redis Structure	Key Design	Description
Like Relationship	Hash	`like:{type}:{targetId}`	Field=userId, Value=timestamp, supports fast querying of a single user's status
Like Count	String	`like_count:{type}:{targetId}`	Stores total like count, INCR/DECR atomic operations
User Like List	ZSet	`user_like:{userId}`	Score=timestamp, Value=targetId, supports paginated queries for "what I liked"

Why choose Hash instead of Set?

Although Set can also store userIds, Hash can additionally store timestamps, making it convenient for subsequent like timeline displays. Furthermore, Hash's HEXISTS command has O(1) time complexity, offering excellent performance.

3.2 Database Table Design

As the final destination for data, two core tables are sufficient:

-- Like record table (records every like action)
CREATE TABLE `like_record` (
    `id` BIGINT PRIMARY KEY AUTO_INCREMENT,
    `user_id` BIGINT NOT NULL COMMENT 'User ID',
    `target_id` VARCHAR(64) NOT NULL COMMENT 'Target business ID',
    `target_type` TINYINT NOT NULL COMMENT 'Target type: 1 post 2 comment',
    `status` TINYINT DEFAULT 1 COMMENT 'Status: 1 like 0 cancel',
    `create_time` DATETIME DEFAULT CURRENT_TIMESTAMP,
    `update_time` DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    UNIQUE KEY `uk_user_target` (`user_id`, `target_id`, `target_type`),
    KEY `idx_target` (`target_id`, `target_type`)
) ENGINE=InnoDB;

-- Like count table (stores a snapshot of the total like count for each target)
CREATE TABLE `like_count` (
    `id` BIGINT PRIMARY KEY AUTO_INCREMENT,
    `target_id` VARCHAR(64) NOT NULL,
    `target_type` TINYINT NOT NULL,
    `count` BIGINT DEFAULT 0,
    `update_time` DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    UNIQUE KEY `uk_target` (`target_id`, `target_type`)
) ENGINE=InnoDB;

Key point: The unique index uk_user_target on the like_record table is the basis for implementing batch Upsert and also the last line of defense against duplicate likes.

4. Core Process: The Complete Chain of a Like Operation

4.1 Overall Flowchart


User clicks ❤️
    │
    ▼
① Gateway Rate Limiting (Sliding Window)
    │
    ▼
② Execute Lua Script (Redis Atomic Operation)
   ├─ Check if already liked
   ├─ Update Hash relationship
   └─ INCR/DECR count
    │
    ▼
③ Send Kafka Message (Async)
    │
    ▼
④ Immediately return "Like successful"  ← User perception ends here
    │
    ▼ (Background async processing)
⑤ Kafka Consumer batch pull
    │
    ▼
⑥ Batch Upsert to MySQL
    │
    ▼
⑦ Scheduled Reconciliation (nightly fallback)

4.2 Step 1: Atomic Operation — Lua Script

This is the most core code of the entire system. Why must Lua be used?

If split into two steps: check if liked → update count, a race condition occurs under high concurrency: two threads simultaneously find "not liked", then both execute INCR, causing the count to be over-calculated. Lua scripts execute serially within Redis, making them naturally atomic.

Like/Unlike Lua Script:

-- KEYS[1] = Like relationship Hash (like:post:123)
-- KEYS[2] = Like count (like_count:post:123)
-- ARGV[1] = User ID
-- ARGV[2] = Operation type (1 like, 0 unlike)
-- ARGV[3] = Timestamp

local relation_key = KEYS[1]
local count_key = KEYS[2]
local user_id = ARGV[1]
local operate = tonumber(ARGV[2])
local ts = ARGV[3]

local is_liked = redis.call('HEXISTS', relation_key, user_id)

if operate == 1 then
    -- === Like ===
    if is_liked == 1 then
        return 0  -- Idempotent, already liked, no repeat processing
    end
    redis.call('HSET', relation_key, user_id, ts)
    redis.call('INCR', count_key)
    return 1
else
    -- === Unlike ===
    if is_liked == 0 then
        return 0  -- Idempotent, was not liked originally
    end
    redis.call('HDEL', relation_key, user_id)
    local cnt = redis.call('DECR', count_key)
    if cnt < 0 then
        redis.call('SET', count_key, 0)
    end
    return 1
end

Spring Boot Integration Execution:

@Component
public class LikeRedisService {
    
    @Autowired
    private StringRedisTemplate redisTemplate;
    private DefaultRedisScript<Long> likeScript;

    @PostConstruct
    public void init() {
        likeScript = new DefaultRedisScript<>();
        likeScript.setLocation(new ClassPathResource("lua/like.lua"));
        likeScript.setResultType(Long.class);
    }

    /**
     * Execute like operation, returns 1=status changed, 0=idempotent no change
     */
    public Long execute(Long userId, String targetId, Integer targetType, Boolean isLike) {
        String relationKey = "like:" + targetType + ":" + targetId;
        String countKey = "like_count:" + targetType + ":" + targetId;
        
        return redisTemplate.execute(
            likeScript,
            Arrays.asList(relationKey, countKey),
            userId.toString(),
            isLike ? "1" : "0",
            String.valueOf(System.currentTimeMillis())
        );
    }
}

4.3 Step 2: Service Layer Orchestration


@Service
@Slf4j
public class LikeService {

    @Autowired
    private LikeRedisService redisService;
    @Autowired
    private LikeEventProducer producer;  // Kafka Producer

    @Transactional  // Note: This only manages local transactions, not involving Redis
    public Boolean toggle(Long userId, String targetId, Integer targetType, Boolean isLike) {
        // 1. Execute Redis atomic operation (< 1ms)
        Long result = redisService.execute(userId, targetId, targetType, isLike);
        
        // 2. Send async message (does not block the main flow)
        LikeEventDTO event = LikeEventDTO.builder()
            .userId(userId)
            .targetId(targetId)
            .targetType(targetType)
            .isLike(isLike)
            .timestamp(System.currentTimeMillis())
            .build();
        producer.send(event);
        
        // 3. Return result (frontend does its own optimistic update)
        return isLike;
    }
}

A small detail here: even if result == 0 (idempotent operation), we still send an MQ message. Why? Because cache and DB might become inconsistent due to some exception; sending an extra message allows the DB's final state to align with the user's operation, essentially adding an extra layer of fallback.

5. Asynchronous Persistence: Kafka Batch Consumption

The eventual consistency of likes is guaranteed by the MQ consumer. The key lies in batch consumption + batch Upsert, reducing database IO times from O(N) to O(1).

5.1 Producer: Fast Delivery

@Component
@Slf4j
public class LikeEventProducer {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;
    private static final String TOPIC = "like_event";

    public void send(LikeEventDTO event) {
        String json = JSON.toJSONString(event);
        // Use targetId as the partition key to ensure ordered consumption for the same target
        kafkaTemplate.send(TOPIC, event.getTargetId(), json)
            .addCallback(
                r -> log.debug("Send successful: {}", event),
                ex -> log.error("Send failed: {}", event, ex)
            );
    }
}

5.2 Consumer: Batch Upsert

@Component
@Slf4j
public class LikeEventConsumer {

    @Autowired
    private LikeRecordMapper recordMapper;

    @KafkaListener(topics = "like_event", batch = "true", containerFactory = "batchFactory")
    public void consume(List<ConsumerRecord<String, String>> records) {
        if (records.isEmpty()) return;
        
        List<LikeRecord> list = records.stream()
            .map(r -> JSON.parseObject(r.value(), LikeEventDTO.class))
            .map(this::convert)
            .collect(Collectors.toList());
        
        // Batch insert or update, 1000 records per batch
        int batchSize = 1000;
        for (int i = 0; i < list.size(); i += batchSize) {
            int end = Math.min(i + batchSize, list.size());
            recordMapper.batchInsertOrUpdate(list.subList(i, end));
        }
        
        log.info("Batch persistence completed, count: {}", list.size());
    }
    
    private LikeRecord convert(LikeEventDTO dto) {
        LikeRecord record = new LikeRecord();
        record.setUserId(dto.getUserId());
        record.setTargetId(dto.getTargetId());
        record.setTargetType(dto.getTargetType());
        record.setStatus(dto.getIsLike() ? 1 : 0);
        record.setUpdateTime(new Date(dto.getTimestamp()));
        return record;
    }
}

Corresponding MyBatis XML (using unique index to implement Upsert):

<insert id="batchInsertOrUpdate" parameterType="list">
    INSERT INTO like_record (user_id, target_id, target_type, status, update_time)
    VALUES
    <foreach collection="list" item="item" separator=",">
        (#{item.userId}, #{item.targetId}, #{item.targetType}, #{item.status}, #{item.updateTime})
    </foreach>
    ON DUPLICATE KEY UPDATE
        status = VALUES(status),
        update_time = VALUES(update_time)
</insert>

6. Anti-Fraud Governance: Sliding Window Rate Limiting

The like API is one of the most easily abused APIs. We implement sliding window rate limiting based on Redis ZSet, which is smoother compared to fixed windows:

@Component
public class SlidingWindowRateLimiter {

    @Autowired
    private StringRedisTemplate redisTemplate;

    /**
     * Check if allowed to pass
     * @param key Rate limit Key (suggestion: rate:like:{userId})
     * @param windowSeconds Window size (seconds)
     * @param maxRequests Maximum number of requests
     */
    public boolean tryAcquire(String key, int windowSeconds, int maxRequests) {
        long now = System.currentTimeMillis();
        long windowStart = now - windowSeconds * 1000L;
        String member = now + "-" + UUID.randomUUID().toString().substring(0, 6);
        
        String lua = 
            "redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, ARGV[1]) " +
            "local cnt = redis.call('ZCARD', KEYS[1]) " +
            "if cnt < tonumber(ARGV[2]) then " +
                "redis.call('ZADD', KEYS[1], ARGV[3], ARGV[4]) " +
                "redis.call('EXPIRE', KEYS[1], ARGV[5]) " +
                "return 1 " +
            "else return 0 end";
        
        Long result = redisTemplate.execute(
            new DefaultRedisScript<>(lua, Long.class),
            Collections.singletonList(key),
            String.valueOf(windowStart),
            String.valueOf(maxRequests),
            String.valueOf(now),
            member,
            String.valueOf(windowSeconds * 2)  // Expiry time leaves enough buffer
        );
        
        return result != null && result == 1L;
    }
}

Usage in Controller:

@PostMapping("/toggle")
public ResultVO toggle(@RequestBody LikeRequest req) {
    String limitKey = "rate:like:" + req.getUserId();
    if (!rateLimiter.tryAcquire(limitKey, 60, 30)) {
        throw new BusinessException("Operation too frequent, please try again later");
    }
    // ... normal business logic
}

7. Fallback Solution: Scheduled Reconciliation

Even with MQ, extreme situations (network jitter, consumer restarts) can still cause inconsistency between Redis and DB. We add a scheduled calibration task executed in the early morning:


@Component
@Slf4j
public class LikeConsistencyTask {

    @Autowired
    private LikeRecordMapper recordMapper;
    @Autowired
    private StringRedisTemplate redisTemplate;

    @Scheduled(cron = "0 0 3 * * ?")
    public void reconcile() {
        log.info("Starting like data reconciliation...");
        
        // Group by target from DB to count real like numbers
        List<CountStat> stats = recordMapper.selectCountGroupByTarget();
        
        for (CountStat stat : stats) {
            String key = "like_count:" + stat.getTargetType() + ":" + stat.getTargetId();
            // Overwrite Redis with DB as the source of truth
            redisTemplate.opsForValue().set(key, String.valueOf(stat.getCount()));
        }
        
        log.info("Reconciliation completed, processed {} targets", stats.size());
    }
}

8. Extended Thoughts

8.1 What if the daily like volume is extremely large and MySQL can't handle it?

Hot/Cold Separation: Keep only the last 3 months of hot data in MySQL, archive historical data to OSS or ClickHouse
Sharding: Shard databases and tables by target_id modulo to distribute write pressure
TiDB: Directly use a distributed database for automatic horizontal scaling

8.2 How to handle hotspot content (celebrity announcement)?

Multi-level Caching: Add a local Caffeine cache on the application server; if hit, return directly without accessing Redis
Hotspot Discovery: Use the HeavyKeeper algorithm to identify sudden hotspot Keys in real-time and automatically push them to the local cache
Rate Limiting and Degradation: Configure stricter rate limiting policies specifically for hotspot Keys, prioritizing system stability

8.3 How to ensure messages are not lost?

Producer side uses acks=all + callback retry
Consumer side manually commits Offset, only committing after business processing succeeds
Enable Kafka's idempotent producer (enable.idempotence=true)

9. Summary

The like system is small but complete. Through the full implementation in this article, we can see:

Design Principle	Specific Implementation
Read/Write Separation	Reads go to Redis, writes go to Redis first then async flush to DB
Atomic Operation	Lua scripts guarantee atomicity of multiple operations within Redis
Async Decoupling	Kafka asynchronizes the operation of writing to DB
Batch Processing	Batch consumption + Batch Upsert reduce DB pressure
Eventual Consistency	MQ + Scheduled reconciliation guarantee eventual data consistency
Anti-Fraud Governance	Sliding window rate limiting + user behavior frequency control

This solution has been running stably in multiple medium-scale projects, supporting tens of millions of daily active users. You can flexibly tailor it according to your own business scenarios—if concurrency is not high, you can even remove the MQ layer and directly use Spring's @Async for asynchronous DB writes; if the volume is extremely large, you need to introduce more refined hotspot governance strategies.

Comments

Top 2 from juejin.cn, machine-translated. The original thread is authoritative.

吃饱了得干活

[grin]

先吃饱再说

Awesome, the likes are so heavy that MySQL can't handle it [angry]