Java · Spring Boot

Idempotency Under Fire: Load-Test Proof That Check-Then-Act Fails, and the Redis+Lua Layered Fix

By 踏浪的码者 · Oct 30, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Duplicate orders and duplicate payments are ticket-generating production incidents; this load-tested breakdown gives Western teams the actual numbers and the layered pattern—front-door atomic guard plus resource-layer constraint—that holds up under concurrency, not just a theory.

Summary

A 100-thread concurrency load test deliberately amplifies a 5ms gap between EXISTS and SET, and the non-atomic check-then-act implementation lets all 100 requests through while the atomic versions allow exactly 1. The same test at 200 concurrency with limit=5 shows SETNX still passes only 1 (placeholder semantics), while a Lua sliding window correctly passes 5. That concrete data is the foundation of the article's core claim: idempotency checks must be a single atomic operation.

The article then layers the defense: Redis SETNX with NX and expiration as a fast entry gate, Lua-based sliding window when you need "N times in a window," and a database unique index as the final resource-layer backstop because Redis keys can expire, fail over, or crash before the DB write. It walks through production boundaries like TTL expiry windows (where a duplicate can slip in if the business takes longer than the key lives), idempotency record tables as a persistent alternative, MQ at-least-once handling with unique constraints, why distributed locks are mutual exclusion but not idempotency, and conditional state updates for status-machine flows.

Takeaways

— A 100-thread load test with a non-atomic check-then-set implementation allowed 100 passes, completely breaking idempotency; the atomic SETNX and Lua sliding window both allowed exactly 1.

— With 200 threads and limit=5, SETNX still allowed only 1 because it is a placeholder/first-occurrence primitive, while the sliding window correctly allowed 5.

— Redis SETNX must be used with NX plus an expiration (PX/EX) to avoid a permanently stuck key after a crash.

— A database unique index enforces idempotency at commit time by the storage engine, with no race window, making it the strongest backstop after Redis entry interception.

— If a placeholder key expires before the business operation finishes (e.g., TTL 10s but operation takes 12s), duplicates can recur; TTL should exceed interface P99 by a factor, and strong consistency still relies on unique constraints.

— An idempotency record table using the business key as primary key and checking insert result is a persistent, auditable alternative when Redis is not available.

— MQ consumers should handle duplicates with a business unique key plus unique constraint (at-least-once semantics), not rely on the broker; ack only after successful consumption.

— Distributed locks guarantee mutual exclusion but not idempotency—replay after lock release can still succeed; locks protect composite critical sections, while final idempotency needs unique constraints or state machines.

— State conditional updates (e.g., UPDATE ... WHERE status='INIT') let only legal state transitions happen and ignore duplicate callbacks.

Conclusions

The load-test numbers turn a common warning about check-then-act into a measured fact: at 100 concurrency the race window let every request through, so the 'just add EXISTS' advice is demonstrably not theoretical.

The SETNX vs. sliding-window distinction is a clean reminder that idempotency primitives answer different questions: first-occurrence vs. count-in-window, and mixing them produces either over-rejection or under-protection.

The layered defense (entry atomic guard + resource-layer unique index) reflects a practical reality: any single fast mechanism has an expiry or failover edge, so the cost of duplicates must be caught at the database when it matters.

The TTL-expiry-before-business-finishes scenario is the kind of edge case that escapes casual code review; the proposed trio—generous TTL, unique index backstop, idempotency table for full lifecycle—treats it as a first-class design problem.

Concepts & terms

Idempotency

A property where executing the same operation once or multiple times produces the same final system state, written as f(f(x)) = f(x).

Check-then-act

A pattern that first checks a condition (e.g., key existence) and then decides to act; under concurrency, multiple threads can pass the check before any write completes, creating a race window that breaks idempotency.

SETNX

A Redis command (SET with NX flag) that atomically sets a key only if it does not already exist, optionally with expiration; used as a placeholder to ensure a given business key is handled only once.

Sliding window (rate limiting)

An algorithm that counts requests within a continuously moving time window, evicting expired timestamps before checking whether the count exceeds a limit, and requires atomic 'cleanup + count + enqueue' (done via Lua in Redis).

At-least-once delivery

A message delivery guarantee where a consumer may receive the same message more than once; since most MQ systems provide this rather than exactly-once, consumers must use idempotency mechanisms like unique constraints.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗