Five Redis Distributed Lock Traps and the Go Code That Survives Them

In a microservices architecture, distributed locks are a powerful tool for solving "concurrent resource contention" (such as flash-sale inventory deduction, preventing duplicate submissions, etc.). Many developers think that implementing a distributed lock with Redis is just a matter of one line of SETNX. But in reality, the gap between a "toy code that can run" and a "production-grade high-availability component" is filled with countless hidden pitfalls.

Today, we will re-examine Redis distributed locks. We will not only fill in those 5 famous "blood pits," but also introduce Go's unique concurrency philosophy (such as context control, Goroutine leak prevention, and reentrant lock design without a Goroutine ID), and finally even discuss the soul-searching critique of Redis locks by distributed systems titan Martin Kleppmann.

Ready? Let's go!

Evolution 1: Basic Implementation and the Atomicity Trap (The Basics)

The beginner's approach: First SetNX, then Expire. If the program crashes in between (OOM or restart), the lock can never be released, resulting in a direct deadlock. The advanced approach: Use the atomic command SET key value NX EX time.

But in Go engineering practice, we must never write spaghetti code. A mid-to-senior Go developer first thinks of encapsulation and object-oriented design. We will use Go's classic Functional Options pattern to construct the lock object.

package redislock

import (
	"context"
	"crypto/rand"
	"encoding/hex"
	"errors"
	"time"

	"github.com/redis/go-redis/v9"
)

var ErrLockFailed = errors.New("failed to acquire lock")

// Client abstracts the Redis client for easy mock testing
type Client interface {
	SetNX(ctx context.Context, key string, value interface{}, expiration time.Duration) *redis.BoolCmd
	Eval(ctx context.Context, script string, keys []string, args ...interface{}) *redis.Cmd
}

type Options struct {
	Expiration time.Duration
	RetryWait  time.Duration
	MaxRetries int
}

type Option func(*Options)

// RedisLock is a production-grade distributed lock struct
type RedisLock struct {
	client Client
	key    string
	token  string // Unique lock identifier to prevent accidental deletion
	opts   Options
}

// NewRedisLock constructor
func NewRedisLock(client Client, key string, optFuncs ...Option) *RedisLock {
	opts := Options{
		Expiration: 30 * time.Second,
		RetryWait:  50 * time.Millisecond,
		MaxRetries: 3,
	}
	for _, f := range optFuncs {
		f(&opts)
	}
	
	return &RedisLock{
		client: client,
		key:    key,
		token:  generateToken(), // Generate unique Token
		opts:   opts,
	}
}

// generateToken generates a 16-byte random hex string
func generateToken() string {
	b := make([]byte, 16)
	rand.Read(b)
	return hex.EncodeToString(b)
}

---

Evolution 2: Safe Unlocking and Lua Scripts (Safe Unlock)

Pitfall: Goroutine A's lock times out, and Goroutine B acquires the lock. Goroutine A then finishes its task and executes DEL key, deleting Goroutine B's lock and causing a system avalanche. Expert solution: The token must be verified during unlocking, and the "check-and-delete" operation must be atomic.

In Go, we usually preload the Lua script (SCRIPT LOAD) or use Eval directly.

const unlockScript = `
if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end
`

// Unlock safely unlocks
func (l *RedisLock) Unlock(ctx context.Context) error {
	res, err := l.client.Eval(ctx, unlockScript, []string{l.key}, l.token).Result()
	if err != nil {
		return err
	}
	if n, ok := res.(int64); !ok || n == 0 {
		return errors.New("unlock failed: lock not held by this token")
	}
	return nil
}

---

Evolution 3: Lock Timeout and the Watchdog Mechanism (Watchdog)

Pitfall: The lock TTL is set to 30 seconds, but a slow query causes the business logic to run for 40 seconds. At the 30-second mark, the lock automatically expires, and a concurrency conflict occurs. Expert solution: Introduce a Watchdog background goroutine for automatic renewal.

As a mid-to-senior Go developer, writing background Goroutines requires considering lifecycle management and preventing Goroutine leaks. Never just write go func() { for {} }(). You must combine context and chan for graceful exit.

// Renewal Lua script
const renewScript = `
if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("pexpire", KEYS[1], ARGV[2])
else
    return 0
end
`

// startWatchdog starts the watchdog
func (l *RedisLock) startWatchdog(ctx context.Context, done chan struct{}) {
	// Renewal interval is typically 1/3 of the timeout duration
	ticker := time.NewTicker(l.opts.Expiration / 3)
	defer ticker.Stop()

	for {
		select {
		case <-ticker.C:
			// Execute renewal
			res, err := l.client.Eval(ctx, renewScript, []string{l.key}, l.token, int64(l.opts.Expiration/time.Millisecond)).Result()
			// If the lock has been released (returns 0), or an error occurs, the watchdog exits
			if err != nil || res.(int64) == 0 {
				return 
			}
		case <-done:
			// Business logic actively unlocks, notifying the watchdog to exit
			return
		case <-ctx.Done():
			// Parent Context cancelled, watchdog exits
			return
		}
	}
}

Note the select here listens on ctx.Done(), which is standard for production-grade code. If an entire HTTP request is aborted by the user, the business Context will be cancelled, and the watchdog must be able to perceive this and exit safely to prevent Goroutine leaks.

---

Evolution 4: The "Reentrant Lock" Paradox in Go

Pitfall: Method A acquires the lock and internally calls Method B, which also tries to acquire the same lock, causing a self-deadlock. Expert solution: In Java, reentrant locks are very simple because Java has ThreadLocal and ThreadID. But in Go, Goroutines do not expose an ID!

Many beginners try to hack the Goroutine ID using assembly, which is extremely dangerous in engineering and not officially recommended. Implementing a reentrant lock in Go usually involves two advanced approaches:

1. Token Passing: Pass the token returned by the lock acquisition to downstream functions, which use this token to prove they belong to the "same call chain."
2. Context Passing (Recommended): Store the lock identifier in context.Context and pass it transparently through the call chain.

If reentrancy must be implemented at the Redis level (referencing the Redisson approach), we need to use Redis's Hash structure.

• Key: Lock name
• Field: Unique goroutine identifier (can use uuid + trace TraceID)
• Value: Reentry count

-- Reentrant lock Lua script
local key = KEYS[1]
local thread_id = ARGV[1]
local ttl = ARGV[2]

if (redis.call('exists', key) == 0) then
    redis.call('hset', key, thread_id, 1)
    redis.call('pexpire', key, ttl)
    return 1
end

if (redis.call('hexists', key, thread_id) == 1) then
    redis.call('hincrby', key, thread_id, 1)
    redis.call('pexpire', key, ttl)
    return 1
end

return 0

---

Evolution 5: The Ultimate Question — Master-Slave Failover and Fencing Tokens

All the efforts above are flawless on a single-instance Redis. But production environments are all Redis Cluster or Master-Slave Sentinel architectures.

The ultimate pitfall: The Master node goes down, and asynchronous replication means the lock wasn't synced to the Slave. The new Master takes over, the lock is lost, and two goroutines acquire the lock simultaneously!

Many will say: "Use Redlock!"

But distributed systems expert Martin Kleppmann once wrote a fierce critique of Redlock. This is because Redlock heavily relies on server clock synchronization. If a Redis server experiences a clock jump, or a process undergoes a long GC pause, Redlock will still collapse.

The ultimate expert's solution: For financial-grade scenarios requiring absolute strong consistency, the expert's choice is:

1. Replace Redis: Use etcd or ZooKeeper, which are based on the Raft protocol and the CP model. etcd's Lease and Watch mechanisms are inherently designed for distributed coordination.
2. Introduce Fencing Tokens: No matter how perfect the lock is, we do not trust it. When acquiring the lock, we simultaneously obtain a monotonically increasing Token from a sequencer. When the goroutine carries data to operate on a database (or downstream system), it must include this Token. The downstream system rejects requests with lower-version Tokens through optimistic locking or unique indexes.

(The Fencing Token mechanism proposed by Martin Kleppmann is the ultimate defensive bottom line for solving distributed lock timeouts/failures)

---

Summary and Selection Advice

From writing a single line of SetNX to building a production-grade distributed lock, it reflects a Go developer's respect for system boundaries:

1. Basics: Atomic operations (SetNX EX) and Lua scripts are the bottom line.
2. Robustness: Use context to control timeouts, use Watchdog to prevent premature release, and resolutely prevent Goroutine leaks.
3. Architecture: Understand the peculiarities of Go's concurrency model (no GID), and implement reentrancy by passing context through Context.
4. Vision: Look beyond the limitations of Redis and understand the differences between the AP and CP models.

Technology selection advice:

• Routine business (such as limiting user operation frequency, ordinary e-commerce inventory deduction): Redis locks are sufficient. It is recommended to directly use the open-source library go-redsync/redsync.
• Core capital chains (such as account settlement): Honestly use etcd, and combine it with database optimistic locking (version number mechanism) as a fallback.

If you are using PHP for project development, you can also directly use the https://github.com/pudongping/wise-locksmith library, which implements distributed locks and Redlock. If you are using the Hyperf framework, you can also directly use the https://github.com/pudongping/hyperf-wise-locksmith library; the author has made specific compatibility adjustments.

I hope this article can be helpful to you~