A Broken Snowflake ID Generator Caused Duplicate Order Numbers in Production
A single bad ID generator can silently corrupt orders, payments, and audit trails across an entire distributed system. The fix is trivial — swap in a battle-tested library — but the blast radius of getting it wrong is permanent data damage.
A production outage traced back to a hand-rolled Snowflake ID generator that collapsed under its own design flaws. The timestamp field was truncated to 31 bits, cycling every 24.85 days and wrapping repeatedly since the 2018 epoch. Worker and data center IDs were never configured, leaving every instance at zero, while the business ID was derived from the last octet of an IP address — a value that collided across nodes.
The combination of a short timestamp window, absent node differentiation, and a weak uniqueness factor produced full ID collisions that broke core order and transaction flows. The incident is a case study in why distributed ID generation is not a casual weekend project.
Mature libraries like Hutool and Baomidou handle clock rollback, bit layout, and worker coordination out of the box. For larger deployments, worker ID assignment can scale from simple config files up to Redis- or Zookeeper-backed coordination, but the baseline rule is the same: use a proven implementation and verify its uniqueness guarantees before it reaches production.
The failure mode is especially dangerous because it is silent: IDs look valid right up until they collide, and by then the corrupted data is already written.
Custom Snowflake implementations often break on the timestamp field first, because developers underestimate how quickly a shortened bit width wraps in real calendar time.
Deriving uniqueness from IP address segments is a recurring anti-pattern in distributed ID generation; it assumes a network topology that rarely holds in containerized or cloud environments.
The post's worker-ID scaling ladder — config file, IP+port hash, registry, coordination service — is a practical maturity model that maps directly to infrastructure complexity without over-engineering early.