WeChat Sports pushes the leaderboard at 10pm every night: how hard is it technically?

Every night a little after ten, WeChat pushes a WeChat Sports message, reporting today's step count and incidentally telling you to check the leaderboard. Everyone may have noticed a detail: the push time is not uniform. From my own perception, it drifts by about one minute after 10:00.

This message looks like it just 'reads a number and sends it', but at full user scale it involves at least three things: scheduling, computation, and sending. There is no public material on how WeChat implements this internally, so I won't guess at the actual implementation. We'll only analyze the problem from a learning perspective.

---

First, do the math

Assume 100 million daily active users (just an assumption; the real scale is not public), and all reminders are expected to be sent within 1 minute:

$$ \frac{10^8}{60} \approx 1.67 \times 10^6 $$

That's an average of 1.67 million per second. This is still the result spread over 1 minute; if all tasks were triggered at 22:00:00 exactly, 100 million would land in the same second.

Three problems: how do you schedule 100 million pending tasks? Each person's friend leaderboard is different, so how is it computed? Do all 100 million rankings really need to be precomputed and wait for people to look?

---

Difficulty 1: How do 100 million pushes get sent

Where a single-machine scheduled task gets stuck

The most direct approach is a task table with a remind_at column, and a scheduled task scans it every minute:

select uid from user_remind where remind_at <= now();

The scan itself is not a problem; with an index it can hold up. The problem is that after scanning, each one still has to be sent one by one. A few thousand per second on a single machine is already good. At 5,000/second:

$$ \frac{10^8}{5 \times 10^3} = 2 \times 10^4\ \text{s} \approx 5.5\ \text{h} $$

It would finish at 3:30am, and the machine cannot die in the middle. The task table itself also cannot handle it: the scan condition moves forward over time, which is naturally a hotspot; failures have no retry; scaling up means replacing the machine.

Scheduling only enqueues

Separate 'when to send' from 'how to send': when the time comes, the scheduler does only one thing, stuff this batch of uids into a queue, and the sending is left to consumers. The queue here is not decoupling upstream and downstream; it is buffering: upstream writes 100 million at once, downstream pulls according to its own ability, and the queue length first rises then falls.

The consumer side also needs rate limiting. The downstream channel has a rate ceiling; exceeding it gets batch rejections, and retries amplify the backlog, so rate limiting belongs on the consumer side.

Sharding and staggering

100 million tasks are split by uid % N to N scheduler instances, each instance scans only its own shard, and single-machine pressure drops to 1/N.

Then cut another layer: 1 minute = 60 seconds, and hash(uid) % 60 decides which second this batch of users gets sent in. That is 1.67 million per second, and the hash guarantees the distribution is roughly even, with no clustering.

Back to the start: what users can perceive is only 'received the notification a little after ten'; differences at the second level no one can see. How the sending times are arranged within that 1 minute is something we can decide ourselves.

	Single-machine scheduled task	Sharding + queue
Task storage	One row per task in the database	Queue message, replayable
Scaling	Replace with a better machine	Add consumers
Failure handling	Check logs and patch manually	Retry / dead-letter queue
Peak	All pressed into the exact second at the hour	Spread out by second

---

Difficulty 2: Every person's friend leaderboard is different

A global leaderboard means everyone sees the same ranking, for example 'top 100 steps on the whole site'. A ZSet fits exactly: ZINCRBY updates a score in $O(\log N)$, and ZREVRANGE takes the top M in $O(\log N + M)$. Storing 100 million members is about ten GB; that is a lot of memory, but the direction is right.

A friend leaderboard is not the same thing. What we look at is 'who among my friends walked the most'. Every person's member set is different, and so are the ranks. The rank given by a global ZSet cannot answer this.

Is one ZSet per person okay?

Assume an average of 200 friends:

$$ 10^8 \times 200 = 2 \times 10^{10} $$

Twenty billion entries, at tens of bytes each, is TB-level, and this is resident memory for data that is settled only once a day.

Storage is secondary; the trouble is writes. When one person's step count changes, the entry in all of that person's friends' leaderboards has to change, a 200x write amplification. And step counts are not written at 10pm; they are reported continuously during the day, so every increment becomes 200 operations.

Real-time sorting at query time is a different picture: take 200 friend uids, batch-fetch the step counts, then sort. The number of comparisons is on the order of $200 \times \log_2 200 \approx 1500$, a single query completes in milliseconds, and the write cost is zero.

Approach	One query	One step update	Space
One ZSet per person (precomputed)	$O(\log k)$, cheapest	$O(k \log k)$, must update k leaderboards	$O(Nk)$
Real-time sort at query	$O(k \log k)$	$O(1)$	none

k is not a fixed value

Which path to choose depends on friend count $k$ and access frequency $f$, and it depends on the distribution of $k$, not the average. The public statement about WeChat's friend limit is around 10,000. $k$ ranges from dozens to 10,000, with a long tail. If you design for an average of 200 and hit an account with 8,000 friends, the read cost goes up 40x directly.

There is another point that is easy to overlook: the people who actually enter the leaderboard are not the same as the friend count. People who have not enabled WeChat Sports do not participate, and people who have set their step count to private do not participate either.

So the workable approach is to handle it by scenario:

• Few friends and infrequent access: compute directly at query time, low cost, and no consistency problem
• Many friends, or accessed repeatedly: compute once and cache it, set a short TTL, and read it directly next time
• A small number of accounts with concentrated access and many friends: handle them separately

The tradeoff is not 'precompute or real-time'. It is whether this user's $k$ multiplied by access frequency $f$ is worth storing a result. When $k$ is small, storing it is wasteful; when $k$ is large, not computing it cannot hold up.

---

Difficulty 3: Does it really need to be precomputed?

The previous two sections both assume that these 100 million leaderboards need to be precomputed. But pushing and computing are two separate things, and at the product level they can be split.

If the reminder only contains 'you walked 8,324 steps today', no ranking is needed; it just reads the user's own data for the day. Only when it needs 'you rank 7th among your friends' does sorting become necessary, and that sorting can also be done when the user taps open.

The benefit depends on the open rate. Assume 10% of users will tap open (still just an assumption), that is 10 million sorts, an order of magnitude less than computing all 100 million. More importantly, the 90% who never tap open are never computed at all.

But on-demand computing is not free. It shifts the pressure from the sending side to the reading side and adds several new problems:

Problem	Manifestation	Common countermeasure
Read surge	In the minutes after the push, users tap open in a concentrated burst and QPS spikes sharply	Ingress rate limiting, queueing, skeleton screen
Cache cold start	The first batch of requests all go back to origin	Prewarm hotspots, merge requests
Dependency amplification	One request needs to read the friend list + batch-fetch step counts	Friend list local cache, batch API
Degradation	Cannot compute during the peak	Return only your own step count first, load the ranking later

One reminder here: sorting itself is not expensive. What is expensive is the two things needed before sorting: the friend list and each person's step count. So the focus of on-demand optimization is not sorting; it is those two reads.

No matter which path is chosen, the core design idea is the same: the most effective optimization is often not making computation faster, but making unnecessary computation not happen at all. Speeding up 100 million computations by 10x and computing only 10 million save the same order of magnitude, but the latter also saves machines and money.

---

Summary

1. For work that arrives in a concentrated burst, first use sharding and queues to spread it out, then talk about sending faster.
2. How a leaderboard is computed depends on the distribution of $k$ and the access frequency $f$, not on the feature name 'leaderboard'.
3. Load that can be eliminated by the business flow should not be endured by architecture.

But each of these comes with extra complexity: queues need operations, caches can be inconsistent, and on-demand computing needs to handle cold starts and degradation. At 10,000 users, a crontab plus an in-memory sort is enough, and an extra queue is just a burden. At 100 million, none of the complexity from the earlier steps can be avoided.