A Danmaku System from Standalone to Million-User Clusters
1. First, understand what a danmaku system is
Simply put, a danmaku system allows a group of people to send and receive messages in real time within the same live room.
Imagine you are watching a live stream:
- You send a "666", and it must immediately be visible to everyone in the room.
- If there is a 3-second delay, everyone will already be saying "The host is so handsome" while you are still saying "What is the host doing?".
- If 100,000 people are online, the system cannot crash or lag.
There are two core things:
- How are messages quickly delivered to everyone? (Push mechanism)
- How are messages stored? (Storage design)
It's that simple. Let's solve them one by one.
2. How to send messages out in a flash?
2.1 Method 1: HTTP Polling (Keep asking)
The easiest solution for a beginner to think of:
Frontend: Server, is there a new danmaku?
Server: No.
Frontend: Server, is there a new danmaku?
Server: No.
Frontend: Server, is there a new danmaku?
Server: Yes, here you go.
This is called polling, like asking every second if a package has arrived.
What's the problem?
- 100,000 people online = 100,000 "any new messages?" requests per second.
- Most of the time the answer is "no", wasting server resources.
- Even asking once per second still has a 1-second delay.
Measured data: Under ten-thousand-level concurrency, the CPU overhead of HTTP polling is 3.2 times higher than WebSocket.
2.2 Method 2: WebSocket (Always connected)
A better way: Connect the phone call and stay online.
Frontend: Help me connect to the server (handshake)
Server: Okay, connected.
Frontend: I sent a danmaku.
Server: Received, I'll forward it to everyone.
Server: Zhang San says "666" (pushed directly)
Server: Li Si says "Hahaha" (pushed directly)
WebSocket is like a phone call; once connected, you don't hang up and can talk anytime.
Benefits:
- No need to keep asking; messages are pushed directly.
- Latency drops from seconds to milliseconds (< 300ms).
- Server pressure is greatly reduced.
Core metric comparison:
| Metric | HTTP Polling | WebSocket |
|---|---|---|
| End-to-end latency | 500ms+ | < 100ms |
| Server CPU usage | High (frequent connections) | Low (persistent connections) |
| Real-time capability | Poor | Good |
| Implementation complexity | Simple | Medium |
2.3 WebSocket Principle (One-sentence version)
WebSocket is based on TCP. It upgrades to a persistent connection through a single HTTP handshake, after which both parties can send data to each other at any time without repeating the handshake.
HTTP Handshake:
Client: I want to upgrade to WebSocket
Server: Agreed, protocol switching
WebSocket Communication:
Client: Send message anytime → Server
Server: Push message anytime → Client
3. Standalone Danmaku System (Writing from scratch)
3.1 Server-side core code (Spring Boot + Netty)
Netty is a high-performance network communication framework in the Java ecosystem, very suitable for building WebSocket servers.
First, import the dependency
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.53.Final</version>
</dependency>
Step 1: Start the WebSocket service
@Component
@Slf4j
public class WebSocketServer {
@Value("${websocket.port:8088}")
private int port;
@PostConstruct // Runs automatically when the project starts
public void start() throws InterruptedException {
// Netty's master-slave Reactor thread model
EventLoopGroup bossGroup = new NioEventLoopGroup(1); // Handles connections
EventLoopGroup workerGroup = new NioEventLoopGroup(); // Handles IO
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new WebSocketServerInitializer());
bootstrap.bind(port).sync();
log.info("WebSocket service started on port: {}", port);
}
}
Step 2: Initializer (Configure protocol)
public class WebSocketServerInitializer extends ChannelInitializer<SocketChannel> {
@Override
protected void initChannel(SocketChannel ch) {
ChannelPipeline pipeline = ch.pipeline();
// HTTP codec (for WebSocket handshake)
pipeline.addLast(new HttpServerCodec());
// HTTP aggregator
pipeline.addLast(new HttpObjectAggregator(65536));
// WebSocket protocol handler (upgrades HTTP to WebSocket)
pipeline.addLast(new WebSocketServerProtocolHandler("/ws"));
// Custom business handler
pipeline.addLast(new DanmakuHandler());
}
}
Step 3: Core business handler
@Component
@Slf4j
public class DanmakuHandler extends SimpleChannelInboundHandler<TextWebSocketFrame> {
// Room → Set of all user connections in that room
private static final Map<String, Set<Channel>> ROOM_CHANNELS = new ConcurrentHashMap<>();
// Connection → User ID
private static final Map<Channel, String> CHANNEL_USER = new ConcurrentHashMap<>();
// Connection → Room ID
private static final Map<Channel, String> CHANNEL_ROOM = new ConcurrentHashMap<>();
@Autowired
private DanmakuService danmakuService;
/**
* Triggered when a message is received (core method)
*/
@Override
protected void channelRead0(ChannelHandlerContext ctx, TextWebSocketFrame frame) {
String text = frame.text();
Channel channel = ctx.channel();
try {
// Parse JSON message
DanmakuMessage msg = JSON.parseObject(text, DanmakuMessage.class);
switch (msg.getType()) {
case "join": // Join room
handleJoin(channel, msg);
break;
case "danmaku": // Send danmaku
handleDanmaku(channel, msg);
break;
case "ping": // Heartbeat
sendPong(channel);
break;
default:
log.warn("Unknown message type: {}", msg.getType());
}
} catch (Exception e) {
sendError(channel, "Message format error");
}
}
/**
* User joins a room
*/
private void handleJoin(Channel channel, DanmakuMessage msg) {
String roomId = msg.getRoomId();
String userId = msg.getUserId();
// 1. Record user info
CHANNEL_USER.put(channel, userId);
CHANNEL_ROOM.put(channel, roomId);
// 2. Join room connection pool
ROOM_CHANNELS.computeIfAbsent(roomId, k -> ConcurrentHashMap.newKeySet())
.add(channel);
// 3. Push recent danmaku (new users see historical messages)
List<DanmakuMessage> history = danmakuService.getRecent(roomId, 50);
for (DanmakuMessage h : history) {
sendMessage(channel, h);
}
// 4. Send system welcome
sendMessage(channel, "System", "Welcome to the live room");
log.info("User {} joined room {}", userId, roomId);
}
/**
* Handle danmaku message
*/
private void handleDanmaku(Channel channel, DanmakuMessage msg) {
String roomId = msg.getRoomId();
String userId = msg.getUserId();
String content = msg.getContent();
// 1. Sensitive word filtering
if (danmakuService.containsSensitive(content)) {
sendError(channel, "Content contains sensitive words");
return;
}
// 2. Anti-spam rate limiting (max 5 per person per second)
if (!danmakuService.checkRateLimit(userId)) {
sendError(channel, "Sending too frequently, please try again later");
return;
}
// 3. Complete the message
msg.setTimestamp(System.currentTimeMillis());
msg.setSender(userId);
// 4. Async storage
danmakuService.saveAsync(msg);
// 5. Broadcast to everyone in the room (except self)
broadcastToRoom(roomId, msg, channel);
}
/**
* Broadcast to everyone in the room (except the sender)
*/
private void broadcastToRoom(String roomId, DanmakuMessage msg, Channel exclude) {
Set<Channel> channels = ROOM_CHANNELS.get(roomId);
if (channels == null || channels.isEmpty()) return;
String json = JSON.toJSONString(msg);
for (Channel ch : channels) {
// Send to everyone except the sender
if (ch != exclude && ch.isActive()) {
ch.writeAndFlush(new TextWebSocketFrame(json));
}
}
}
/**
* Clean up resources when connection is disconnected
*/
@Override
public void handlerRemoved(ChannelHandlerContext ctx) {
Channel channel = ctx.channel();
String roomId = CHANNEL_ROOM.remove(channel);
String userId = CHANNEL_USER.remove(channel);
if (roomId != null) {
Set<Channel> channels = ROOM_CHANNELS.get(roomId);
if (channels != null) {
channels.remove(channel);
if (channels.isEmpty()) {
ROOM_CHANNELS.remove(roomId);
}
}
}
log.info("User {} disconnected", userId);
}
private void sendMessage(Channel channel, DanmakuMessage msg) {
if (channel.isActive()) {
channel.writeAndFlush(new TextWebSocketFrame(JSON.toJSONString(msg)));
}
}
private void sendMessage(Channel channel, String sender, String content) {
DanmakuMessage msg = new DanmakuMessage();
msg.setType("system");
msg.setSender(sender);
msg.setContent(content);
sendMessage(channel, msg);
}
private void sendError(Channel channel, String error) {
DanmakuMessage msg = new DanmakuMessage();
msg.setType("error");
msg.setContent(error);
sendMessage(channel, msg);
}
private void sendPong(Channel channel) {
DanmakuMessage pong = new DanmakuMessage();
pong.setType("pong");
sendMessage(channel, pong);
}
}
The code looks long, but the core is just three things:
- Someone joins → Record which room they are in, push historical messages.
- Someone sends a danmaku → Check sensitive words and spam, store it, broadcast it.
- Someone leaves → Clean up the records in the room.
3.2 Frontend connection (Browser natively supports WebSocket)
// 1. Connect to the danmaku server
const ws = new WebSocket('wss://your-domain:8088/ws');
// 2. Join the room after connecting
ws.onopen = function() {
ws.send(JSON.stringify({
type: 'join',
roomId: 'live_1001',
userId: getUserId() // Get from login info
}));
};
// 3. Receive danmaku → Display on screen
ws.onmessage = function(event) {
const data = JSON.parse(event.data);
if (data.type === 'danmaku') {
showDanmaku(data.content, data.color, data.sender);
}
};
// 4. Send danmaku
function sendDanmaku(text) {
ws.send(JSON.stringify({
type: 'danmaku',
roomId: 'live_1001',
userId: getUserId(),
content: text
}));
}
// 5. Auto-reconnect on disconnect
ws.onclose = function() {
setTimeout(() => {
connect(); // Reconnect after 3 seconds
}, 3000);
};
3.3 How many people can a single machine support?
Implemented with Netty, a single server (4 cores 8GB RAM) can support approximately 50,000-100,000 WebSocket connections.
What if there are more people? Add more machines!
4. Cluster Deployment: What if one machine isn't enough?
4.1 Problem: How do messages cross servers?
Suppose you have 3 servers, and users are distributed across different machines:
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Server 1 │ │ Server 2 │ │ Server 3 │
│ User A │ │ User B │ │ User C │
└─────────────┘ └─────────────┘ └─────────────┘
User A sends a danmaku on Server 1. How does User B on Server 2 receive it?
A message middleware is needed for "broadcasting"!
4.2 Solution Comparison
| Solution | Implementation | Reliability | Performance | Applicable Scenario |
|---|---|---|---|---|
| Redis Pub/Sub | Like a loudspeaker broadcast | Lower | Extremely High | Small scale, acceptable message loss |
| Message Queue (Kafka/RocketMQ) | Like a courier station distribution | High | High | Recommended for production environments |
4.3 Solution 1: Redis Pub/Sub (Simple version)
Server 1: Shouts through a loudspeaker "User A says 666"
Redis: Forwards to all subscribed servers
Server 2: Receives, forwards to User B
Server 3: Receives, forwards to User C
@Component
public class RedisMessageRouter {
@Autowired
private StringRedisTemplate redisTemplate;
// Publish message
public void publish(String roomId, DanmakuMessage msg) {
String channel = "danmaku:room:" + roomId;
redisTemplate.convertAndSend(channel, JSON.toJSONString(msg));
}
// Subscribe to messages (each service instance subscribes on startup)
@PostConstruct
public void subscribe() {
// Use MessageListener to listen to all "danmaku:room:*" channels
}
}
Pros: Simple implementation, fast speed. Cons: If a server goes offline, messages during the reconnection period are lost.
4.4 Solution 2: Message Queue (Recommended)
Using RocketMQ or Kafka, every server can receive the full set of messages:
@Component
public class DanmakuMQProducer {
@Autowired
private RocketMQTemplate rocketMQTemplate;
public void sendDanmaku(DanmakuMessage msg) {
// Broadcast mode: all consumers can receive
rocketMQTemplate.convertAndSend("danmaku_topic", msg);
}
}
@Component
public class DanmakuMQConsumer {
@RocketMQMessageListener(topic = "danmaku_topic",
consumerGroup = "danmaku_group",
messageModel = MessageModel.BROADCASTING)
public class Consumer implements RocketMQListener<DanmakuMessage> {
@Override
public void onMessage(DanmakuMessage msg) {
// Every server receives this, then broadcasts to its local users
danmakuHandler.broadcastLocal(msg.getRoomId(), msg);
}
}
}
Pros: Reliable messages, supports retries, can handle peak loads. Cons: Introduces a new component, slightly higher operational cost.
💡 Selection advice:
- For small projects (< 100k users), Redis is sufficient.
- For large projects (million-level), use RocketMQ/Kafka.
5. How to store danmaku? Hot-cold separation
5.1 Data characteristics
Danmaku has a characteristic: the newer it is, the more important; the older it is, the less anyone looks at it.
- Most recent 500: New users need to see these.
- 1 hour ago: Basically no one scrolls back to look.
- All history: Used for data analysis after the live stream ends.
5.2 Two-tier storage architecture
User sends danmaku
│
▼
┌─────────────────────┐
│ Hot Data: Redis │ ← Stores the most recent 500, millisecond-level read/write
│ ZSet sorted by time│
│ Expiry: 1 hour │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Cold Data: MySQL │ ← Stores all history, written asynchronously
│ Queryable history │
└─────────────────────┘
5.3 Redis Storage (Hot Data)
Using ZSet (Sorted Set), with the timestamp as the score, naturally sorted by time:
@Component
public class DanmakuStorage {
@Autowired
private StringRedisTemplate redisTemplate;
private static final String KEY_PREFIX = "danmaku:room:";
private static final int MAX_RECENT = 500; // Max 500 per room
/**
* Store danmaku (hot data)
*/
public void saveRecent(String roomId, DanmakuMessage msg) {
String key = KEY_PREFIX + roomId;
String value = JSON.toJSONString(msg);
double score = (double) msg.getTimestamp();
// Store in ZSet
redisTemplate.opsForZSet().add(key, value, score);
// Keep only the most recent 500
redisTemplate.opsForZSet().removeRange(key, 0, -MAX_RECENT - 1);
// Auto-cleanup after 1 hour
redisTemplate.expire(key, 1, TimeUnit.HOURS);
}
/**
* Get recent danmaku (pushed when a new user joins)
*/
public List<DanmakuMessage> getRecent(String roomId, int limit) {
String key = KEY_PREFIX + roomId;
Set<String> values = redisTemplate.opsForZSet()
.reverseRange(key, 0, limit - 1); // Newest first
return values.stream()
.map(json -> JSON.parseObject(json, DanmakuMessage.class))
.collect(Collectors.toList());
}
}
5.4 MySQL Storage (Cold Data)
All danmaku are eventually stored in the database:
CREATE TABLE danmaku_record (
id BIGINT PRIMARY KEY AUTO_INCREMENT,
room_id VARCHAR(64) NOT NULL COMMENT 'Room ID',
user_id VARCHAR(64) NOT NULL COMMENT 'User ID',
content VARCHAR(500) NOT NULL COMMENT 'Danmaku content',
color VARCHAR(20) DEFAULT '#FFFFFF' COMMENT 'Color',
timestamp BIGINT NOT NULL COMMENT 'Send timestamp',
create_time DATETIME DEFAULT CURRENT_TIMESTAMP,
KEY idx_room_time (room_id, timestamp),
KEY idx_user (user_id)
) ENGINE=InnoDB;
5.5 Asynchronous Persistence (Key Optimization)
Absolutely do not write to MySQL synchronously! Writing every danmaku to MySQL will instantly overwhelm the database.
The correct approach:
User sends danmaku
│
▼
Store in Redis (Fast! 1ms completion)
│
▼
Tell user "Sent successfully"
│
▼
Drop danmaku into message queue (Async)
│
▼
Consumer fetches in batches (1000 per batch)
│
▼
Batch insert into MySQL at once
@Component
public class DanmakuPersistConsumer {
@Autowired
private DanmakuRecordMapper recordMapper;
@RocketMQMessageListener(topic = "danmaku_persist_topic",
consumerGroup = "persist_group")
public class Consumer implements RocketMQListener<List<DanmakuMessage>> {
@Override
public void onMessage(List<DanmakuMessage> messages) {
// Batch conversion
List<DanmakuRecord> records = messages.stream()
.map(this::convert)
.collect(Collectors.toList());
// Batch insert (1000 per batch)
for (int i = 0; i < records.size(); i += 1000) {
int end = Math.min(i + 1000, records.size());
recordMapper.batchInsert(records.subList(i, end));
}
}
}
}
6. Anti-spam and Sensitive Words
6.1 Rate Limiting: Prevent Spamming (Sliding Window)
Max 5 messages per person per second, implemented using Redis sliding window:
@Component
public class RateLimiter {
@Autowired
private StringRedisTemplate redisTemplate;
public boolean allow(String userId, int windowSeconds, int maxRequests) {
long now = System.currentTimeMillis();
long windowStart = now - windowSeconds * 1000L;
String key = "ratelimit:danmaku:" + userId;
// Lua script ensures atomicity
String lua =
"redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, ARGV[1]) " +
"local cnt = redis.call('ZCARD', KEYS[1]) " +
"if cnt < tonumber(ARGV[2]) then " +
" redis.call('ZADD', KEYS[1], ARGV[3], ARGV[4]) " +
" redis.call('EXPIRE', KEYS[1], ARGV[5]) " +
" return 1 " +
"else return 0 end";
String member = now + "-" + UUID.randomUUID().toString().substring(0, 6);
Long result = redisTemplate.execute(
new DefaultRedisScript<>(lua, Long.class),
Collections.singletonList(key),
String.valueOf(windowStart),
String.valueOf(maxRequests),
String.valueOf(now),
member,
String.valueOf(windowSeconds * 2)
);
return result != null && result == 1L;
}
}
6.2 Sensitive Word Filtering (DFA Algorithm)
The core idea of DFA (Deterministic Finite Automaton): Build sensitive words into a tree, and a single scan of the text can match all sensitive words.
Sensitive word library: ["porn", "gambling", "violence"]
Built into a tree:
Root
/ \
P G V
| | |
O A I
| | |
R M O
| | |
N B L
| |
L E
| |
I N
| |
N C
| |
G E
@Component
public class SensitiveWordFilter {
private final Map<Character, Map> dfaMap = new HashMap<>();
@PostConstruct
public void init() {
// Load sensitive word library (from database or config file)
List<String> words = Arrays.asList("porn", "gambling", "violence", "politics");
buildDFA(words);
}
/**
* Build DFA tree
*/
private void buildDFA(List<String> words) {
for (String word : words) {
Map<Character, Map> currentMap = dfaMap;
for (char c : word.toCharArray()) {
currentMap = currentMap.computeIfAbsent(c, k -> new HashMap<>());
}
// End marker
currentMap.put('End', new HashMap<>());
}
}
/**
* Check if text contains sensitive words (O(n) complexity)
*/
public boolean contains(String text) {
if (text == null || text.isEmpty()) return false;
for (int i = 0; i < text.length(); i++) {
Map<Character, Map> currentMap = dfaMap;
for (int j = i; j < text.length(); j++) {
char c = text.charAt(j);
Map nextMap = currentMap.get(c);
if (nextMap == null) break;
if (nextMap.containsKey('End')) {
return true; // Matched a sensitive word
}
currentMap = nextMap;
}
}
return false;
}
}
7. Frontend Display of Danmaku (Canvas Rendering)
7.1 Comparison of Two Approaches
| Approach | Implementation Difficulty | Performance | Applicable Scenario |
|---|---|---|---|
| DOM Rendering | Simple | Poor (lags with many danmaku) | Low danmaku volume |
| Canvas Rendering | Medium | Good | High danmaku volume (Recommended) |
7.2 Core Canvas Rendering Code
class DanmakuRenderer {
constructor(canvas) {
this.canvas = canvas;
this.ctx = canvas.getContext('2d');
this.danmakuList = [];
this.tracks = []; // Track occupancy status
this.running = false;
}
/**
* Add danmaku
*/
addDanmaku(data) {
// Find an empty track
const track = this.findAvailableTrack();
const danmaku = {
text: data.content,
color: data.color || '#FFFFFF',
size: data.size || 24,
x: this.canvas.width,
y: track * 30 + 30,
speed: 2 + Math.random() * 1.5, // Random speed
track: track
};
this.danmakuList.push(danmaku);
this.tracks[track] = true;
}
/**
* Find an available track
*/
findAvailableTrack() {
const totalTracks = Math.floor(this.canvas.height / 30);
for (let i = 0; i < totalTracks; i++) {
if (!this.tracks[i]) return i;
}
return 0; // Reuse the first one if all are full
}
/**
* Animation loop
*/
start() {
this.running = true;
this.loop();
}
loop() {
if (!this.running) return;
// Clear canvas
this.ctx.clearRect(0, 0, this.canvas.width, this.canvas.height);
const toRemove = [];
for (let i = 0; i < this.danmakuList.length; i++) {
const d = this.danmakuList[i];
d.x -= d.speed; // Move left
// Draw danmaku
this.ctx.font = `${d.size}px "Microsoft YaHei"`;
this.ctx.fillStyle = d.color;
this.ctx.shadowColor = 'rgba(0,0,0,0.5)';
this.ctx.shadowBlur = 4;
this.ctx.fillText(d.text, d.x, d.y);
// Recycle track if moved off screen
const width = this.ctx.measureText(d.text).width;
if (d.x + width < 0) {
this.tracks[d.track] = false;
toRemove.push(i);
}
}
// Remove disappeared danmaku
for (let i = toRemove.length - 1; i >= 0; i--) {
this.danmakuList.splice(toRemove[i], 1);
}
requestAnimationFrame(() => this.loop());
}
}
// Usage
const renderer = new DanmakuRenderer(document.getElementById('canvas'));
renderer.start();
// When receiving danmaku
ws.onmessage = function(event) {
const data = JSON.parse(event.data);
if (data.type === 'danmaku') {
renderer.addDanmaku(data);
}
};
8. Three Performance Optimization Techniques
8.1 Batch Push (Reduce IO)
Pushing every single danmaku upon receipt incurs high IO overhead under high concurrency.
Optimization: Accumulate a batch every 50ms and push once.
@Component
public class BatchPushScheduler {
private final Map<String, List<DanmakuMessage>> pending = new ConcurrentHashMap<>();
private final ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();
@PostConstruct
public void init() {
// Execute batch push every 50ms
scheduler.scheduleAtFixedRate(this::flush, 50, 50, TimeUnit.MILLISECONDS);
}
private void flush() {
for (Map.Entry<String, List<DanmakuMessage>> entry : pending.entrySet()) {
String roomId = entry.getKey();
List<DanmakuMessage> messages = entry.getValue();
if (messages.isEmpty()) continue;
// Batch push to everyone in the room
danmakuHandler.broadcastBatch(roomId, messages);
messages.clear();
}
}
public void add(String roomId, DanmakuMessage msg) {
pending.computeIfAbsent(roomId, k -> new ArrayList<>()).add(msg);
}
}
8.2 Local Cache (Reduce Redis Access)
For popular live rooms, cache recent danmaku locally on the application server:
@Component
public class LocalDanmakuCache {
// Caffeine local cache, 5-second expiry
private final Cache<String, List<DanmakuMessage>> cache = Caffeine.newBuilder()
.maximumSize(1000) // Max 1000 rooms
.expireAfterWrite(5, TimeUnit.SECONDS) // Expire after 5 seconds
.build();
public List<DanmakuMessage> get(String roomId) {
return cache.getIfPresent(roomId);
}
public void put(String roomId, List<DanmakuMessage> messages) {
cache.put(roomId, messages);
}
}
8.3 Hotspot Identification and Isolation
Scenarios like matches or celebrity streams can cause a single live room to suddenly have millions of users.
Strategies:
- Allocate dedicated server resources for popular live rooms.
- Degrade non-core features (like danmaku colors, emojis).
- Enable rate limiting, prioritizing core danmaku sending.
9. Architecture Evolution Roadmap
A danmaku system is not built in one step; it can evolve gradually based on business growth:
| Stage | User Scale | Technical Solution | Characteristics |
|---|---|---|---|
| MVP Stage | < 10k | HTTP Polling + Standalone | Quick launch, simple implementation |
| Growth Stage | 10k - 100k | WebSocket + Standalone | Introduces persistent connections, good real-time capability |
| Maturity Stage | 100k - 1M | WebSocket Cluster + Redis + MQ | Message routing, async decoupling |
| Large Scale | 1M+ | Multi-datacenter + Service Mesh | Global acceleration, disaster recovery, multi-active |
10. Frequently Asked Questions
Q1: What if the WebSocket disconnects?
The frontend listens for the onclose event and auto-reconnects:
ws.onclose = function() {
setTimeout(() => connect(), 3000); // Reconnect after 3 seconds
};
Q2: What if the message queue loses messages?
The message queue itself has a retry mechanism, supplemented by a scheduled reconciliation task:
@Scheduled(cron = "0 0 3 * * ?") // Execute at 3 AM
public void reconcile() {
// Compare data in Redis and MySQL, correct based on DB if inconsistent
}
Q3: Does it matter if one or two danmaku are lost?
Not really. Danmaku has low reliability requirements; losing a few doesn't affect the experience. The key is the system doesn't crash and latency is low.
Q4: How many people can one server support?
A single Netty server (4 cores 8GB RAM) can handle approximately 50,000-100,000 connections. It depends on the message volume.
Q5: Should I use Go or Java?
Both Java (Netty) and Go (gorilla/websocket) can build a good danmaku system. Choose the tech stack your team is familiar with.
11. Summary
A danmaku system seems complex, but the core is just three things:
| Problem | Solution | Key Technology |
|---|---|---|
| How to push in real time? | Use WebSocket persistent connections | Netty + WebSocket protocol |
| How to broadcast in a cluster? | Forward via message middleware | Redis Pub/Sub / Kafka |
| How to store data? | Hot-cold separation + async persistence | Redis (Hot) + MySQL (Cold) + MQ |
Remember three "Don'ts":
- Don't use HTTP polling (wastes resources, high latency).
- Don't write to MySQL synchronously (will block the main process).
- Don't ignore anti-spam (will be overwhelmed by spam).
That's what a complete danmaku system is all about. Start from the simplest standalone version, and gradually evolve to a cluster version as users grow, with a clear path at every step.
If you found this helpful, please give it a like! Questions are welcome in the comments～ 👇
(Special thanks to ```@先吃饱再说``` for providing the frontend code)
Top 1 of 2 from juejin.cn, machine-translated. The original thread is authoritative.
[strength][strength][strength]
The big boss has arrived [seductive]