跪拜 Guibai
← Back to the summary

Zhiwen Group Drops MySQL Sharding for TiDB: One Cluster for Real-Time OLTP and OLAP

Zhiwen Group (Momo, Tantan): From MySQL Sharding to TiDB Real-Time Unified Architecture

Editor's Note

Zhiwen Group is a global internet company focused on social entertainment, operating domestic social products such as Momo and Tantan, as well as multiple social products covering overseas markets. This article is based on a live presentation by Li Xiang, a senior engineer at Zhiwen Group, at the TiDB Community event in Beijing, introducing Zhiwen Group's practice of moving from MySQL sharding to a TiDB real-time unified data platform.

Zhiwen Group's business spans domestic and overseas markets. Domestically, there are Momo and Tantan; overseas, there are products like SoulChill, Amar, and MiraiMind, covering regions including the Middle East, Europe, and Southeast Asia. A common characteristic of social businesses is a high requirement for real-time: when users post moments, send gifts, or place orders, the data must be immediately visible.

The original architecture ran on MySQL. Different business lines were deployed in different MySQL clusters. Large data volumes were handled with sharding and table splitting. Offline analytics used real-time data sync tools to move data to Hive, producing T+1 results. As overseas business grew rapidly, architectural bottlenecks emerged one by one.

Dozens of Clusters Across Related Businesses, Cross-Database Queries Were Once a Luxury

Li Xiang categorized the pain points of the original architecture into several types.

Architectural complexity kept increasing with business growth. MySQL sharding, message queues, middleware, offline data warehouses — the more components, the longer the chain, the more complex the dependencies between systems.

Real-time analytics capabilities could not keep up. "MySQL core data is imported into Hive through transfer tools, which takes time. As daily data volume grows, transfer latency increases. For some real-time business analysis scenarios, this can no longer satisfy requirements."

Data silos were a prominent problem. Different apps, different business lines had multiple MySQL clusters, making cross-database query consistency difficult to guarantee.

MySQL's own scalability also hit a bottleneck. For vertical scaling, single-machine CPU, memory, and storage have limits; the primary database's pressure kept increasing but was hard to expand. For horizontal scaling, "expanding from two shards to four shards requires modifying the front-end proxy routing rules, which severely slows product iteration." Large-table DDL causing master-slave latency and high operational costs were also long-standing issues.

One TiDB Cluster Achieves Mixed OLTP and OLAP Workloads

During the database selection process, Zhiwen Group primarily considered four aspects, ultimately choosing TiDB:

  1. HTAP integration capability. "One database can satisfy both some online business and real-time analytics needs."

  2. MySQL compatibility. Migrating from MySQL to TiDB, the business-level syntax and semantics require almost no changes; often only a connection string changes. "The migration cost is very low compared to other solutions."

  3. Flexible horizontal scaling. Expanding shards or scaling vertically in MySQL requires a master-slave switch, which impacts core businesses — during the switch, the primary database cannot accept writes for a period. With TiDB, to add compute resources, you scale the TiDB compute engine; to add storage, you scale the TiKV row-based storage engine; no switch is needed.

  4. Distributed SQL processing capability. With query optimizer and operator pushdown, the full cluster's compute power can be leveraged, solving the pain point where MySQL analytical queries are constrained by single-machine hardware resources.

Two Real Application Scenarios

Scenario One: Overseas Real-Time Unified Data Platform

Zhiwen Group's overseas business faced problems such as dispersed database deployments and difficult cross-database queries. The business wanted real-time data analysis and operational queries, but different business lines were deployed in different MySQL clusters, making cross-database query consistency difficult to guarantee.

TiDB was used to build the overseas real-time unified data platform, deployed across multiple availability zones, each with its own redundant resources. Data sync was done with TiDB DM — small tables use full plus incremental export; tables above TB level use Dumping to export full data then write into TiDB; incremental data goes through DM; data validation uses sync-diff-inspector.

Scenario Two: SPAM Moderation System

Zhiwen Group's moderation system has high requirements for real-time and availability — data unavailability affects SPAM (spam content detection and moderation system) and the security of other user data.

Previously with MySQL, if the primary database went down, recovery via backup to restore a node took a very long time. TiDB's horizontal scaling allows the team to simply scale TiKV to handle growth in moderation order volume.

Four Safety Measures in Migration

Zhiwen Group's migration emphasized a validation loop. Li Xiang shared four layers of protection.

First, structure validation. Compare upstream and downstream table structures, field types, index constraints. TiDB and MySQL are basically fully compatible, but pay attention to differences between clustered and non-clustered tables — when validating data, you can check primary key type and index structure consistency, focusing on key fields and indexes.

Second, data validation. A three-layer strategy: table row count comparison, sample data comparison, full data validation. "Full data validation is the most complex. Stored data can be compared with tools, but for incremental data, it's recommended to combine tool-based and business-side dual validation."

Third, business validation. This is particularly emphasized. "Incremental data validation requires tools alone are not enough. If core business data is incorrect, it may cause financial losses." Specific practices include comparing results of core business SQL queries, daily business reconciliation, and key metric comparison.

Fourth, two switchover plans. For non-core businesses, use an observation window mode — full plus incremental sync, latency catch-up, data validation, switchover, observe for 30 minutes to 1 hour, and roll back if problems arise. For core businesses, use a dual-write mode — enable dual writes to MySQL and TiDB, verify data consistency, switch read traffic to TiDB, and roll back to MySQL if needed. Li Xiang's advice: "The switchover window is for switching, not for migration — complete the migration and validation in advance."

Migration Experience Summary

Migration is more than data sync; risk control and switchover assurance are crucial. Li Xiang summarized three points from multiple migration practices.

  1. Evaluate migration cycles for large tables in advance. "The migration window might be a week, but some tables are tens or hundreds of TB. You must evaluate the migration cycle to avoid reaching the switchover window with data still not fully exported."

  2. Try to freeze DDL during migration. "Although DM supports Ghost or PT tools for schema changes, it's strongly recommended to coordinate with the business in advance to control DDL changes."

  3. Never stop monitoring latency. "Some large tables take a long time to sync. Latency monitoring must be in place to avoid situations where the sync link has broken but you are unaware."

Conclusion

When a social business faces both domestic and overseas markets, both transaction and moderation workloads, and both online and offline pipelines, Zhiwen Group replaced the traditional complex architecture with TiDB. This shortens not just query time, but the cycle from data generation to business decision-making.

If your business is experiencing similar changes — sharding getting more complex, T+1 data analysis lagging behind operational pace, cross-database queries becoming the norm — consider validating early by starting with one scenario.