Sharding Is Dying? Why TiDB Pulls Distribution Back Into the Database
Preface
Have you ever encountered this situation: business data grows to hundreds of millions of rows, single-table queries start to slow down, and everyone's first reaction might be to do sharding and table splitting.
So you introduce ShardingSphere and split the orders table into 64 shards.
The result: you have to maintain sharding rules, manually stitch cross-shard queries, migrate data during expansion, and worry about master-slave delay for DDL on large tables.
So, the question is: can you solve the problem without sharding and table splitting?
Answer: you can use TiDB.
No need to manage sharding rules anymore. Add machines to scale out. Cross-database JOINs can be written directly. Analytical queries and transactional queries run through the same entry point.
Today's article is all about discussing TiDB with you. I hope it helps you.
For more project practice, visit the Java Commando Network: susan.net.cn/project
1. What exactly is TiDB?
Some friends might say: "Isn't TiDB just a domestic distributed database? What's the difference from sharding and table splitting?"
The difference is: sharding and table splitting is you "simulating" distribution at the application layer, while TiDB "is" distributed at the database level itself.
MySQL sharding and table splitting, in essence, adds a routing layer on top of single-machine databases.
Data is still stored in multiple independent MySQL instances. Cross-shard queries, distributed transactions, and global consistency all have to be solved by you at the application layer.
TiDB is a natively distributed database.
Its computing layer and storage layer are separated, and the storage layer itself is multi-replica and strongly consistent.
You write a SQL statement, and TiDB decides how to split, compute, and merge on its own. You don't need to tell it "how many shards this table should be split into."
To put it in one sentence: TiDB moves "distribution" from the application layer back to the database layer.
You face a logically complete database; how it is distributed behind the scenes is its own business.
2. One diagram to understand TiDB's architecture
TiDB's architecture can be divided into three layers:
Computing layer (TiDB Server): stateless SQL engine. It is responsible for parsing SQL, generating execution plans, sending requests to the storage layer, and summarizing results to return. Adding machines can increase computing power without downtime.
Storage layer (TiKV + TiFlash): TiKV is a row-store engine for high-concurrency read/write of OLTP. TiFlash is a column-store engine for complex analytical queries of OLAP. The two synchronize automatically and maintain strong consistency.
Scheduling layer (PD): responsible for cluster metadata management, load balancing, and timestamp allocation. Region scheduling and replica distribution are all decided by it.
3. Underlying principles
3.1 Automatic sharding: Region is TiDB's "shard"
With MySQL sharding and table splitting, you define the sharding rules.
TiDB doesn't require you to define them; it splits on its own.
TiDB divides data into Regions by key range, with each Region defaulting to roughly 96MB.
When a Region grows larger or becomes hot, PD automatically splits it into two and schedules them to different TiKV nodes.
What does this mean?
You don't need to predict "how large this table will grow or how many shards it should have."
TiDB automatically adjusts based on actual data volume and access pressure.
Scale-out is just adding TiKV nodes; PD automatically migrates Regions over, and data automatically balances.
3.2 Raft protocol: how strong consistency is guaranteed
Each Region in TiKV has multiple replicas, usually 3. Replicas synchronize using the Raft protocol.
When you write a piece of data, the Leader replica writes first, then syncs to Followers. The write only succeeds after a majority (2) acknowledges. If the Leader fails, Raft automatically elects a new Leader, and no data is lost.
This is the foundation of TiDB's high availability. A single machine failure triggers automatic Raft failover with RTO in seconds. No need to configure MHA or master-slave switch scripts like with MySQL.
3.3 HTAP: one system, two workloads
This is where TiDB diverges most from traditional sharding solutions.
After sharding, what do you do for real-time analysis? Usually you synchronize data to Hive or ClickHouse and get results T+1. The longer the sync pipeline, the greater the latency and the harder consistency becomes.
TiDB's approach: row-store TiKV and column-store TiFlash coexist, and TiFlash synchronizes from TiKV in real time via Raft Learner. You write one SQL; TiDB's optimizer automatically decides whether the query is OLTP and goes to TiKV, or OLAP and goes to TiFlash.
The Qingwen Group case is typical. They originally used MySQL sharding and table splitting, with offline analysis via Hive and T+1 results. After migrating to TiDB, a single database supports both online business and real-time analysis, with no need to maintain a sync pipeline.
4. Migrating from MySQL to TiDB: what changes?
4.1 Change the connection string
// Original MySQL connection
String url = "jdbc:mysql://mysql-host:3306/mydb";
// Switch to TiDB
String url = "jdbc:mysql://tidb-host:4000/mydb";
The port changes from 3306 to 4000, everything else remains basically the same.
TiDB is compatible with the MySQL protocol, so JDBC drivers, connection pool configuration, and most SQL syntax don't need to change.
4.2 CREATE TABLE statement: the AUTO_INCREMENT pitfall
TiDB supports AUTO_INCREMENT, but there is a detail to note: if a table has no explicit primary key, TiDB uses _tidb_rowid as the row identifier, which shares an allocator with the auto-increment column and can cause ID gaps.
-- Recommended: explicitly specify a primary key and consider using AUTO_RANDOM
CREATE TABLE orders (
id BIGINT PRIMARY KEY AUTO_RANDOM,
user_id BIGINT,
amount DECIMAL(10,2),
created_at DATETIME
);
AUTO_RANDOM is a TiDB-specific feature used to avoid write hotspots caused by auto-increment primary keys.
If your table has heavy write volume, use AUTO_RANDOM instead of AUTO_INCREMENT.
4.3 Queries: cross-database JOINs you can write directly
In the sharding era, cross-shard JOINs were either stitched at the application layer or used broadcast tables.
In TiDB, you just write:
SELECT o.id, u.name, o.amount
FROM orders o
JOIN users u ON o.user_id = u.id
WHERE o.created_at > '2026-01-01';
TiDB decides how to split, compute, and merge on its own.
You don't need to care about which Region the data lives on.
5. Use AI to analyze TiDB slow queries
TiDB's slow query log is structured. You can feed the slow query log and EXPLAIN results to AI and let it analyze "why this SQL is slow and what index to add."
Scenario: you have a query that runs slowly on TiDB, and EXPLAIN shows a full table scan.
Step one: paste the EXPLAIN result to AI.
"Here is my TiDB EXPLAIN result. This query scanned 2 million rows but returned only 10. Help me analyze why it didn't use an index and what index I should add."
AI will analyze the execution plan and find that the query condition uses WHERE status = 'PENDING' AND created_at > '2026-01-01', but the table only has a single-column index on created_at. It will suggest adding a composite index (status, created_at).
Step two: have AI verify.
"What should I pay attention to when adding a composite index in TiDB? Will it affect write performance?"
AI will tell you TiDB's DDL is online and doesn't lock the table, but adding an index consumes resources, so it's recommended to execute during off-peak hours. It will also remind you: if status has low selectivity, you can consider TiDB's partial index, which only indexes rows with status = 'PENDING', reducing index size.
What AI cannot do for you: determine "whether this index should actually be added" — perhaps your write volume is so high that the cost of adding an index outweighs the benefit of query optimization. AI helps you understand the execution plan; you make the trade-off.
6. Advantages and disadvantages
Advantages
1. No sharding and table splitting
Regions automatically split, schedule, and balance. Scale-out is adding nodes, and data migrates automatically. No need to maintain sharding rules.
2. MySQL compatibility, low migration cost
JDBC drivers, connection pools, and most SQL syntax don't need changes. Business code runs after changing a connection string.
3. HTAP: one system, two workloads
TiKV handles OLTP, TiFlash handles OLAP. No need to set up another sync pipeline to Hive/ClickHouse.
4. Financial-grade high availability
Raft multi-replica, majority acknowledgment, automatic failover. RTO in seconds, zero data loss.
5. Online DDL, no locking
DDL on tables with hundreds of millions of rows executes online without blocking business reads and writes.
6. Validated by major companies
Kimi.ai uses TiDB Cloud as the database infrastructure for its AI agents, creating isolated databases in under 1 second. Flipkart uses TiDB on Kubernetes to handle 1 million queries per second.
Disadvantages
1. Not 100% MySQL compatible
Stored procedures, triggers, events, user-defined functions, full-text indexes, and spatial data types are not supported by TiDB. If your business relies heavily on these, migration requires modification.
2. Different operational model
TiDB's operations differ from MySQL. You need to understand concepts like PD, Region, and Raft. Although TiUP and TiDB Operator simplify operations, there is a learning curve.
3. Write hotspots need handling
Auto-increment primary keys can easily create write hotspots in TiDB. You need to use AUTO_RANDOM or generate IDs at the application layer to spread writes.
4. Higher resource consumption than single-machine MySQL
TiDB is a distributed system; the smallest production deployment requires multiple nodes. For scenarios with small data volume and low concurrency, TiDB's resource consumption is not cost-effective.
7. Applicable scenarios
| Scenario | Recommendation | Reason |
|---|---|---|
| Large data volume, painful sharding operations | ✅✅✅ Strongly recommended | Region auto-sharding, no need to maintain sharding rules |
| Need HTAP (transaction + real-time analysis) | ✅✅✅ Strongly recommended | One system with two engines, no sync pipeline needed |
| Financial-grade high availability requirement | ✅✅✅ Strongly recommended | Raft multi-replica, automatic failover |
| AI agent database infrastructure | ✅✅✅ Strongly recommended | Validated by Kimi.ai, creates isolated databases in 1 second |
| Relies on stored procedures/triggers/full-text indexes | ⚠️ Needs evaluation | TiDB doesn't support these features |
| Small data volume, low concurrency | ❌ Not recommended | Resource consumption not cost-effective, MySQL is sufficient |
More project practice on the Java Commando Network: susan.net.cn/project
8. Final thoughts
Back to the original question: why are more and more people using TiDB?
Because it changes the question of "what to do when data volume gets large" from "how to split at the application layer" to "how the database distributes itself."
Sharding and table splitting solve the problem of "a single machine can't handle it," but they introduce new problems: sharding rules, cross-shard queries, distributed transactions, and expansion migration.
TiDB pulls these problems back into the database layer.
You face a logically complete database; how it distributes, replicates, and schedules is its own business.
HTAP is another capability where TiDB pulls ahead.
After sharding, doing real-time analysis requires building a sync pipeline and maintaining two systems. TiDB's TiFlash lets transactions and analysis run through the same entry point, with real-time consistency.
Of course, TiDB doesn't come without costs.
It's not 100% MySQL compatible — stored procedures, triggers, and full-text indexes are not supported. The operational model also differs; you need to understand PD, Region, and Raft.
For scenarios with small data volume and low concurrency, TiDB's resource consumption is not cost-effective.
But for teams whose data volume is too large for MySQL to handle and who don't want to maintain sharding, TiDB is a battle-tested, continuously iterated, MySQL-compatible choice.
Top 2 from juejin.cn, machine-translated. The original thread is authoritative.
Fine, then let's just use PolarDB.
TiDB is a heap of sponsored fluff [smirk]