Announcing TiDB 6.5 LTS Release

We are proud to announce the TiDB 6.5 Long-Term Support (LTS) release. Compared with the 6.1 LTS release, TiDB 6.5 introduces more than 30 new features and over 60 enhancements and fixes. It delivers an enterprise-grade database that offers:

Continuously mature kernel capabilities
Increased ease of use
Improved developer experience

In this post, I will highlight some of these features and enhancements. If you’d like more details, see the TiDB Release Notes.

Continuously maturing kernel features

TiDB 6.5 puts a high priority on kernel feature sets. This ensures that TiDB continues to evolve in quality and stability.

Accelerated DDL

One of TiDB’s core strengths is support for online Data Definition Language (DDL). Over the past year, TiDB has added support for parallel DDL so that sequential DDL executions will not block each other in multi-tenant scenarios such as Software as a Service (SaaS). TiDB 6.5 also introduces Metadata Lock, which greatly mitigates the issue of DDL interfering with DML. Further, this release significantly improves index creation through optimizations on index creation mode of full data, data transmission, and parallel import. This enhancement accelerates DDL construction by 10 times.

As shown in the Sysbench result below, TiDB 6.5 speeds up DDL construction by 10 times compared with TiDB 6.1. It is also 2-3 times faster than CockroachDB v22.2 and the current version of AWS Aurora.

Comparing the acceleration ratios of fast DDLs

Better JSON support

JSON data types are widely adopted in agile businesses such as Web3 and gaming, with strong real-time data analysis requirements. However, due to its complexity, analyzing JSON data is very challenging. Since 6.2, TiDB has continuously introduced MySQL 5.7 compatible functions that include expression index support for JSON and other JSON-related tools. TiDB 6.5 supports pushing down JSON functions to TiFlash, including `->`, `->>`, and `JSON_EXTRACT()`. Not only does this approach make analyzing JSON data more efficient, it also expands the application scenarios of real-time analytics for TiDB.

Enhanced global memory control

Starting with TiDB 6.5, you can track main memory consumption. When the global memory consumption reaches the memory threshold defined in TiDB Server, TiDB will try to limit memory usage through garbage collection or canceling SQL operations. This ensures the stability of TiDB and makes memory usage more efficient.

Reduced performance compromise from pessimistic locks

In some transaction systems, especially in the banking business, applications will use SELECT FOR UPDATE to lock a record first in the pessimistic transaction. This protects data consistency and reduces the possibility of operation failure. However, repeatedly using SELECT FOR UPDATE on a row may degrade the query performance of this record. By recording lock marks, TiDB 6.5 greatly reduces the impact of accumulated pessimistic locks on query performance. It also increases queries per second (QPS) by more than 10 times.

Improved ease of use

TiDB started out as a solution to address the hassle of MySQL sharding. Ease of use has been a constant part of the TiDB development philosophy. In this new version, we continuously provide more capabilities in this area.

General availability of Flashback Cluster

Data manipulation (DML) misoperation in an online environment can easily lead to production accidents. The FLASHBACK CLUSTER TO TIMESTAMP feature introduced in TiDB 6.5 lets you quickly roll back the entire cluster to a specified point within the garbage collection lifetime.

Flashback Cluster supports fast cluster rollback within minutes without requiring any backup. It also allows multiple rollbacks on the timeline to minimize the data risk caused by DML misoperations.

Support for large transaction splits

When the database is processing data at a large scale, a single SQL statement with a large transaction might slow down performance and make the cluster unstable.TiDB 6.5 supports automatically splitting large transactions, with full support for non-transactional statements including DELETE, INSERT, REPLACE, and UPDATE. The split statements may compromise transaction atomicity and isolation but greatly improve the cluster performance.

Support for global monotonically increasing primary keys

TiDB 6.5 adds support for global monotonically increasing primary keys. In this mode, TiDB adopts a centralized primary key allocation method to ensure monotonic increasing for primary keys. IDs won’t fall back even—in cross-TiDB instance access scenarios. Writing can also easily reach tens of thousands of transactions per second (TPS).

TTL for row-level life cycle control (Experimental)

The Time to Live (TTL) capability in TiDB 6.5 allows users to set row-level life cycle control policies. By setting the TTL attribute for a table, TiDB will automatically check and clean up expired data in the table.

Diversified disaster recovery

In previous versions, TiDB mainly relied on Backup and Restore (BR) for static backup and recovery. However, this requires continuous full backups to allow restoring to multiple points in time. This is expensive and may affect online business. TiDB 6.2 introduced Point-in-Time Recovery (PITR). With TiDB PITR, users can restore data to any point in time through a full backup combined with incrementals. This eliminates the need for continuous full backups. TiDB 6.5 reduces the size and number of PITR backup files, and greatly improves stability and performance. In the log recovery phase, the recovery speed of a single TiKV node can reach 30 GiB/h, which is 150% faster than TiDB 6.2. The recovery point objective (RPO) in common scenarios is as short as 5 minutes.

TiDB PITR architecture

This release also improves by several times the throughput of the change data capture tool, TiCDC. When the downstream is a Kafka cluster, a single TiCDC node can allow a QPS of 35 k rows/s, with the throughput up to 50 MB/s, and the latency within two seconds. The memory usage is only about 50% of the previous version. In the disaster recovery scenario for cross-regional, active-standby clusters, the throughput of a TiCDC node can reach 30 MB/s, and the delay remains within two seconds. In the new version, TiCDC can generate a special timestamp when writing to the downstream. The downstream cluster will automatically use this timestamp for consistent reading.

Dig deeper in developer experience

Improving the developer experience has always been a key goal of TiDB. TiDB 6.5 greatly strengthens the ecosystem required for application development. With function additions such as Savepoint, User Level Lock, and Multiple Schema Change, TiDB now has more comprehensive support for application frameworks such as Django and Prisma. At the same time, PingCAP has also extended collaborations with more upstream and downstream partners, including Vercel, Hashicorp, and Retool.

Another feature for developers is the support of writing to object storage. This means TiCDC can synchronize data directly to object storage. This support makes it easy for users to achieve long-term data storage and cross-region replication with TiDB. It also enables TiDB to directly connect to data warehouses and data lakes in the cloud environment. Developers now have more freedom and flexibility when building applications using TiDB.

Conclusion

With the release of TiDB 6.5 LTS, users will have an easier-to-use and more mature enterprise-grade database. We hope these improvements inspire more developers to build applications on TiDB.

You can get started with TiDB 6.5 today or request a demo from our technical experts. Join our community on Slack and the TiDB Forum to share your thoughts and feedback.

Book a Demo