Announcing TiDB 6.2 with Better Observability

TiDB 6.2, the latest version of our Hybrid Transactional and Analytical Processing (HTAP) database was released on August 24, 2022. TiDB 6.2 has improved observability, performance, stability, ecosystem tooling, and MySQL compatibility. We have packed this release with some in-demand features, including:

A concurrent data definition language (DDL) execution framework
Point-in-time recovery (PITR)
A monitoring page for TiDB Dashboard
FastScan for TiFlash

In this post, I will highlight some of these enhancements. If you’d like more details, see the TiDB Release Notes.

Better support for multi-tenancy scenarios

Earlier versions of TiDB could only execute one schema change at a time. However, in SaaS and multi-tenancy situations, each tenant might have their own set of tables with different schemas. Large scale schema changes on a large scale could be a huge showstopper for the overall database performance.

TiDB 6.2 includes a new Data Definition Language (DDL) parallel execution framework to execute DDLs on different table objects concurrently. This solves the DDL blocking issue between different tables found in previous versions. TiDB 6.2 also lets you apply parallel execution in multiple scenarios such as appending indexes and column type changes for different table objects. It greatly improves the DDL execution efficiency, especially for multi-tenancy scenarios.

Enhanced disaster recovery

If your business has mission critical tasks, you must be able to restore your cluster to any point in time. TiDB 6.2 introduces point-in-time recovery (PITR), which lets you back up and recover data based on change logs or snapshot data to ensure data durability. PITR enables you to:

Reduce the Recovery Point Objective (RPO) for disaster recovery to about ten minutes.
Handle write errors by rolling back the data to a point before the error.
Meet industry compliance requirements during history data auditing.

The Backup and Restore (BR) tool can now recover user and permission data. This makes backup and recovery smoother. Users no longer need to deal with the user and permission information separately after cluster recovery.

In addition, TIDB 6.2 introduces a DDL filtering mechanism for TiCDC, the change data capture tool. With this feature, TiCDC can filter DDL events by type, and filter DML events by SQL expressions. And TiDB can adapt to more data replication scenarios; for example, it can filter TiDB incremental data change logs to sort out high-risk DDL statements like DROP TABLE.

Better observability for TiDB Dashboard

In TiDB 6.2, TiDB Dashboard supports visual execution plans. In the past, users could only troubleshoot TiDB issues by reviewing the text version of the execution plan. This is fine for simple transactional SQL processing, but not so friendly for analytical SQL statements that may generate massive execution plan data. In TiDB 6.2, the Dashboard provides visual execution plans for basic troubleshooting in the Statements and Slow Query sections. You can see each step of the query plan graphically so you can understand the execution process more intuitively. This is especially helpful for understanding complex and large-scale queries.

Visual execution plans

The TiDB Dashboard also has a new monitoring interface for performance analysis and optimization. This interface can help you analyze time-related metrics from a global or top-down perspective and confirm whether the response bottleneck is in the database. If this is the case, the Database Time View and the Latency Break Down View of SQL latency can help you locate the bottleneck quickly and perform optimizations accordingly.

Performance analysis on TiDB Dashboard

The Lock View in TiDB 6.2 can now show the blocking status of optimistic transactions. Lock conflict location is one way to troubleshoot performance degression in major lock conflicts. Prior to TiDB 6.2, users could only check lock conflicts through the system view of INFORMATION_SCHEMA.DATA_LOCK_WAITS. However, this approach doesn’t include information about optimistic transactions blocked by pessimistic locks. TiDB 6.2.0 extends the DATA_LOCK_WAITS view to cover this situation. This enhancement not only helps users quickly locate lock conflicts and provide directions for business improvement, but it also reduces the frequency of lock conflicts and improves system performance.

Strengthened HTAP capabilities

TiDB 6.2 further strengthens the database’s HTAP capabilities.

First off, the support for PageStorage V3, a new storage format, is generally available in TiDB 6.2. This format greatly mitigates high CPU usage issues caused by high concurrency and high workloads in garbage collection. Saving data in the PageStorage V3 format can greatly reduce I/O traffic for background tasks, and it improves the system stability in high concurrency scenarios. Starting with TiDB 6.2, PageStorage V3 is the default storage format.

Previously, to ensure data consistency, TiFlash checked consistency several times during data scanning to find the required data from multiple versions. However, this process slows down the query and isn’t necessary if your application doesn’t need strict accuracy, such as offline analytics. In TiDB 6.2, TiFlash provides the Fast Scan read mode. . This mode allows users to trade some degree of accuracy for faster scanning. With Fast Scan enabled, TiFlash ignores Multiversion Concurrency Control (MVCC) information and can scan the data up to three times faster.

Note: This feature is experimental and disabled by default.

As for the computing engine, TiFlash now supports fine-grained shuffle. Window functions pushed down to TiFlash can leverage multi-threaded parallel computing to significantly shorten the query response time, making it 4 to 5 times faster in typical scenarios. Through reduced data skew, the analytical engine improves the efficiency of the deduplication calculation through COUNT(DISCINT).

Improved stability

TiDB 6.2 is also packed with stability improvements.

Database admins have faced some long-standing problems. How can we balance the resource overhead between background and front-end operations? How do we collect information more efficiently without affecting the regular performance? Beginning in TiDB 6.2, TiDB supports setting the CPU usage of background requests through the TiKV configuration file. This limits the CPU usage of background operations such as automatic statistics collection and, in extreme cases, avoids resource contention. TiDB also supports the automatic adjustment of CPU usage. In addition, TiKV can dynamically adjust the CPU usage of background requests based on the instance’s CPU usage. This feature is disabled by default.

Now, TiFlash can better handle large data volumes. It reduces memory amplification for distributed transaction processing and therefore reduces its peak memory usage by more than 50% compared to the best usage scenarios before version 6.1. This minimizes the risk of memory contention for concurrent tasks in large-scale analytical scenarios.

Better MySQL compatibility

This release improves MySQL compatibility, by adding the SAVEPOINT mechanism. TiDB 6.2 also supports the use of a single ALTER TABLE statement to add, delete, or change multiple columns or indexes.

In some complex business scenarios, you may need to manage a large number of operations in a transaction and sometimes need to fall back on some of the operations within the transaction.

SAVEPOINT lets you name a save point through which you can flexibly control the fallback node within the transaction. This feature allows you to manage complex transactions and more diverse business designs.

Conclusion

This article shared some of the highlights of TiDB 6.2, a Development Milestone Release (DMR) with new features, improvements, and bug fixes. For more details, please refer to the TiDB Release Notes.

TiDB 6.2 is already available on TiDB Cloud Developer Tier.

You are welcome to get started with TiDB 6.2 or request a demo from us. Feel free to join our community on Slack and TiDB Internals to share your thoughts and feedback with us.

Book a Demo