In a previous post, our CTO Ed Huang shared his thoughts about the future of databases. We're glad that we've always been on the journey to “building a better database.” It has been more than nine months since we released TiDB 3.0. On April 8, PingCAP's fifth anniversary, we released the first release candidate (RC) for our landmark TiDB 4.0. With a set of groundbreaking tools that makes TiDB easier to use than ever, TiDB 4.0 enhances its Hybrid Transactional/Analytical Processing (HTAP) capabilities and continues to solve issues for large-scale, distributed SQL databases.
In this post, I'll walk you through TiDB 4.0's highlights across its deployment, use, operations and maintenance, ecosystem, and cloud services. You're welcome to join us in TiDB Community Slack to give us advice or feedback on your user experience.
“How long does it take to deploy a TiDB cluster on a single machine? “ Previously, we actually had a hard time answering this question, but now we can proudly say, “Just one minute.”
Why is it so fast? It's because TiDB 4.0 will introduce TiUP, a component manager that streamlines installing, configuring, and managing a TiDB cluster into a few easy commands.
To install a TiDB cluster for testing on a single machine, you only need to perform two steps:
Install TiUP. On your terminal, run the following command:
curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
Start a TiDB cluster on a single machine. Run
Then, you can connect a MySQL client to the TiDB cluster to begin your testing.
“How long does it take to deploy a 10-node TiDB cluster for production?" The answer is also “One minute.”
To deploy a TiDB cluster in a production environment:
Prepare the deployment topology. You can refer to this example of a TiUP cluster.
Deploy a TiDB cluster. Run the tiup-cluster command. This example deploys a TiDB v4.0.0-rc cluster named “test.”
tiup cluster deploy test v4.0.0-rc topology.yaml -i ~/.ssh/id_rsa
Now you can operate and maintain this cluster. For example, to start the cluster, you can run
tiup cluster start test.
What's more, you can use TiUP to manage TiDB ecosystem components and tools, like TiFlash. You can also add components or tools in TiUP.
Perhaps you're not sure whether your application scenario is online analytical processing (OLAP) or online transaction processing (OLTP), and you just want quick results from a single system for your application. This requirement seems challenging, but we're one step closer to getting it done.
TiDB 4.0 offers a one-stop solution for both OLTP and OLAP—TiDB plus TiFlash—to handle HTAP workloads.
In this HTAP framework, TiDB processes OLTP workloads while TiFlash processes OLAP workloads.
Compared with traditional extract, transform, load (ETL) pipelines or other HTAP solutions, we have these advantages:
Before TiDB 4.0, it could be difficult to efficiently troubleshoot system problems.
Database administrators (DBAs) needed to understand TiDB's basic architecture, get familiar with thousands of TiDB monitoring metrics, and gain experience in the field to ensure that when they encountered similar problems next time, they could fix them more quickly.
Now, you can say goodbye to that big learning curve. TiDB 4.0 provides TiDB Dashboard with a variety of built-in tools that let DBAs more easily diagnose, monitor, and manage the cluster.
We've always believed that a picture is worth a thousand words and that we can directly observe many problems in a visual interface. TiDB Dashboard offers:
TiDB 4.0 provides users with a complete set of ecological tools, such as TiUP, the deployment and maintenance tool mentioned above, TiDB Data Migration, the data migration tool, and TiDB Lightning, the data import tool. These tools easily integrate TiDB with other ecosystems of users and offer users more high-value services.
Although TiDB uses three replicas by default to ensure high availability of data, many users, especially those in the financial and securities industries, hope that they can back up their data on a regular basis. If TiDB clusters are small, we can use traditional backup tools. But when the cluster data grows to tens or even hundreds of terabytes, we need to consider another way.
TiDB 4.0 provides Backup & Restore (BR), a distributed backup and restore tool, which offers high backup and restore speeds for TiDB clusters and stores the data in your shared storage or AWS S3 on the cloud. The larger the cluster size and the more TiKV instances, the faster BR's backup speed. In our test, BR backs up and restores a cluster with 10 TB of data at 1+ GB/s.
In addition to the full backup tool BR, TiDB 4.0 also provides TiCDC, an incremental replication tool that directly subscribes to TiDB's data changes and replicates data at the second or even the millisecond level.
We always hope that users can use TiDB imperceptibly and that they only need to focus on their applications. TiDB is a database resource for users, who can use it on demand. Serverless computing is a very important concept in the field of cloud services.
Before TiDB 4.0, to ensure that a TiDB cluster could withstand peak application requests, users would plan the entire cluster size from the beginning. However, most of the time these resources were underutilized. It's very hard to predict the resources you'll need.
But now TiDB 4.0, based on Kubernetes, has implemented an elastic scheduling mechanism, which makes TiDB a truly serverless architecture on cloud.
Now, you only need to deploy a TiDB cluster in the minimum cluster topology. Then, based on your own application load, TiDB will automatically:
Sounds cool, right? You can start a TiDB cluster for very little cost, and then pay as you go. Your future costs depend on your application and the resources it uses. You can directly experience this on the upcoming TiDB database as a service (DBaaS) cloud platform.
You've had an overview of TiDB 4.0's core features. Now let's take a look at its performance by comparing TiDB 4.0.0 with TiDB 3.0.8 in the following tests.
In the TPC-C test:
In the TPC-H 10 G test:
In the Sysbench test:
This post describes only a few of the highlights in TiDB 4.0. For more features, see TiDB 4.0 RC.1 Release Notes.
We hope you find TiDB 4.0 as exciting as we do, and that you'll try our release candidate. This release is also a big step forward for TiDB on its way to becoming “the future database.” Our way ahead is challenging but also promising. On this adventurous journey, we'd like to thank our contributors and TiDB users. Without you, we couldn't have come so far. We believe that TiDB will grow more robust to power more enterprises in the future.