How Chainbase Reduced Web3 Infrastructure Costs Up to 50% with a Streamlined Tech Stack Powered by TiDB

Chainbase is a leading open Web3 data infrastructure platform that empowers developers to utilize and analyze blockchain data easily. With over 5,000 active developers handling 200 million daily data requests, Chainbase is at the forefront of Web3 data innovation. 

This case study explores how Chainbase leveraged TiDB Serverless, a fully-managed MySQL-compatible DBaaS optimized for auto-scaling workloads, to transform its Web3 infrastructure that offers developers unparalleled access to blockchain data. TiDB is an advanced open-source, distributed SQL database for scaling demanding modern applications.

The Problem: Challenges with Performance and Scalability

Chainbase’s primary challenge was to build high-performance APIs that provide fluid access to both on and off-chain data for diverse blockchain applications. Their two pivotal features – the Web3 API and the SQL API – used different data stacks including ClickHouse, Amazon Aurora for PostgreSQL, and Amazon RDS for MySQL. 

Web3 API Challenges

Chainbase’s Web3 API simplifies web development processes by providing easy access to on and off-chain data. It’s essential for building blockchain-based applications like NFT marketplaces and decentralized finance platforms. Chainbase previously used Amazon Aurora for PostgreSQL to support Web3 API. The challenges included:

• Data querying: Complex join queries with Amazon Aurora isn’t ideal when Web3 data volume is large. The need for a system capable of handling extensive data queries with speed and precision was paramount.
• Scalability: As Web3 data surged, the platform struggled to scale effectively, especially during peak traffic.
• Cost-Efficiency: The team had to manage multiple Amazon Aurora clusters for Web3 API data. Due to different tech stacks for Web3 and SQL APIs, they also needed to maintain two copies of data in Aurora and ClickHouse. The extra operational and development overhead makes it a critical challenge to balance costs and maintain a high level of service.

SQL API Challenges

Tailored for users with specific data requirements, Chainbase’s SQL API addresses unique needs outside the scope of its Web3 API. Previously, they used ClickHouse as the data infrastructure for its SQL API, with the following challenges:

• Performance issues: The Chainbase team always ran into slow SQL responses with high latency.
• Concurrency handling: ClickHouse can’t support high-concurrency queries or secondary indexes. High concurrency often led to query failures within ClickHouse. 
• Query execution reliability: Ensuring consistent, error-free SQL execution on ClickHouse was a major challenge, highlighting the need for an improved system to handle SQL queries effectively.

The Solution: Embracing TiDB Serverless for a Consolidated, Fully-Managed Solution

Chainbase turned to TiDB Serverless to overcome these challenges.  

Chainbase’s Web3 API

• Mixed workload processing: Featuring Hybrid Transactional and Analytical Processing (HTAP) capabilities, TiDB Serverless can process online transactions (i.e., OLTP) and online analytics (i.e., OLAP)  in a single database. The combined strength of TiDB’s SQL query layer and TiFlash, TiDB’s columnar storage engine, outperform traditional databases like MySQL and PostgreSQL in high-concurrency point and aggregate queries.
• Automatic scaling: TiDB Serverless provides elastic, on-demand scaling to efficiently handle varying workloads, ensuring performance optimization and cost reduction.

Chainbase’s SQL API

Chainbase revamped its SQL API with TiDB Serverless. Key implementations include:

• Streamlined architecture: The new SQL API pipeline supports complete streaming writes and reads, ensuring freshness and accurate data flow.
• Enhanced point queries: The SQL API now has significantly boosted efficiency for direct data queries.
• Transformed SQL execution: The upgraded data infrastructure transforms SQL executions into streaming data jobs and synchronizes them into TiDB for direct access by the SQL API.

Benefits of the Migration

Chainbase’s adoption of TiDB Serverless has improved its data management, particularly enhancing its Web3 and SQL API services.

Key benefits include:

• Simplified technology stack: TiDB’s HTAP capabilities enable efficient data aggregation and joining, handling extensive datasets with ease. Moreover, the rapid response of TiKV, the distributed KV storage, to high-concurrency demands further boosts performance.
• Data consolidation and cost reduction: By consolidating data from diverse systems like ClickHouse, Amazon Aurora, and MySQL into TiDB, Chainbase has achieved more efficient resource utilization. This integration has led to a significant decrease in maintenance and hardware expenses, cutting costs by approximately 50%.
• Focus on core business operations: TiDB Serverless as a fully-managed service allows  Chainbase to dedicate more resources and attention to its core business activities and strategic projects.

Web3 API Improvements

• Enhanced data management: The system now adeptly handles simultaneous queries and large-scale data operations.
• Scalability and cost-effectiveness: TiDB Serverless’ auto-scaling capability ensures optimal resource use, yielding substantial cost savings.

SQL API Enhancements

• Elevated performance: Chainbase’s SQL API now offers lower latency, higher concurrency, and consistent availability, marking a notable performance improvement. A key aspect of the improvement is the optimization of complex SQL queries for custom-designed APIs, with the query latency reduced from several seconds or minutes down to mere hundreds of milliseconds.
• Boosted efficiency and reliability: The upgrade has led to better query per second (QPS) rates, ensuring data is both more reliable and current. 

Chainbase’s migration to TiDB Serverless has not only streamlined the company’s technological framework. It’s also fortified its data-handling capabilities, setting a new benchmark in efficiency and reliability.