Elevate modern apps with TiDB.
Trip.com is a leading global one-stop travel platform, encompassing brands such as Trip.com, Qunar, and Skyscanner. Trip.com offers services to over 90 million members, including hotel reservations, reviews, special hotel searches, flight bookings, status and schedule inquiries, ticket price searches, and flight queries.
Use Case 1: Speed Up the 1-billion-data Real-time Customer Data Platform
Scenario Description
In the international business realm, Trip.com operates in a diverse market with complex products, multiple distribution channels, and high customer acquisition costs. This complexity necessitates detailed management and optimization of business operations and products to meet market needs, reduce costs, and improve efficiency and conversion rates. To address these challenges, Trip.com developed the International Business Dynamic Real-Time Tagging Processing Platform (CDP).
Trip.com’s data is characterized by diverse sources and formats, and a mix of offline and online processing. The challenge is to collect, manage, and process this data to produce tags that meet business, operational, and market needs. The processed data must be immediately applicable to business systems, demanding high timeliness, accuracy, stability, and flexibility.
To solve these issues, the CDP system must enhance its data processing capabilities. Previously, traditional solutions involved T+1 calculations through a data warehouse, with data then imported into ES clusters. Queries were assembled based on conditions, leading to slower response times due to offline computation and reliance on ES indexes. With over a hundred tags deployed and more than 50% used for querying, this offline approach led to poor data timeliness.
Solution
The CDP aims to improve data processing timeliness while meeting business flexibility requirements. Data processing and update logic can be dynamically configured through system rules to process messages (Kafka or QMQ) and update tags. The business layer focuses only on data filtering logic and conditional queries.
Based on business needs, data tagging is divided into two main scenarios:
- Real-Time Trigger Scenario: Dynamic rules are configured to subscribe to changes in business systems in real-time. Data meeting dynamic rule conditions is filtered and pushed to downstream business entities.
- Tag Persistence Scenario: Real-time business change messages are processed into business-related feature data and persistently stored in a storage engine. Queries are assembled based on business needs to access data from OLAP (analytical) and OLTP (transactional) engines.
The CDP system uses a Kappa-like architecture for streaming data and a Lambda-like architecture for tag persistence.
TiDB is utilized for storing persistent business tags and dynamically configured to handle real-time data changes, ensuring the timeliness of business tags. TiDB supports both OLTP and OLAP query scenarios, addressing different business needs.
The system adopts Lambda data processing architecture concepts, with new data routed to different channels based on source. Historical full data is batch-processed (e.g., using Spark) and written to TiDB. Incremental data is sent as messages to Kafka or QMQ queues, processed according to tagging persistence rules, and incrementally written to TiDB, addressing timeliness issues.
TiDB offers two main storage methods: TiKV (row storage) for OLTP scenarios and TiFlash (column storage) for OLAP scenarios. TiDB’s internal mechanisms handle synchronization between these engines, allowing clients to choose query methods based on their needs. TiDB also ensures strong consistency and isolation, effectively addressing isolation and column storage synchronization issues in HTAP scenarios.
Currently, the CDP has been deeply integrated with Trip.com’s various business systems, providing data and services to support international business growth.
Application Value
- HTAP Mixed Load: Perfectly supports OLTP and OLAP mixed loads, simplifies IT system architecture, and significantly improves real-time query performance.
- Horizontal Elastic Scaling: Overcomes MySQL sharding issues, enabling Trip.com to scale horizontally in response to business growth.
Use Case 2: Trip.com Hotel Settlement Business
Scenario Description
The hotel settlement business is used to settle commissions with hotel suppliers based on orders. It is a core P0 level business.
The business focuses mainly on financial calculations, with a large data volume (billion-level). Data integrity is crucial, with high transaction performance requirements for timely order data processing, real-time monitoring, and handling. There are also significant analytical processing needs, including batch calculations (range calculations) that need to be completed quickly (in seconds), various dimension monitoring and reporting, and multi-dimensional queries (for financial settlement processing).
Original Technology Stack:
- SQL Server (handled TP business for hotel settlement)
- Clickhouse (handled AP business for hotel settlement)
Solution
TiDB now handles all TP and some AP aspects of the hotel settlement business.
Why TiDB
The TP part of the hotel settlement business requires replacing SQL Server. Although Trip.com’s primary TP technology stack is MySQL, this business scenario does not fit well with MySQL’s sharding solution due to the need to manage data across three dimensions: suppliers, hotels, and users. None of these dimensions, such as sharding keys, satisfy the cross-database and cross-shard transactional consistency requirements, leading to the decision to use a distributed database.
The data volume for the hotel settlement business is expected to reach around 8TB. The business model involves mixed TP and AP read-write requests, aligning with Trip.com’s internal use case for TiDB.
The Trip.com DBA team has the most experience and expertise with TiDB among various distributed databases. Given that TiDB allows for technical innovation and meets the team’s innovation requirements, it was ultimately chosen.
Business Process Diagram
Workload
- Read/Write Ratio: Greater than 9:1
- QPS Peak: 2.3K, Average: 1.9K
- 999 Duration < 40ms, 99 Duration < 10ms
Benefits of Using TiDB
- Meets the multi-dimensional settlement needs of the hotel business, avoiding issues with MySQL sharding that cannot handle cross-database and cross-shard transactional queries.
- Helps the DBA team complete the task of decommissioning SQL Server, consolidates the technology stack to MySQL/Redis/TiDB/Oceanbase (with minor use of MongoDB and Elasticsearch).
- Achieves team innovation goals, provides a distributed database deployment case, and benefits both team and individual evaluations.
