Key Takeaways
- Real-time processing lets fraud and risk systems act inside the authorization window instead of after a T+1 batch.
- Legacy stacks fail twice: ETL lag between transactional and analytical systems, and single-node limits at peak volume.
- TiDB keeps a columnar TiFlash replica of TiKV data, so analytical queries run on current transactions with consistent reads.
- Trip.com runs hotel settlement on TiDB: all transactional work and part of the analytics, with P99 latency under 10 ms.
A bank has to flag a fraudulent transaction in the same second it happens, not the next morning after a batch job runs. That gap between when a transaction occurs and when a bank can act on it is where real-time data processing in financial services earns its keep: fraud detection, risk monitoring, and personalization run against live transaction data instead of yesterday’s warehouse snapshot. This piece walks through why legacy systems hit that wall, how TiDB’s TiKV and TiFlash architecture closes the gap, and how Trip.com runs a financially critical workload on it in production.
What Is Real-Time Data Processing in Financial Services?
Real-time data processing in financial services means analyzing transactions, market data, and customer activity as they occur, rather than in scheduled batches. Instead of waiting for a nightly ETL job to load yesterday’s transactions into a warehouse, a bank’s fraud model scores a transaction within moments of the card swipe or transfer request, and a risk system updates exposure as trades settle rather than at end of day.
The difference matters most in fraud detection and risk management. A batch system on a T+1 cycle finds a fraudulent transaction after the money has already moved. A real-time system flags the same transaction inside the authorization window, before it clears.
Why Legacy Systems Fall Short
Most legacy financial-services databases grew up in a batch world: nightly loads, scheduled reports, and a hard separation between the system that takes transactions and the system that analyzes them. That separation is the problem. Moving data from a transactional database into a separate warehouse takes an ETL pipeline, and every hop adds delay, typically minutes to hours depending on load. That is longer than the window a fraud team has to act.
Single-node databases add a second constraint: they scale up, onto bigger hardware, rather than out, across more nodes. During a trading spike or a holiday shopping surge, a single-node system runs out of headroom exactly when transaction volume, and therefore fraud risk, peaks.
How TiDB Solves Real-Time Financial Data Processing
TiDB is an open-source, distributed SQL database that separates the transactional and analytical paths without splitting them into different systems. TiKV, a distributed row store, handles transactional writes with strong consistency. TiFlash, a columnar engine, holds a continuously updated analytical replica of the same data.
The two stay in sync through Raft. TiFlash joins each TiKV Raft group as a learner replica, so every committed transaction reaches its columnar store in near real time without blocking TiKV’s writes. Reads stay consistent: before answering, TiFlash checks the Raft index with the leader and waits until it has caught up, so a fraud-scoring query sees transactions committed moments earlier. See the TiFlash architecture diagram for how the engines connect.
Enabling the analytical replica for a table takes one statement:
ALTER TABLE transactions SET TIFLASH REPLICA 1;A velocity check that flags accounts with unusual activity in the last five minutes then runs against current data:
-- Flag accounts with unusually high transaction velocity in the last 5 minutes
SELECT account_id, COUNT(*) AS txn_count, SUM(amount) AS total_amount
FROM transactions
WHERE transaction_time >= NOW() - INTERVAL 5 MINUTE
GROUP BY account_id
HAVING txn_count > 10 OR total_amount > 50000
ORDER BY total_amount DESC;
The optimizer decides which engine answers. A five-minute window on an indexed timestamp column may stay on TiKV, because a narrow range scan is cheap there. The baseline the alert compares against, each account’s typical velocity across weeks of history, is the wide scan TiFlash exists for. Run EXPLAIN on both queries to see the split. Either way, no overnight load sits between the transaction and the query.
For a measured fraud result on TiDB, China Zheshang Bank cut queries in its telecom fraud risk inquiry system from tens of seconds to tens of milliseconds, covered in Achieving Ultra-Low Latency in Financial Transactions With TiDB.
Real-Time Data Processing in Practice: Trip.com
Trip.com, the travel platform behind Trip.com, Qunar, and Skyscanner, runs its hotel settlement business on TiDB. Settlement reconciles commissions with hotel suppliers based on orders at billion-record scale, and Trip.com treats it as a P0 business: data integrity is non-negotiable, and batch range calculations have to finish in seconds while monitoring and multi-dimensional reporting run alongside.
Before TiDB, settlement ran on a split stack: SQL Server for transactions and ClickHouse for analytics, two systems to keep in sync. TiDB now handles all of the transactional work and part of the analytical work on one cluster. The workload reads more than nine times as often as it writes, peaks at 2,300 queries per second, and holds P99 latency under 10 ms and P999 under 40 ms.
How TiDB Compares With Traditional Financial Databases
Traditional databases common in financial services, such as Oracle, SQL Server, and older MySQL deployments, meet modern requirements only with added machinery:
- Scalability: traditional databases scale through manual sharding or bigger hardware. TiDB scales horizontally by adding nodes, with no manual resharding.
- Availability: traditional systems reach high availability through separately configured replication and failover products. TiDB builds it into the storage layer, with Raft electing a new leader automatically when a node fails.
- Hybrid workloads: most legacy databases optimize for OLTP or OLAP, not both. TiDB runs both on one system, removing the second database and the pipeline between them.
- Cost: self-managed TiDB carries no license fee, where proprietary vendors charge per core or per node.
Implementation Best Practices
A few practices separate a smooth financial-services rollout from a painful one:
- Start with a pilot: learn TiDB’s operations on a smaller, non-critical application before touching a core system.
- Use the migration toolchain: Dumpling exports from the source, TiDB Lightning bulk-loads, and TiDB Data Migration (DM) keeps TiDB in sync until cutover.
- Let the optimizer route, then isolate: the optimizer picks TiKV or TiFlash per query. To keep heavy analytics off the transactional path, pin analytics sessions to TiFlash with tidb_isolation_read_engines and cap them with a resource group.
- Monitor from day one: track latency and resource use with TiDB’s Grafana and Prometheus monitoring stack to catch bottlenecks before they reach customers.
- Plan disaster recovery up front: built-in high availability handles node failures, but a financial deployment still needs a documented, tested recovery plan for larger incidents.
Get Started With Real-Time Financial Data Processing
Real-time processing is no longer optional for fraud detection, risk management, and compliance. Batch-oriented, single-node systems cannot close the gap between when a transaction happens and when a bank can act on it, and that gap is where fraud and risk exposure grow. TiDB closes it by running transactional and analytical queries against the same live data.
Start a TiDB Cloud Starter cluster for free to test HTAP workloads against your own transaction data, or read more about TiDB for fintech.
Real-Time Financial Data Processing FAQs
What is real-time data processing in financial services?
- Analyzing transactions, market data, and customer activity as they happen, not in scheduled batches.
- It lets fraud detection, risk monitoring, and personalization act inside the transaction window.
- It needs a database that serves transactional writes and analytical queries on the same live data.
How does TiDB support real-time fraud detection?
- TiFlash replicates every committed TiKV write into a columnar store within moments.
- Fraud-scoring queries read that replica with consistent, current results.
- No ETL pipeline sits between the transaction and the analysis.
What is the difference between real-time and batch processing in banking?
- Batch processing analyzes data on a schedule, often T+1, so insights trail transactions by hours or a day.
- Real-time processing analyzes transactions as they occur, within seconds.
- Batch systems are simpler to build but miss the window most fraud and risk cases require.
Can TiDB handle both transactional and analytical workloads for finance?
- Yes. Its HTAP architecture runs TiKV for OLTP and TiFlash for OLAP on one cluster.
- Trip.com runs hotel settlement this way, with P99 latency under 10 ms.
- One system removes the cost and sync overhead of maintaining two.