How Micoworks Consolidated Amazon DynamoDB and Redshift Workloads with TiDB

In today’s data-driven world, databases serve as the backbone for not just recording and analyzing business data, but also for accumulating diverse data sets such as customer interactions and social media metrics for effective engagement with customers. 

Micoworks is the Japanese company behind MicoCloud, an SNS communication platform that helps businesses engage with their customers. MicoCloud aims to “maximize sales and reduce operating costs” by optimizing interactions between companies and their customers. Through integrations with various communication channels like LINE, MicoCloud encourages customer loyalty while providing business visibility into customer behavior. MicoCloud is scaling fast, serving over 600 corporate customers, including  Hankyu Hanshin Department Store, Kyoto Toyopet, and Watami.  

This case study explores how Micoworks successfully integrated TiDB, an advanced open-source, distributed SQL database, to streamline its large-scale data operations and analytics.

The Problem: Challenges with Multi-database Infrastructure

Originally, MicoCloud’s data infrastructure included three different databases:

• Amazon DynamoDB: A NoSQL database holding diverse customer attributes.
• Amazon Aurora MySQL: A relational master database for corporate data.
• Amazon Redshift: A data warehouse tailored for analytical needs.

Figure 1: MicoCloud’s Architecture before TiDB

As MicoCloud scaled rapidly, operating these three databases became increasingly complex. Specifically, this database system had several challenges:

• Data transfer complexity: Transferring data from DynamoDB and Amazon Aurora MySQL to Amazon Redshift for data aggregation was complex and time-consuming.
• Operational overhead: Each database required different monitoring and management procedures, which added to the operational complexity.
• Performance bottlenecks: Micoworks’ data warehouse—Amazon Redshift—was a bottleneck for data analysis. This was because Redshift is not designed for high-volume data analysis.

“As we couldn’t fully optimize the performance of Amazon Redshift, we needed a more scalable solution,” said Mr. Han Chen, Senior Specialist, SRE Team, Product Management Division. 

Micoworks needed a unified database infrastructure that could handle large amounts of data and provide high performance for data analysis. As the team put it, the alternative solution should have “high write performance like that of DynamoDB, high data analysis performance like that of Amazon Redshift, and high compatibility with MySQL”.

The Solution: Consolidate Databases with TiDB Dedicated

Micoworks evaluated several database solutions before choosing TiDB Dedicated, a fully-managed cloud DBaaS offering of TiDB for predictable workloads. TiDB Dedicated has several features that make it well-suited for Micoworks:

• MySQL compatibility: By being wire compatible with MySQL, TiDB Dedicated allows the company to leverage MySQL’s rich ecosystem of tools and frameworks without having to learn anything new. 
• Horizontal scalability: TiDB Dedicated can be scaled horizontally to handle increasing data volumes and workloads.
• Strong consistency: TiDB Dedicated provides strong consistency for all reads and writes, which is important for data analysis.
• Mixed workload processing: By incorporating TiFlash, a columnar data store, TiDB Dedicated delivers both transaction processing and high-speed data analytics within a single database system. 
• Fully-managed service: TiDB Dedicated is available with stable operation and solid support from PingCAP, the company behind TiDB.  

After extensive testing, TiDB Dedicated met all of Micoworks’ criteria for write performance, data analysis, and MySQL compatibility. Micoworks migrated to TiDB Dedicated successfully in a single month. 

Figure 2: MicoCloud’s Architecture after TiDB

The Results

Since migrating to TiDB Dedicated, Micoworks has experienced several tangible benefits to its business:

• Reduced operational complexity: With DynamoDB and Amazon RedShift consolidated by TiDB Dedicated, Micoworks now has a much simpler database to manage. Using the managed service simplifies their operations even more.
• Improved analytics performance: TiDB Dedicated has significantly improved the performance of Micoworks’ analytics workloads. The query latency times have gone from one minute down to one second. 
• Reduced costs: TiDB Dedicated has reduced the number of database instances Micoworks pays for each month.

The migration to TiDB Dedicated solved major complexity and performance issues for Micoworks. “As MicoCloud aspires to be the leading product in conversational commerce across Asia, we need a database that is scalable, real-time, and cost-effective,” Mr. Awashima, the product manager of MicoCloud, noted, “TiDB’s flexible, high-performance architecture seems to be just the solution that Micoworks needed for their ambitious plans”.