TiDB Data Migration (DM) is an integrated data replication task management platform that supports the full data migration and the incremental data migration from MySQL/MariaDB into TiDB. It can help to reduce the operations cost and simplify the troubleshooting process.
DM replicates data to TiDB in the form of SQL statements, so each version of DM is compatible with all versions of TiDB. In the production environment, it is recommended to use the latest released version of DM. To install DM, see DM download link.
The Data Migration tool includes three components: DM-master, DM-worker, and dmctl.
DM-master manages and schedules the operation of data replication tasks.
DM-worker executes specific data replication tasks.
After DM-worker is started, it automatically replicates the upstream binlog to the local configuration directory (the default replication directory is
<deploy_dir>/relay_log if DM is deployed using
DM-Ansible). For details about DM-worker, see DM-worker Introduction. For details about the relay log, see Relay Log.
dmctl is the command line tool used to control the DM cluster.
This section describes the data replication features provided by the Data Migration tool.
The schema and table routing feature means that DM can replicate a certain table of the upstream MySQL or MariaDB instance to the specified table in the downstream, which can be used to merge or replicate the sharding data.
The black and white lists filtering rule of the upstream database instance tables is similar to MySQL
replication-rules-table, which can be used to filter or only replicate all operations of some databases or some tables.
Binlog event filtering is a more fine-grained filtering rule than the black and white lists filtering rule. You can use statements like
TRUNCATE TABLE to specify the binlog events of
schema/table that you need to replicate or filter out.
DM supports merging the original sharded instances and tables into TiDB, but with some restrictions.
Before using the DM tool, note the following restrictions: