Data Migration (DM) provides a variety of features, including table routing, black & white table lists, and binlog event filter. However, these features also increase the complexity of using DM, especially when users are modifying DM task configurations.
To address this problem, DM provides a simple web application, DM Portal. DM Portal enables users to visually configure the required replication tasks, and generates a
task.yaml file that can be directly executed by DM.
This sections describes the features of DM Portal.
DM Portal supports three replication modes:
DM Portal supports configuring table routing, and merging sharded schemas and tables in DM.
DM Portal supports configuring the binlog event filter in schemas and tables.
DM Portal supports generating configuration files and downloading these files to your local computer. Meanwhile, it automatically creates a file in the
/tmp/ directory on the dm-portal server.
Currently, DM Portal's visualized pages cover most DM configuration scenarios, but with the following restrictions:
The SQL pattern of binlog event filter is not supported.
The editing feature does not support parsing the
task.yaml file created by the user. The user can only edit the
task.yaml file generated by the page.
The editing feature does not support modifying the instance configuration. If the user need to adjust the instance configuration, the
task.yaml file has to be regenerated.
The upstream instance configuration on the page can only be used to obtain the upstream table schema. The related upstream instance information still needs to be configured in DM-worker.
In the generated
task.yaml file, mydumper-path is
./bin/mydumper by default. If you use another path, modify the generated
task.yaml file manually.
This section describes how to deploy DM Portal in two ways: using binary or DM Ansible.
Download DM Portal at dm-portal-latest-linux-amd64.tar.gz. To start DM Portal, run the
127.0.0.1:8280in your browser.
To deploy DM Portal using DM Ansible, refer to Deploy Data Migration Using DM-Ansible for details.
This section describes how to use DM Portal.
This feature is used to create a
Access the dm-portal page, and click Create New Rule.
This feature is used to fill in the task name and select a task type.
Create New Sync Rule is already selected.
This feature is used to configure the upstream and downstream instance information, including Host, Port, Username, and Password.
Task Name and Task Type are already filled in.
If you choose Incremental or All in Task Type, you need to configure binlog-file and binlog-pos when configuring the upstream instance information.
This feature is used to filter the upstream binlog. You can choose the DDL or DML that needs to be filtered. The filter configured on the database is automatically inherited by tables in that database.
- The binlog filter configuration can only be modified in the upstream instance. Once the database or table is moved to the downstream instance, the configuration cannot be modified.
- The binlog filter configured on the database is automatically inherited by tables in that database.
Select the databases or tables that need to be configured.
Click the Edit button, and select the binlog types to be filtered.
This feature is used to perform the following operations:
After the task configuration is completed, DM Portal generates the corresponding
The required binlog filter rules are configured.
- Batch operation is not supported when you merge databases and tables. You can only drag them one by one.
- You can only drag tables when you merge databases and tables. You cannot drag databases.
Select the databases and tables that need to be synced from Upstream Instance.
Click the Move button and move the selected databases and tables to Downstream Instance.
Right click the databases and tables to rename them.
Select the required table to perform the following operation:
To merge two tables, drag the table onto another table
To move the table to an existing database, drag the table onto the database
To move the table to a new database, drag the table onto the
Click Go Back to undo the last operation.
Click Reset to clear the downstream instance.
Click Finish & Download. DM Portal automatically downloads the
task.yaml file to the local computer, and creates a
task.yaml configuration file in the
/tmp/ directory on the DM Portal server.