4.1.9.3
Allowed customizing field mapping rules and configuring valid data connections.
Prioritized the customized mapping rules of data connections over the default rules in scenarios including automatic table creation, the synchronization of DDL changes via pipeline tasks, and DDL changes prompting in scheduled tasks.
4.2.2.1
Supported Field Mapping Rule in a cluster environment.
Supported Field Mapping Rule in the SelectDB database.
During the configuration of data synchronization for pipeline tasks and scheduled tasks, you may encounter situations where the expected field mapping relationship does not match the default mapping relationship provided by FineDataLink.
You can modify the field type of the target table if Target Table is set to Auto Created Table. However, for scheduled tasks, if this mapping modification needs to be applied to multiple tasks or nodes in this data source, you need to configure mapping rules repeatedly. For pipeline tasks, you may synchronize data in dozens or hundreds of tables at once. It is impractical for you to manually search and modify fields table by table and field by field.
Therefore, you may want to uniformly customize field mapping rules for multiple scheduled tasks and pipeline tasks that use the same data connection.
1. Types of data sources currently supported by Field Mapping Rule: MySQL, Oracle, SQLServer, PostgreSQL, Greenplum, StarRocks, Doris, and SelectDB.
2. You can only use the admin account set during the initialization configuration to set General Configuration, and you cannot assign Use and Management permission on General Configuration to other users.
3. For FineDataLink versions before V4.2.2.1, the Field Mapping Rule function is not supported in a cluster environment. For FineDataLink of V4.2.2.1 and later versions, this function is supported in a cluster environment.
Rule Name
You can customize the name of a field mapping rule.
Data Source Mapping
You can select the source database type and target database type.
Applicable Data Connection
You can select a created data connection that meets the conditions.
Field Mapping Rule
You can select the field types of the source and target ends as needed, set multiple field type mapping relationships between the source and target ends, and add or delete them.
Field types of the source and target ends
You can select from all available field types in the source and target databases from the drop-down list.
Length
For the field mapping on the source end:
When the upper and lower limits are initialized, the system provides default values. The limits are given based on the specific database type and field type.
You can modify the upper and lower limits as needed.
For the field mapping on the target end:
If the selected field type of the source end does not support configuration for Length, Precision, or Scale, but the field type of the target end does, you can set fixed values for length, precision, or scale on the target field.
If the selected field types of both the source and target ends support configuration for Length, Precision, or Scale, you can set precise values or multipliers for the target field.
Precision
Scale
For example, when you create a scheduled task or a pipeline task, some source tables in the MySQL database contain char-type fields.When StarRocks is the target database, you cannot select char-type fields as primary keys, and an error will occur during table creation.
You may want the char-type fields to be mapped to varchar-type fields in StarRocks by default, so that you can achieve successful configuration and data synchronization.
Before configuring Field Mapping Rule, you need to first create a data connection. For details, see Data Source Creation and Management.
1. Choose System Management > General Configuration and click Field Mapping Rule, as shown in the following figure.
2. Enter the rule name, select the source and target database types in Data Source Mapping, choose the previously created data connection with permissions on the platform, and select the field types that need mapping modification from the source end and target end.
You can choose to set Length or Precision to Fixed Value or Multiple of N. If Fixed Value is selected, you need to enter an integer ranging from 1 to 65533. If Multiple of N is selected, you need to enter a multiplier such that a multiple of n ranges from 1 to 65533.
In this example, the value of n set for Length or Precision ranges from 1 to 255, and the value set for Multiple of N is 3, as shown in the following figure.
When you create a pipeline task in Data Pipeline, the char-type fields in Table Field Mapping are changed to varchar-type fields during data synchronization, and the field length and precision are also modified according to the configured field mapping rule, as shown in the following figure.
When you create a scheduled task in Data Development, the char-type fields in Table Field Mapping are changed to varchar-type fields during data synchronization, and the field length and precision are also modified according to the configured field mapping rule, as shown in the following figure.
滑鼠選中內容，快速回饋問題
滑鼠選中存在疑惑的內容，即可快速回饋問題，我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy