GaussDB 200 Instruction

Data Source

You can read partition tables from the Greenplum database in scheduled tasks, as shown in the following figure.

You can select partition tables as data destinations in scheduled tasks, as shown in the following figure.

You can set partition keys when the Target Table is set to Auto Created Tables in scheduled tasks, as shown in the following figure.

Note: Only level-one partition is supported.

For details, see Partition Table Creation and Data Reading/Writing.

Details about the configuration method are as follows:
When using the GaussDB 200 data source, FineDataLink supports RANGE and LIST.  Specifically, it supports RANGE (using the LESS THAN method) and LIST.  You can select multiple fields as partition fields.

1. RANGE and LIST do not allow you to leave the Partition Name empty, in which case the database will assign a default name based on the partition position, without requiring FineDataLink to process it.

2. RANGE supports two methods to define boundaries. (Inclusive or exclusive ranges are not supported.)

• When the field data is of the date type, you can select Year/Month/Day as the interval unit in Automatic Partition Interval.

• When the field data is of the numeric type, you can input a positive integer as the interval.

Method One:  You need to set the start value and end value, and you can set the Automatic Partition Interval for automatic partitioning only when both start and end values are valid. For example, set the start value to 2015-01-01, the end value to 2020-12-31, and the interval to 1 Year (same as that of Greenplum databases).

Method Two: You can specify Less than XXX conditions separately, and set the condition to Less than MAXVALUE (which is different from that of Greenplum databases).

Note: The partitions specified by Method One and Method Two cannot coexist (which is different from that of Greenplum databases).

3. RANGE doesn't allow you to set a default partition, while LIST allows you to set a default partition.

Write Method

You can select the loading method on the Write Method page, as shown in the following figure.

 You cannot switch the loading method while the task is running. You can switch the loading method when the task is paused or terminated. After you switch the loading method, the task will run according to the selected loading method.

Load Method	Description
Common Loading	It is selected by default. It is selected for JDBC-based serial loading. Application scenario: It is recommended when the data volume is small or when the necessary user permission for COPY loading is not available.
COPY Loading	Application scenario: It is recommended when the data volume is large. If you select COPY Loading, you need to create an fdl_temp schema in the target database to store temporary tables and assign users permissions to create tables within the specified schema. (If the schema has been created and granted permissions by the database administrator, database users do not need permissions to create schemas.)  Write Method Description Write Data into the Target Table Directly 1. Use COPY Loading when the target table has no primary key and Primary Key Mapping is not configured. 2. When the target table has a primary key or Primary Key Mapping is configured, three primary key conflict strategies are available: Ignore Source Data If Same Primary Key Value Exists, Record as Dirty Data If Same Primary Key Value Exists, and Overwrite Data in Target Table If Same Primary Key Value Exists. After selecting one of them as the Strategy for Primary Key Conflict, COPY Loading and Common Loading are used. Write Data into the Target Table after Emptying It COPY Loading and Common Loading are used. Add/Modify/Delete Data Based on the Identifier Field When COPY Loading and Common Loading are used: If the COPY loading method fails, the batch of data will be written using the common loading method. Any data that fails to be written will be recorded as dirty data. Once the write of this batch is completed, the next batch will again prioritize the COPY method for loading.

You cannot enable Logical Deletion at Target End for the GaussDB database in pipeline tasks.

Writing Data to GaussDB 200 in Pipeline Tasks

1. You can select the Load Method at the Pipeline Task Configuration - Target Selection step, as shown in the following figure.

If you select the COPY Loading method, you need to first create an fdl_temp schema in the target database to store temporary tables. Therefore, you must grant the database user used to create the data connection the permission to create schemas in the corresponding database. 

• COPY Loading: It is recommended when the data volume is large

• Common Loading: It is recommended when the data volume is small or when the necessary user permission for COPY loading is not available.

2. You can set partition keys when the Target Table is set to Auto Created Tables in pipeline tasks, as shown in the following figure.

For details, see Overview of Data Service.

Data Service allows you to select a partition table as the data source, as shown in the following figure.