Supported partition table creation and data writing into partition tables when you wrote data into Hive and Transwarp Inceptor databases.
Allowed selecting partition tables in PostgreSQL, Greenplum, and GaussDB 200 databases as data sources or destinations in Scheduled Task and Data Service.
Allowed specifying partition keys for target tables in PostgreSQL, Greenplum, and GaussDB 200 databases and distribution keys for target tables in Greenplum and GaussDB 200 databases in Scheduled Task and Pipeline Task when Target Table was set to Auto Created Table.
For YMatrix databases:
Allowed selecting partition tables as data sources and destinations in Scheduled Task.
Allowed selecting partition tables as data sources in Data Service.
Allowed defining partitions for target tables in Scheduled Task and Pipeline Task when Target Table was set to Auto Created Table.
Allowed specifying distribution keys for target tables in Scheduled Task and Pipeline Task when Target Table was set to Auto Created Table.
For PolarDB PostgreSQL databases:
Allowed configuring partition keys for target tables in MaxCompute databases in Scheduled Task when Target Table was set to Auto Created Table.
Added a Partition Write Setting item, which would be displayed on the Write Method tab page when a partition table (with defined partition keys) was selected as the target table or when you had defined partition keys for the target table when selecting Auto Created Table.
Supported data reading from Kudu tables in Impala databases.
Supported data writing into Kudu tables in Impala databases.
Many databases support partition tables to enhance query performance in scenarios with large data volumes. You may want to create, read data from, and write data into partition tables using FineDataLink.
You can select partition tables in some databases as data sources and destinations and set partition keys and distribution keys for target tables when setting Target Table to Auto Created Table.
You can read data from partition tables in these data sources in Scheduled Task and Data Service, as shown in the following figure.
YMatrix (supported in FineDataLink from V4.1.11.2)
Partition tables in PolarDB PostgreSQL (supported in FineDataLink from V4.1.11.3) and Kudu tables in Impala (supported in FineDataLink from V4.2.0.2)
You can select partition tables as target tables in Scheduled Task and Pipeline Task, as shown in the following figure.
In the Data Synchronization node and the DB Table Output operator in the Data Transformation node:
You can select fields in the target table as partition keys when Target Table is set to Auto Created Table, as shown in the following figure.
Click Partition Key Setting. The page is shown in the following figure.
For details about data types of fields that can be selected as partition keys, see Hadoop Hive Data Connection, Transwarp Inceptor Instruction, and MaxCompute Instruction.
In scheduled tasks and pipeline tasks:
You can write data into partition tables when Target Table is set to Auto Created Table.
For details, see PostgreSQL Instruction, Greenplum Instruction, GaussDB 200 Instruction, YMatrix Instruction, PolarDB PostgreSQL Instruction, and Apache Impala Data Connection.
You can select a partitioning method. Supported partitioning methods vary with data sources.
RANGE
LIST
HASH
You can select fields in Target Table Field as partition fields.
RANGE and HASH: You can select one field or multiple fields as the partition field.
LIST: You can select only one field as the partition field.
After selecting the partitioning method, you can add multiple partitions. The setting items of different partitioning methods are described below.
RANGE:
Partition name
Partition start value (inclusive): Enter text, a number, or a date.
Partition end value (exclusive): Enter text, a number, or a date.
LIST:
Value list: You can add and delete values. Multiple values are supported.
HASH:
Modulus: Enter a number.
Remainder: Enter a number.
You can specify HASH distribution keys for target tables when Target Table is set to Auto Created Table in Scheduled Task and Pipeline Task.
Note:
You can leave the distribution key unconfigured. In this case, databases will follow their default logic to specify one. (By default, the primary key or the first column of the table is used as the distribution key.)
When you write data to partition tables (with defined partition keys) in Hive, Transwarp Inceptor, or MaxCompute databases, or you have defined partition keys for target tables to be created automatically, a Partition Write Setting item is displayed on the Write Method tab page.
Partitioning levels and partition keys are displayed and uneditable. You can configure write methods and partition values.
Write Method: You can choose from Static Partition and Dynamic Partition. Note: MaxCompute databases support Static Partition only.
Partition Value: You need to configure partition values for partition keys if you select Static Partition.
滑鼠選中內容，快速回饋問題
滑鼠選中存在疑惑的內容，即可快速回饋問題，我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy