MaxCompute Instruction- FineDataLink Help Document

Last update: October 22, 2024

Version

FineDataLink Version	Functional Change
4.1.11.4	Allowed configuring partition keys for MaxCompute-based scheduled tasks using an automatically created table as the target table. Added Partition Write to Write Method if a partition table (including partition keys) was selected as the target table or partition keys were customized when you configured the automatically created table.

FineDataLink Version

Functional Change

4.1.11.4

Allowed configuring partition keys for MaxCompute-based scheduled tasks using an automatically created table as the target table.

Added Partition Write to Write Method if a partition table (including partition keys) was selected as the target table or partition keys were customized when you configured the automatically created table.

Usage Restriction

Due to the limitation of the MaxCompute connection through JDBC, the non-null constraint information of the field cannot be obtained, resulting in theincorrect display of non-null columns on the frontend.

Task Development

Data Connection Configuration

For details, see MaxCompute Data Source Configuration.

Data Development

For details, see Overview of Data Development.

Data Source

Data Destination and Mapping

The following figure displays the configurations on the Data Destination and Mapping tab page to export data to the MaxCompute database.

If MaxCompute is used as the data source of the write end of scheduled tasks, you need to configure Tunnel settings by filling in the Tunnel Endpoint corresponding to the domain and connection method of the MaxCompute service. For details, see the third chapter of MaxCompute Data Source Configuration.

Note:

JDBC is not supported in Write Method.

1. Uploading/Downloading restrictions of Tunnel:

Currently, you cannot perform operations on external tables when using Tunnel functions and Tunnel SDK.

You cannot use Tunnel commands to upload and download complex data types such as ARRAY, MAP, and STRUCT.

2. Notes for using the Tunnel service:

Tunnel Endpoint does not support cross-domain connections. Ensure the Endpoint matches the domain and network type of the client to ensure normal access.

If you select Auto Created Table as Target Table, you can configure Partition Key Setting and select certain fields from the target table as partition keys.

A maximum of six levels of partitioning can exist in a single table.

Field types that can be selected as partition keys: TINYINT, SMALLINT, INT, BIGINT, CHAR, VARCHAR, and STRING.

Write Method

Configuration Item	Description
Write Method	Write Data into Target Table Directly is supported. Write Data into Target Table After Emptying It is supported. Add/Modify/Delete Data Based on Identifier Field is not supported.
Primary Key Mapping	MaxCompute does not support primary key restrictions.
Partition Write Setting	In the Data Synchronization node and the DB Table Output operator of the Data Transformation node: 1. When you write data to the MaxCompute database, if a partition table (including partition keys) is selected as the target table or partition keys are customized when you configure the automatically created table, Partition Write will be added to Write Method. Content of Partition Level and Partition Key are automatically fetched and cannot be customized. Content of Write Method and Partition Value can be configured. Write Method: Static partition is supported. Partition Value: When writing static partitions, you need to configure partition values for partition keys. 2. If no partition exists in the target table, a prompt will appear: "The current target table has no partition field."

Configuration Item

Description

Write Method

Write Data into Target Table Directly is supported.
Write Data into Target Table After Emptying It is supported.
Add/Modify/Delete Data Based on Identifier Field is not supported.

Primary Key Mapping

MaxCompute does not support primary key restrictions.

Partition Write Setting

In the Data Synchronization node and the DB Table Output operator of the Data Transformation node:

1. When you write data to the MaxCompute database, if a partition table (including partition keys) is selected as the target table or partition keys are customized when you configure the automatically created table, Partition Write will be added to Write Method.

Content of Partition Level and Partition Key are automatically fetched and cannot be customized. Content of Write Method and Partition Value can be configured.

Write Method: Static partition is supported.
Partition Value: When writing static partitions, you need to configure partition values for partition keys.

2. If no partition exists in the target table, a prompt will appear: "The current target table has no partition field."

Field Types

The JSON type is not supported for both reading and writing.

Writing null into the string type will result in error reporting(a bug in ODPS), so null will be replaced with an empty string.

The field mapping of the automatically created table is filled with the 2.0 data type by default. If you want to create a table with the 1.0 data type, you need to manually modify the data type in the field mapping.