4.2.16 Update Log

GTID Read Mode for MySQL Databases

In real-time pipeline tasks and real-time tasks, if the source end is a MySQL database, the GTID read mode is supported.

Optimization of Database and Table Management

All Data Connections and Databases/Tables in the Left List

Before optimization: 

Only databases/tables related to a single data connection were displayed in the left panel, making it impossible to view files related to different data connections simultaneously.

After optimization: 

All folders, data connections, databases/tables, and other files are displayed.

For details, see Database Table Management.

Independent Management of SQL Scripts

For details, see Database Table Management.

Before optimization: 

SQL scripts for data querying or table processing in Data Management were tightly coupled with the selected data connection. Switching to another data connection would close the SQL script related to the current connection, which was cumbersome.

After optimization:

1. The original SQL Editor function has been changed to the Debugging Script group in each data connection. This group contains all SQL scripts about this data connection for easy viewing and management.

2. You can open and manage multiple scripts related to different data connections on the right panel, with each script being completely independent. Switching data connections does not affect the editors in the right area.

SQL Query Result Export

The total row count of SQL query results is now displayed, and you can export the query results.

Optimization of the File Input Operator

Before optimization: 

If the folder address in Target File Location contained a parameter, an error would occur after execution if the folder did not exist in the corresponding path.

After optimization: 

You can set the handling policy for the nonexistent folder address. Options include Auto Create and Fail with Error. If a missing folder path is detected during execution, the system can automatically create the folder (with Auto Create selected) and output data.

For details, see File Transfer.

New JSONPath Configuration Method for Request Body of Data Services

Before optimization: 

For FineDataLink upgraded from an older version to a version between V4.2.6.2 and V4.2.16.3, if the API before the upgrade used the POST request method, the application/json body format, and multiple custom parameters, and modifications were made after the upgrade, you must ensure that the parameter order was unchanged during API calls. Otherwise, parameter mapping would be misaligned.

After optimization:

• This issue does not occur in projects upgraded from an older version to V4.2.16.4 and later versions.

• If the issue has already occurred, upgrade FineDataLink to V4.2.16.4 or a later version, click the Custom JSONPath button under API Configuration > Request Parameter to modify the JSONPath.

For details, see Function Description of Data Query API.

Optimization of Field Mapping Rules

For details, see General Configuration - Field Mapping Rules.

Optimization of Source/Target Data Connection Configuration

1. The original Applicable Data Connection has been divided into Source-End Data Connection and Target-End Data Connection.

2. An All Data Connections option has been added to the dropdown list, allowing you to select all data connections of the same data source type for unified configuration of field mapping rules. It can reduce repetitive work and avoid rule inconsistencies due to individual configuration.

Mapping Rule Optimization

1. Identifying data of number and varchar types without precision is supported.

2. For decimal and numeric types of data in SQL Server databases, the default scale starts from 0.

3. A toggle button has been added next to the dropdown trigger button of source field types, allowing you to switch to an input box for configuring special data types, as shown in the following figure.

Copy Logic Optimization for Real-Time Pipeline Tasks

Before optimization: 

After clicking Copy to of a real-time pipeline task, you were redirected to the task creation page, which might mislead you to think that clicking Save was required to save the copy. Additionally, clicking Save would cause the loss of some custom configuration from the original task.

After optimization: 

After clicking Copy to of a real-time pipeline task, you are redirected to the details page of the created duplicate.

Optimization of Error Prompt for Capture Task Aborts

Before optimization: 

When a capture task aborted, the corresponding real-time pipeline task and real-time task were logged as manually paused. Additionally, the data-sharing center API did not distinguish between manual pauses and aborts, leading to confusion.

After optimization: 

The prompt has been improved and now reads “The capture task Task name stops. The pipeline task stops synchronization.”

Optimization of External Systems Calling Scheduled Task APIs

Before optimization: 

When an external system called a scheduled task API, only the specified task was triggered, and its downstream tasks in the event-based scheduling plan would not be triggered.

After optimization: 

Support has been added to trigger the task and its downstream tasks. You only need to include triggerDownstream: true in the request body.

For details, see Introduction to APIs Related to Scheduled Task.

Optimization of Various Function Module Pages

Homepage Optimization

The homepages of Scheduled Task, Real-Time Task, Real-Time Pipeline, Scheduled Pipeline, Data Service, and Inspection Task now display corresponding flowcharts, task overview, recently edited tasks, and favorited tasks, as shown in the following figure.

For details, see Introduction to the Scheduled Task Page.

New Function of My Favorites

You can now favorite/unfavorite scheduled tasks, real-time tasks, real-time pipeline tasks, scheduled pipeline tasks, data services, and data inspection tasks. Once favorited, the task will be displayed on the module homepage for easy access.

New Field of Last Edit Time in the Task Information Window

The last edit time is now displayed on the task details page or in the task information window for scheduled tasks, real-time tasks, real-time pipeline tasks, scheduled pipeline tasks, data services, and data inspection tasks.

New Fields of Last Editor and Last Edit Time on the Task Management Page

The Last Editor and Last Edit Time fields have been added to the table in Task Management for scheduled tasks, real-time tasks, real-time pipeline tasks, scheduled pipeline tasks, data services, and data inspection tasks.

Optimization of O&M Overview

Under O&M Overview> Scheduled Task, the indicator card now includes the number of scheduled tasks to be updated.

Resource Import Optimization

For details, see Batch Import/Export of Scheduled Tasks and Scheduling Plans.

Data Connection Creation During Imports

Before optimization: 

Certain preparations were required in your environment to adapt to the resource to be imported, and the process had to be strictly followed. Otherwise, the entire import would not go smoothly.

After optimization: 

During data connection replacement, if no data connection is available for selection, you can click the Create button to create a data connection, which will be automatically added to the dropdown list, eliminating the need for preparation before imports.

Optimization of Import Content Display

The imported content has been categorized into Imported Resources and Imported Dependency Configuration. In Imported Dependency Configuration, global parameters and scheduling calendars are displayed separately.

Modification of Global Parameter Values

During resource imports, you can now modify the values of global parameters.

New Parameter Configuration for Doris and StarRocks Data Connections

Parameter Configuration has been added for Doris and StarRocks data connections. By configuring parameters, you can update only specified fields during writes, improving data write performance in high-data-volume scenarios.

For details, see StarRocks Data Connection and Doris Data Connection.

Optimization of Binding Data Services to Apps

App Creation During API Creation

Before optimization:

During API creation, you could only bind the API to an existing app.

After optimization:

During API creation, you can now create an app and bind the API to it.

Optimization of the API Details Page

1. In App Binding on the API details page, you can bind the API to different apps, view and copy the full access path, unbind the app, and modify the authorization validity period.

2. The API details page now displays the last edit time.

Optimization of the App Details Page

1. On the app details page, the table column API Path has been renamed Access Path, where the full API address is displayed.

2. The app details page now displays the last edit time.

Optimization of Effective App Path Display in Rule Management

When you configure blocklist/allowlist rules, rate-limiting rules, and alert rules, the displayed effective path of the added app is the full API address.

New Trigger Mode for Scheduled Tasks

Under O&M Center > Scheduled Task > Execution Record, the trigger mode Invocation by Scheduled Task has been added. When a parent task runs, the trigger mode of its subtasks is Invocation by Scheduled Task.

For details, see Scheduled Task O&M – Execution Record.

NULL as the Debug Value of Parameters in Data Query APIs

In Parameter and Variable settings of data query APIs, the debug value can now be set to NULL or Custom, making it easier to distinguish between NULL values and empty strings during testing.

For details, see Function Description of Data Query API.

Optimization of Task Display (Non-Editing State)

The information display of tasks in the non-editing state, including scheduled tasks, real-time pipeline tasks, and data services, has been simplified to improve readability.

For example, in the production mode, only core configuration information is retained for scheduled tasks, making it clear at a glance.