4.2.9 Update Log

Introducing Rate-Limiting and Anomaly Alert Mechanisms to Data Service 

1. FineDataLink previously did not enforce concurrency limits on data service APIs. During periods of high concurrency, Tomcat threads could be exhausted, making the browser page inaccessible and affecting the entire FineDataLink project.

From this version, FineDataLink allows you to set call limits both globally and per API. In the event of instantaneous high concurrent accesses, requests exceeding the set limit will fail fast.

2. When APIs created in Data Service encountered request timeout or issues, you could only check for errors on the O&M page of Data Service or wait for business feedback, which was not timely.

From this version, FineDataLink allows you to add alert rules per API. The notification will be sent once the configured alert conditions are met.

Data Reception Supporting Data Writing into GaussDB 100 Data Sources

The data-receiving API supports data writing into GaussDB 100 data sources.

Allowing Configuring Field Mapping Rules for ClickHouse Data Sources in General Configuration

You can now configure field mapping rules for ClickHouse data sources in General Configuration. For details, see General Configuration - Field Mapping Rule.

Supporting Browsers Using the Chrome 86 Kernel

FineDataLink supports browsers based on the Chrome 86 kernel. For browsers running on older-version kernels, page display anomalies will occur, and only functional usability is guaranteed.

Optimizing Canvas-Related Operations and Layout in Scheduled Task

Adding Tab Operations to the Editing Page of Scheduled Tasks

Tab operations are available on the editing page of scheduled tasks. You can now right-click a tab and choose to close the current task, other tasks, and all tasks. When too many tasks are open and cause visual clutter during tab switching, you can quickly close multiple tabs at once, as shown in the following figure.

Enabling Quick Node Renaming

You can hover the cursor over the node name on the node panel and click the  icon to rename a node quickly, as shown in the following figure.

Optimizing the Display Format of the Shortcut Overview Panel

The following figure shows the current shortcut overview page in Scheduled Task.

Watermarking Empty Canvases with Shortcut Instructions

Empty canvases are watermarked with shortcut hints. Tooltips are added to shortcut buttons, as shown in the following figure.

New Setting Items During FTP/SFTP Data Connection Configuration

For details, see FTP/SFTP Data Connection.

1. You can now set the encryption method when configuring an FTP data connection, as shown in the following figure. 

2. You can set the maximum wait time to establish a connection when configuring an SFTP data connection. The connection will be closed, and a connection failure will be thrown if this threshold is exceeded.

You can also set the maximum duration to wait for a data response during transmission. Connections will be forcibly terminated if this threshold is exceeded.

New Setting Items During SSH Data Connection Configuration

For details, see SSH Data Connection.

1. The settings are available for OpenSSH of V6.X or later versions.

2. You can set the maximum wait time to establish a connection to the server when configuring an SSH data connection. Connection failure will be thrown if this threshold is exceeded. You can also set Authentication Method to Public Key, as shown in the following figure.

Optimization of Scheduled Task Logs

Before optimization:

The front-end logs were difficult to use for troubleshooting a scheduled task with unsatisfactory execution results because they provided unclear information, were overly verbose, and offered poor operation support.

• You could not obtain the necessary information for troubleshooting from the logs, making it impossible to pinpoint the issue.

• The information was located in the back-end logs, resulting in a lengthy and complex search process.

• Performing troubleshooting in the front-end logs was inconvenient.

After optimization:

1. Execution logs now include cluster node details, pre-execution actions and timing, information and response time of SAP Input and API Input, Kettle invocation results, script connection information, version numbers, and the SQL queries used for table selection.

For details, see Introduction to Execution Logs of Scheduled Tasks.

2. A icon has been added to the Log tab page, enabling keyword-based searches within the logs, as shown in the following figure.

For details, see Introduction to the Scheduled Task Page.

3. The task publication version information is added to facilitate troubleshooting task execution issues, as shown in the following figure.

For details, see Scheduled Task O&M – Running Record.

Optimization of the Default App Path Prefix of Data Service APIs

Before optimization:

The default application path prefix was service/publish. In scenarios such as data reception and other non-publishing use cases, the word "publish" was no longer semantically appropriate.

After optimization:

The default application path prefix is service (without publish).

Optimization of Return Value Sorting Logic

The logic of Return Value Sorting of data query APIs in Data Service has been optimized:

• If the data source is a Doris, StarRocks, or Impala database, and Pagination Query is enabled, a prompt will appear, suggesting you specify a valid field for sorting to ensure proper pagination.

• If the data source is a GreenPlum, YMatrix, GaussDB 200, TiDB, AnalyticDB MySQL, Transwarp Inceptor, KingbaseES (MySQL), KingbaseES (Oracle), or Cache database, and Pagination Query is enabled, Return Value Sorting must be configured.

For details, see Function Description of Data Query API.

Registration Optimization

1. The authentication method Public Cloud has been renamed Online Registration. 

2. When authenticating the project via Local Machine Info, you must send the application project ID to the business personnel. 

For details, see Local Machine Info Authentication and Public Cloud Authentication. 

Spark SQL Supporting Custom Function Plugins 

Processing data using custom Spark SQL functions could only be achieved by creating JAR packages, which involved imperfect connectors and non-standardized processes. 

From this version, FineDataLink provides an interface in Spark SQL for custom function plugins, enabling seamless use of the developed plugin.