Data Source Creation and Management- FineDataLink Help Document

Last update: May 22, 2024

Overview

Version

FineDataLink Version	Functional Change
4.0.21	Independently deployed FineDataLink does not display unsupported data source types on the Data Connection Management page under Management System > Data Connection. Data connection name supports special characters (including '~!@#$ %^&*()_-+={}[]).
4.1.3	The Fetchsize configuration item is added.
4.1.5.1	The setting items Time Threshold for Forcibly Recycling Idle Connection and Time Threshold for Regularly Checking Idle Connection Availability are added.

Application Scenario

Before synchronizing data, you need to configure the source and target databases so that you can select the databases to be read and written when configuring the synchronization task.

Function Description

FineDataLink supports the configuration of multiple data sources.

This article describes the information you need to know when configuring a data source.

Procedure

Data Connection Creation

Log in to FineDataLink as the admin and choose System Management > Data Connection > Data Connection Management > New Data Connection.

Note: You cannot set the data connection unless you have management permission on data connections.

Database Type Selection

Select the required database type. This document takes the MySQL database as an example. ..

Data Connection Configuration

Database Information Configuration

Fill in the database information. The setting items of different databases may vary slightly. The setting page of MySQL databases is shown in the following figure.

The following table describes the data connection information in detail. The information needed varies with databases, so some databases may have missing items.

Configuration Item	Introduction
Data Connection Name	Name the data connection according to your habit.
Driver	Select the driver needed to connect the database. Default: Use the built-in database drivers (applicable to some databases). Custom: Customize drivers if FineDataLink has no built-in drivers. For details, see Driver Management.
Database Name	Enter the name of the database to be connected.
Host	Enter the IP address of the host (or server) where the database is located.
Port Number	Enter the port number of the host (or server) where the database is located.
AUTH Method	If the database supports multiple authentication methods, a drop-down list will appear. Username and Password: Enter the username and the password of the database. Kerberos Authentication: Enter the Kerberos principal name created in KDC and the path of the keytab file. For details, see Kerberos Authentication in Data Connection.
Encoding	You are advised to select Default.
Pattern	Some databases require you to select a pattern after they are connected. You do not need to select a pattern if there is no Pattern option on the setting page. The first pattern is selected by default if no pattern is selected. Include the pattern prefix in SQL statements if the connected database has multiple patterns. Note: Correctly select the database pattern. Otherwise, a prompt saying "No table is available in the current data connection." will occur when you add database tables. In this case, you can only retrieve data via SQL datasets.
Data Connection URL	The URL is automatically generated, and no operation is needed.

Click Test Connection to test if the connection is successful.

Click Save to complete the data connection configuration if you do not need to set up a connection pool.

Connection Pool Setting

Some advanced settings may be required to connect to a database, which in most cases is not needed. You can set these items as needed, as shown in the following figure.

Common Settings

Name	Parameter Name in Druid	Default Value	Definition
Max Active Connections	maxActive	50	The maximum number of active connections that the connection pool can allocate at the same time.
Verification Statement	validationQuery	SELECT x	The SQL statement used to verify whether a connection is valid, which can be customized.
Test Before Connectio	testOnBorrow	Yes	Whether to execute validationQuery to detect connection availability when obtaining connections from the connection pool.
Max Wait Time	maxWait	10000	The maximum waiting time when getting a connection (unit: millisecond).

More Settings (not commonly used; you are advised not to modify the settings except for any special circumstance.)

Name	Parameter Name in Druid	Default Value	Definition
Number of Initial Connections	initialSize	0	The number of physical connections established during initialization.
Min Number of Idle Connections	minIdle	0	The minimum number of connections.
Detection Before Returning Connections	testOnReturn	No	Whether to execute validationQuery to check connection availability when the connection is released to the connection pool.
Detection by Idle Connection Recycler	testWhileIdle	No	Whether to enable idle connection detection for recycling. If a connection fails to validate, it will be dropped from the pool.
Sleep Time of Idle Connection Recycler	timeBetweenEvictionRunsMillis	-1	The number of milliseconds to sleep between runs of the idle connection recycler thread.
Number of Idle Connections Detected by Recycler	numTestPerEvictionRun	3	The number of connections checked during each run of the idle connection recycler thread (if any).
Min Idle Time	minEvictableldleTimeMillis	1800 s	The minimum amount of time an object may sit idle in the pool before it is eligible for eviction If a data connection is idle for longer than this value, check it.
Time Threshold for Forcibly Recycling Idle Connection	maxEvictableldleTimeMillis	25,200 s, or 7 h	If a data connection is idle for more than this value, it will be dropped from the connection pool.
Time Threshold for Regularly Checking Idle Connection Availability	keepAliveBetweenTimeMillis	80000 ms, or 80 s	The duration for keeping idle connections alive, ensuring they won't be inadvertently closed after a period of inactivity.

Fetchsize Setting

The Fetchsize parameter is used to control the number of records to be processed in batches for obtaining data from the database. A smaller value of the Fetchsize parameter can reduce the memory consumption of each database query but may require more database round trips to retrieve the complete result set. A larger value of the Fetchsize parameter can reduce the number of database round trips but will increase memory usage. You can optimize the database query performance by setting Fetchsize as required.

If the parameter value is less than or equal to 0 (for example: -20), the Fetchsize parameter is disabled.

1. The Fetchsize parameter must be set for the Oracle, DB2, and Postgre databases. If the parameter value is less than or equal to 0, the value will be automatically replaced by 128, 50, and 10000 respectively.

2. The Fetchsize parameter is disabled for other databases by default.

3. This parameter will not be configured if you keep it blank.

SSH Setting

If you have security requirements and need to use SSH, tick Use SSH Tunnel and fill in the corresponding information.

Note: There are two options in Verification Method: Password and Public Key. If you select Public Key, you need to put the file in the path %FineDataLink%/webapps/webroot/WEB-INF/resources.

Click Test Connection. If the connection is successful, click Save to save the configuration.

SSL Setting

Note: SSL Setting is only supported for MySQL databases currently.

1. Put the CA certificate, client certificate, and client secret key in the path %FineDataLink%/webapps/webroot/WEB-INF/resources/certificates.

2. Tick Use SSL Tunnel and add the files accordingly.

3. Click Test Connection. If the connection is successful, click Save to save the configuration.

Data Connection Management

Management Scope

Choose System Management > Data Connection > Data Connection Management. Click the icon on the right of a database.

The following table describes the settings of data connection management.

Setting Item	Description
Test Connection	Test whether the database configuration is successful.
Edit	Modify the value of the setting item of a data connection.
Rename	Rename the data connection.
Copy	Copy a database. The copy name is name of the copied data connection 1.
Delete	Delete a database.
Data Connection Information	Display the type of the database and its creator.

You need to note the following points:

The icon is not displayed on the right of unsupported databases.
Do not modify the data connection name arbitrarily when managing the data connection. The name of the data connection upon which locally updated tables depend is matched with the database. If the data connection name is modified, tables retrieved from that connection will no longer be updated.
The Edit SQL button will not exist if the database is an SQL dataset.
If the database upon which locally saved tables depend is updated, for instance, transitioning from a development environment to a production environment, and both the table names and structures remain unchanged, you only need to modify the data connection URL for the locally saved table to be updated via the same data connection.

Connection Pool Status

You can view created data connections and their connection status on the Connection Pool Status page, as shown in the following figure.

The following table describes Number of Active Connections and Number of Idle Connections.

Connection Pool Status	Description
Number of Active Connections	The maximum number of connections can be created. If the value is 10, and ten data connections exist in the pool and have been used, the system will pop up an error message "active 10 maxActive 10" when you apply for the eleventh data connection.
Number of Idle Connections	The number of idle connections that can change into active connections at any time. An idle connection will be dropped automatically if its idle time exceeds the maximum idle connection time. Number of active connections + Number of idle connections = Number of actual physical connections

Connection Pool Status

Description

Number of Active Connections

The maximum number of connections can be created. If the value is 10, and ten data connections exist in the pool and have been used, the system will pop up an error message "active 10 maxActive 10" when you apply for the eleventh data connection.

Number of Idle Connections

The number of idle connections that can change into active connections at any time. An idle connection will be dropped automatically if its idle time exceeds the maximum idle connection time.

Number of active connections + Number of idle connections = Number of actual physical connections

Data Connection Timeout Detection

You can set the timeout duration of a data connection in Data Connection Timeout Detection under Data Connection Management > Global Setting.

If a data connection times out, an error message will pop up, saying "31300010 kdc server is not connected."

Helpful
Not helpful
Only read

中文（简体）

English

Data Source Creation and Management