Overview
Version
FineDataLink Version | Functional Change |
---|---|
4.0.21 |
|
4.1.3 | The Fetchsize configuration item is added. |
4.1.5.1 | The setting items Time Threshold for Forcibly Recycling Idle Connection and Time Threshold for Regularly Checking Idle Connection Availability are added. |
Application Scenario
Before synchronizing data, you need to configure the source and target databases so that you can select the databases to be read and written when configuring the synchronization task.
Function Description
FineDataLink supports the configuration of multiple data sources.
This article describes the information you need to know when configuring a data source.
Procedure
Data Connection Creation
Log in to FineDataLink as the admin and choose System Management > Data Connection > Data Connection Management > New Data Connection.
Note: You cannot set the data connection unless you have management permission on data connections.
Database Type Selection
Select the required database type. This document takes the MySQL database as an example. ..
Data Connection Configuration
Database Information Configuration
Fill in the database information. The setting items of different databases may vary slightly. The setting page of MySQL databases is shown in the following figure.
The following table describes the data connection information in detail. The information needed varies with databases, so some databases may have missing items.
Configuration Item | Introduction |
---|---|
Data Connection Name | Name the data connection according to your habit. |
Driver | Select the driver needed to connect the database.
|
Database Name | Enter the name of the database to be connected. |
Host | Enter the IP address of the host (or server) where the database is located. |
Port Number | Enter the port number of the host (or server) where the database is located. |
AUTH Method | If the database supports multiple authentication methods, a drop-down list will appear.
|
Encoding | You are advised to select Default. |
Pattern | Some databases require you to select a pattern after they are connected.
Note: Correctly select the database pattern. Otherwise, a prompt saying "No table is available in the current data connection." will occur when you add database tables. In this case, you can only retrieve data via SQL datasets. |
Data Connection URL | The URL is automatically generated, and no operation is needed. |
Click Test Connection to test if the connection is successful.
Click Save to complete the data connection configuration if you do not need to set up a connection pool.
Connection Pool Setting
Some advanced settings may be required to connect to a database, which in most cases is not needed. You can set these items as needed, as shown in the following figure.
Common Settings
Name | Parameter Name in Druid | Default Value | Definition |
---|---|---|---|
Max Active Connections | maxActive | 50 | The maximum number of active connections that the connection pool can allocate at the same time. |
Verification Statement | validationQuery | SELECT x | The SQL statement used to verify whether a connection is valid, which can be customized. |
Test Before Connectio | testOnBorrow | Yes | Whether to execute validationQuery to detect connection availability when obtaining connections from the connection pool. |
Max Wait Time | maxWait | 10000 | The maximum waiting time when getting a connection (unit: millisecond). |
More Settings (not commonly used; you are advised not to modify the settings except for any special circumstance.)
Name | Parameter Name in Druid | Default Value | Definition |
---|---|---|---|
Number of Initial Connections | initialSize | 0 | The number of physical connections established during initialization. |
Min Number of Idle Connections | minIdle | 0 | The minimum number of connections. |
Detection Before Returning Connections | testOnReturn | No | Whether to execute validationQuery to check connection availability when the connection is released to the connection pool. |
Detection by Idle Connection Recycler | testWhileIdle | No | Whether to enable idle connection detection for recycling. If a connection fails to validate, it will be dropped from the pool. |
Sleep Time of Idle Connection Recycler | timeBetweenEvictionRunsMillis | -1 | The number of milliseconds to sleep between runs of the idle connection recycler thread. |
Number of Idle Connections Detected by Recycler | numTestPerEvictionRun | 3 | The number of connections checked during each run of the idle connection recycler thread (if any). |
Min Idle Time | minEvictableldleTimeMillis | 1800 s | The minimum amount of time an object may sit idle in the pool before it is eligible for eviction If a data connection is idle for longer than this value, check it. |
Time Threshold for Forcibly Recycling Idle Connection | maxEvictableldleTimeMillis | 25,200 s, or 7 h | If a data connection is idle for more than this value, it will be dropped from the connection pool. |
Time Threshold for Regularly Checking Idle Connection Availability | keepAliveBetweenTimeMillis | 80000 ms, or 80 s | The duration for keeping idle connections alive, ensuring they won't be inadvertently closed after a period of inactivity. |
Fetchsize Setting
The Fetchsize parameter is used to control the number of records to be processed in batches for obtaining data from the database. A smaller value of the Fetchsize parameter can reduce the memory consumption of each database query but may require more database round trips to retrieve the complete result set. A larger value of the Fetchsize parameter can reduce the number of database round trips but will increase memory usage. You can optimize the database query performance by setting Fetchsize as required.
If the parameter value is less than or equal to 0 (for example: -20), the Fetchsize parameter is disabled.
1. The Fetchsize parameter must be set for the Oracle, DB2, and Postgre databases. If the parameter value is less than or equal to 0, the value will be automatically replaced by 128, 50, and 10000 respectively.
2. The Fetchsize parameter is disabled for other databases by default.
3. This parameter will not be configured if you keep it blank.
SSH Setting
If you have security requirements and need to use SSH, tick Use SSH Tunnel and fill in the corresponding information.
Note: There are two options in Verification Method: Password and Public Key. If you select Public Key, you need to put the file in the path %FineDataLink%/webapps/webroot/WEB-INF/resources.
Click Test Connection. If the connection is successful, click Save to save the configuration.
SSL Setting
Note: SSL Setting is only supported for MySQL databases currently.
1. Put the CA certificate, client certificate, and client secret key in the path %FineDataLink%/webapps/webroot/WEB-INF/resources/certificates.
2. Tick Use SSL Tunnel and add the files accordingly.
3. Click Test Connection. If the connection is successful, click Save to save the configuration.
Data Connection Management
Management Scope
Choose System Management > Data Connection > Data Connection Management. Click the icon on the right of a database.
The following table describes the settings of data connection management.
Setting Item | Description |
---|---|
Test Connection | Test whether the database configuration is successful. |
Edit | Modify the value of the setting item of a data connection. |
Rename | Rename the data connection. |
Copy | Copy a database. The copy name is name of the copied data connection 1. |
Delete | Delete a database. |
Data Connection Information | Display the type of the database and its creator. |
You need to note the following points:
The
icon is not displayed on the right of unsupported databases.
Do not modify the data connection name arbitrarily when managing the data connection. The name of the data connection upon which locally updated tables depend is matched with the database. If the data connection name is modified, tables retrieved from that connection will no longer be updated.
The Edit SQL button will not exist if the database is an SQL dataset.
If the database upon which locally saved tables depend is updated, for instance, transitioning from a development environment to a production environment, and both the table names and structures remain unchanged, you only need to modify the data connection URL for the locally saved table to be updated via the same data connection.
Connection Pool Status
You can view created data connections and their connection status on the Connection Pool Status page, as shown in the following figure.
The following table describes Number of Active Connections and Number of Idle Connections.
Connection Pool Status | Description |
---|---|
Number of Active Connections | The maximum number of connections can be created. If the value is 10, and ten data connections exist in the pool and have been used, the system will pop up an error message "active 10 maxActive 10" when you apply for the eleventh data connection. |
Number of Idle Connections | The number of idle connections that can change into active connections at any time. An idle connection will be dropped automatically if its idle time exceeds the maximum idle connection time. Number of active connections + Number of idle connections = Number of actual physical connections |
Data Connection Timeout Detection
You can set the timeout duration of a data connection in Data Connection Timeout Detection under Data Connection Management > Global Setting.
If a data connection times out, an error message will pop up, saying "31300010 kdc server is not connected."