Version
FineDataLink Version | Functional Change |
---|---|
4.0.23 | Scheduled Task supported the reading from and writing into the Doris database. Pipeline Task supported data writing into the Doris database. |
4.1 | Data Service supported the Doris database. |
Restriction on Use
FineDataLink supports connection to the Doris database. For details about the usage instructions, see Doris Instruction.
Preparation
Version and Driver
Supported Database Version | Driver Package |
---|---|
0.14 and above | (Built-in) No need to download. |
Connection Information Collection
Collect the following information before connecting FineDataLink to the database.
IP address and port number of the database server
Database name
Username and password of the database
Notes
1. Ensure that FineDataLink can access the frontend node (FE) and the backend node (BE) before establishing a data connection.
2. Ensure that the FE can access the BE to write the data into a Doris database.
In the process of loading data into Doris using FineDataLink, local text files or program files are imported using the HTTP-based Stream Load method to the FE, which then uses an HTTP redirect to forward the load request to a BE. Alternatively, FineDataLink can submit the import command to a specified BE. The final result of importing is returned by the BE coordinator. This method is suitable for importing 1 GB to 10 GB of data.
3. Ensure that you have ADMIN privileges on the Doris database for using FineDataLink functions normally.
Procedure
1. Log in to FineDataLink, choose System Management > Data Connection > Data Connection Management, click New Data Connection, and click the Doris icon, as shown in the following figure.

2. Fill in the connection information, as shown in the following figure.
In case of high concurrency, you are advised to increase the maximum number of active connections to prevent connection failures due to high concurrent requests.
Configure the FE to write data into the Doris database. The configuration is described as follows.
Configuration Item | Prompt | Description |
---|---|---|
Frontend Node Address | Fill in the IP address or hostname and the port number, with multiple addresses separated by commas (,). | The FE address is the connection target of Stream Load. The format is IP address: Port number. For example, 192. 168. 101. 71:8030,192. 168. 101. 209:9030. You can enter multiple FE addresses, where the FE uses a polling mechanism to decide which BE will serve as a coordinator to receive and process the load requests. |
3. Click Test Connection. If the connection is successful, click Save to save the configuration.
Data Source Usage
After configuring the data source, you can use it in Data Development. For details, see Overview of Data Development.
After configuring the data source, you can use it in Data Pipeline. For details, see Overview of Data Pipeline.
After configuring the data source, you can use it in Data Service. For details, see Overview of Data Service.