Overview
Version
| FineDataLink Version | Functional Change |
|---|---|
| 4.1.11.3 | / |
Application Scenario
You want to write the data to be queried (which is fetched from the relational database) to Elasticsearch, which can be used as an acceleration layer to facilitate subsequent queries or as an ADS layer of the data warehouse.
Function Description
The Elasticsearch Output operator allows you to output the processed data to the specified Elasticsearch database.
Note:
Prerequisite
To output data to Elasticsearch, you must have configured the data connection to Elasticsearch, and have permission to use the data connection. For details, see Elasticsearch Data Connection.
You have registered the NoSQL function point.
Function Description
The Data Destination and Mapping tab page is shown in the following figure.
| Setting Item | Description |
|---|---|
| Data Connection | Select a configured Elasticsearch data connection. |
Target Index |
Definition: An index is the basic data storage unit in Elasticsearch, similar to a database in a relational database. Each index contains many documents with the same defined data structure (mapping). Usage: Indexes are used to store specific types of datasets. As each index supports only one data type starting from V 7.0, creating one index for each data type is a common practice. |
Field Mapping | Choose between Map Fields with Same Name and Map Fields in Same Row to generate the default mapping.
|
The Write Method tab page is shown in the following figure.
Setting Item | Description |
Write Data into Target Index Directly | Primary Key Mapping: You can select the field as the logical primary key and configure the mapping to ensure data uniqueness. Strategy for Primary Key Conflict:
|
Write Data into Target Index After Emptying it | It works the same way as the one in the Data Synchronization node. |
Add/Modify/Delete Data Based on Identifier Field | The configuration of Primary Key Mapping is consistent with that when Write Data into Target Index Directly is used. |
Procedure
You can fetch the data to be queried from the relational database and write it into Elasticsearch, which can be used as an acceleration layer to facilitate queries.
Task Configuration
Create a scheduled task, drag a Data Transformation node onto the page, click the Data Transformation node to enter the editing page, and drag a DB Input operator onto the page, as shown in the following figure.
Process data, as shown in the following figure.
Add an Elasticsearch Output operator. Run the task to write data into Elasticsearch to facilitate subsequent queries, as shown in the following figure.
Effect Display
You can use the RESTful API of Elasticsearch to query the data written to the database through HTTP requests, as shown in the following figure.
Previous：Dataset Output
