Data Synchronization - API

Version 

The setting items of API Input in the Data Synchronization node are the same as those of the API Input operator. For the functional changes, see the section "Version" of API Input.

Application Scenario 

Due to security concerns, direct access to the database is sometimes restricted, and data is made available through APIs. You can use APIs to extract data and store it in relevant databases in this case.

FineDataLink provides two functions for API-based data fetching, with the following differences: 

Function Description 

FineDataLink supports API-based data fetching. This article describes how to fetch data from an API using the Data Synchronization node.

1. Read API Input carefully to understand the prerequisites, usage restrictions, and descriptions of each setting item for fetching data from the API using FineDataLink.

2. You can perform simple JSON parsing of the data fetched from an API under Data Synchronization > API Input and save the data to a database, a file system, or other places, as shown in the following figure. 

The Expand Parsed JSON Data into a Two-Dimensional Table function in Return Value Processing has certain limitations compared with the JSON Parsing operator. For details, see API Input.

API Information Collection 

This section takes the FineDataLink-released API as an example to introduce how to fetch data from an API using the API Input function in the Data Synchronization node.

You can download the document and view the API information: app_test_API Document.docx. This API is created using Data Service.

{
   "paging": {
      "pageSize": 10,
      "pageNum": 1
   },
   "params": [
   {
   "name": "name",
   "value": " "
   }
  ]
}

{
   "totalNum": 1,
   "errCode": 0,
   "pageSize": 10,
   "pageNum": 1,
   "rowCount": 1,
   "message": "success",
   "data": [      
   {         
   "score": "",         
   "name": "",         
   "id": 0      
   }   
  ]
 }
Expand

Scheduled Task Setting 

1. Create a scheduled task and drag the Data Synchronization node into it.

2. Set the request method, the API URL, and the content of Body in the Data Synchronization node according to the API document, as shown in the following figure.

(Optional) Enter $.data in the input box of Response Body Processing to parse the data field. Click Data Preview, as shown in the following figure.

The fetched data without being parsed is shown in the following figure.

3. Click Data Destination and Mapping, where you can store the fetched data in the database, as shown in the following figure.

4. Set Write Method to Add.

Effect Display 

The log after the task runs is shown in the following figure.

The database data is shown in the following figure.

For details about API-based data fetching, see Instructions on API-Based Data Fetching. 

For details about examples of API-based data fetching, see Overview of API-Based Data Fetching.