Description of Loop Container Logic

Version

Application Scenario

• You need to fetch data by calling an API, and the parameter values required for calling this API are derived from data in a specific column in the database. Since an API accepts only a single value for a given parameter at one time, you need to obtain the values of the specific column in the database, output each as a parameter value to the same new parameter, and traverse all parameter values to fetch data from the API.

• When the data volume is large, you may want to fetch data in time-based batches until all time ranges are covered.

Function Overview

The combination of the Loop Container node and parameters allows you to fetch data in batches, as shown in the following figure.

• To use the Loop Container function, you must register the function point.

• Parameters output by the Parameter Assignment node can only be used by its downstream node. This logic also applies to the Parameter Assignment node in the Loop Container node.

• Nodes that cannot be dragged into a Loop Container node include Invocation Task and Loop Container.

For-Each Loop

Usage Instruction

The Loop Container node requires an upstream parameter outputting node, such as Parameter Assignment or Data Transformation (with a Parameter Outputoperator in it). You can use Loop Container to traverse all parameter values to fetch data from the database/API.

Note:

Parameters used in each iteration come from nodes outside the Loop Container node or the upstream node of the Loop Container node.

This document uses an example to illustrate Do-While Loop. For details, see Loop Container Example - API-Based Data Fetching.

Scenario:

The API includes a parameter named Data_ID, which allows you to fetch the specific data associated with the given data ID from the API.

Data IDs (Data_ID values) are stored in a database. You want to fetch data records with these IDs from the API, retrieving one data record per API call.

Solution:

Use the Parameter Assignment node to fetch data IDs from the database and output them as parameter values. Pass these values through the parameter to the Loop Container node.

Drag a Data Synchronization node into the Loop Container node and configure it to fetch data from the API. The loop processes one data ID per iteration and terminates upon exhausting all data IDs.

Note:

You are advised to set a debug value (default value) for the parameter in the Parameter Assignment node to facilitate effect preview at downstream nodes that reference the parameter. The debug value does not participate in actual execution.

Loop Limit

Loop Limit refers to the maximum number of iterations, which defaults to 128 times. You can manually adjust it, with an upper limit of 10,000 times. Loop Limit cannot be left empty.

Note:

If more than 10,000 iterations are needed, you can use the Unlimited Number of Loops for Loop Container plugin in FineDataLink of V4.1.10.2 and later versions.

Loop Type

You can select For-Each Loop or Do-While Loop as Loop Type.

1. For-Each Loop iterates through elements in the loop parameter (array). The loop terminates upon exhausting all elements, with the iteration count matching the element count of the target iterable. The iteration count is determined by the field value count.

• If the current element is not null, the Loop Container node will return true and proceed to the next iteration.

• If the current element is null, the Loop Container node will return false and terminate the loop.

2. For details about Do-While Loop, see the "Do-While Loop" section.

Object

Select the parameter output by the upstream node. Values of the selected parameter are used in iterations one by one. The value of an unselected parameter will be used in all iterations.

If you select For-Each Loop as Loop Type, you must configure Object.

If you select Do-While Loop as Loop Type, Object is optional.

Set Object to the values of the parameters output by the upstream Parameter Assignment node of the Loop Container node. You can select a single parameter or multiple parameters, but you can only select those parameters output by the upstream node of the Loop Container node.

Execution Condition

When you select For-Each Loop as Loop Type, you do not need to configure Execution Condition.

Fault Tolerance Mechanism

You can use this function if you want the Loop Container node to continue iteration upon node errors.

• By default, Continue on Node Error is not ticked. If a node reports an error during one iteration, the next iteration will be canceled.

• When Continue on Node Error is ticked, if a node in the Loop Container node reports an error, the Loop Container node continues the next iteration, but the execution result of this Loop Container node is still a failure. (The error information of the failed iteration is output in the log.)

Do-While Loop

Implementation Method

To select Do-While Loop as Loop Type, the following two conditions should be met simultaneously.

• Parameters for fetching data are available in each iteration: Set the parameter required for data fetching in each iteration using the Parameter Assignment node, Parameter Output, or built-in parameters. Parameters can be output by nodes outside or inside the Loop Container node.

• The loop termination condition is set: Set the loop termination condition according to actual scenarios, and the loop will end when the condition is not met. Parameters can be output by nodes outside or inside the Loop Container node.

This document uses an example to illustrate Do-While Loop. For details, see the "Solution Two" section in Paginated Data Fetch - Data Fetch Based on Page Number.

Scenario:

The parameters included in the API document are shown in the following figure, where pageIndex represents the page number of data and pageSize represents the number of data records per page:

A company now needs to extract all data of a certain business ticket for analysis.

Solution Description:

Loop Type

For details about Loop Limit, Object, and Fault Tolerance Mechanism, see the "For-Each Loop" section.

Do-While Loop:

The loop body is executed once, and then Loop Container determines whether to stop the loop based on the condition. You need to set the conditions for loop execution and termination which determine the iteration count.

• If the conditional judgment result is true, Loop Container proceeds to the next iteration.

• If the conditional judgment result is false, Loop Container terminates the loop.

Starting from FineDataLink V4.1.1, the conditional judgment logic has been standardized. For details, see Conditional Judgment Logic Description.

Execution Condition

When you set Loop Type to Do-While Loop, you need to configure Execution Condition.

You need to set the termination condition. The parameters selected in Execution Condition can be the parameters output by nodes outside/inside the Loop Container node, or custom parameters.

The conditional loop in the Loop Container node operates as a do-while loop. Its key characteristic is that the first iteration executes unconditionally regardless of execution condition satisfaction, while subsequent iterations require condition satisfaction.

For example, if Execution Condition is set to ${loopTimes}<1, during the first iteration, although the condition is not met as the initial value of ${loopTimes} is 1, the iteration will still be executed. After the first iteration is complete, the loop will terminate.

Built-in Parameter

For details, see ${loopTimes} - Current Loop Times in the Loop Container.

Rules for Parameters Inside the Loop Container Node

Parameter Usage

You can use the Parameter Assignment node in the Loop Container node to output parameters whose values will form parameters inside the Loop Container node:

• Parameters inside the Loop Container node can only be used within the Loop Container node and cannot be used by nodes outside the Loop Container node.

• Parameters output by the nodes within the Loop Container node have the highest priority, prior to all parameters output by nodes outside the Loop Container node and built-in parameters.

• The number of field values output by a single parameter in the Parameter Assignment node is restricted to a maximum of 10,000. 

Note:

If there are more than 10,000 parameter values, you can use the Unlimited Number of Loops for Loop Container plugin in FineDataLink of V4.1.10.2 and later versions. 

Parameter Effect Display 

Application Scenario: 

If you set multiple debug values for a parameter, these values will be incorrectly recognized as a single value, preventing you from previewing data properly. In this case, you need to go back to the upstream node to adjust the debug value. 

Function Description:

In FineDataLink of V4.2.4.3 and later versions, if you set multiple debug values for a parameter, you can simulate iterations in Data Preview in the nodes within the Loop Container node, as shown in the following figure.

Note:

A maximum of 5 iterations can be simulated. The parameter value used in each iteration is the debug value you enter.

Upgrade Compatibility Instruction

If parameters are referenced in the Loop Container node, the single quotes ('') of the parameters will no longer be automatically removed. Therefore, you do not need to add the single quotes ('') manually when you use text type (enabling Closure Character) parameters in the Loop Container node.

In the scenario where both the Parameter Assignment node and the Loop Container node are used, if you enable Closure Character when setting the separator of the parameters output by the Parameter Assignment node, you need to manually remove the single quotes ('') added when you use the parameter in the Loop Container node.

You do not need to connect nodes in the container to the Loop Container node with connectors, as there are hidden start and end nodes in the Loop Container node.

When a Loop Container node is created, three internal nodes are automatically generated in it: a start node (loop start node), a loop task node, and an end node (loop termination judgment node). These internal nodes form an internal node flow that enables the loop execution.

1. If a scheduled task contains a Loop Container node, after the task is executed, the latest five iteration records will be displayed on the Log tab page, as shown in the following figure.

You can set Log Level Setting to adjust the detail level of logs under Task Control > Task Attribute.

2. You can view the read and write information of the Data Synchronization node and the Data Transformation node on the Statistics tab page.

If the Loop Container node contains a Data Synchronization or a Data Transformation node, you can click the Loop Container node to view the detailed read and write information, including all successful read and write records.

3. If you tick Continue on Node Error in the Loop Container node and the number of iterations is relatively large, you are advised to check the fanruan.log file for the reasons of failures (if any), as the number of failed iterations cannot be viewed in the Loop Container node.