Shell Script

  • Last update: October 24, 2024
  • Overview

    Version Description

    FineDataLink VersionFunctional Change
    4.0.8/
    4.1.0Displayed configured parameters in the drop-down list of Script Parameter.

    Application Scenario

    You want to invoke Kettle tasks, Python-based computing tasks, and Java/Spark programs, and back up and restore databases in FineDataLink.

    Function Description

    You can run shell scripts on a specified server connected using the Shell Script node to execute external independent data processing tasks.

    For example, you can execute SVN updates, manage and clean files, invoke Kettle tasks, Python-based computing tasks, and Spark-based computing tasks, and back up and restore databases. Also, you can use parameters set in FineDataLink in the shell script.

    Restriction on Use

    Only scripts with the .sh extension on Linux systems can be executed.

    Function Description

    Configuration ItemDescription
    SSH ConnectionSelect the SSH connection from the drop-down list. 
    Script Path

    Enter the complete path of the shell script to be executed on the target server of the SSH connection, where: 

    • Only scripts with the .sh extension on Linux systems can be executed.

    • Referencing parameters is supported, as shown in the following example.

    Script Parameter

    1. Define the parameter to be passed to the script, where:

    • Parameters are passed to the script in a top-to-bottom order.

    • Parameters cannot contain single quotes (''). If single quotes are necessary, customize the parameter usage logic for the Shell script accordingly.

    • Unconfigured script parameters are not allowed.

    • You can select existing parameters only. Parameters with the same name are not supported. 

    • Built-in parameters are not allowed.

    2. From version 4.1.0 onwards, the drop-down list displays the configured parameters, which are divided into task parameters and global parameters: 

    • Parameters with identical names are also included, among which the effective ones are arranged by priority from high to low as follows: parameters set in Parameter Assignment nodes, parameters of the task containing an invocation task, custom parameters, global parameters, and built-in parameters.

    • Unavailable parameters are shown in gray.


    Command Timeout Configuration

    Set the maximum execution time for the command. 

    • It is set to Unlimited by default.

    • The command will be terminated if execution exceeds X second(s), with X ranging from 0 to 1000000.

    Character Encoding

    Specify the output character encoding.

    Options in the drop-down list include: 

    • UTF-8

    • GBK 

    • BIG5

    • ISO-8859-1

    Example

    Configuring the SSH Protocol Data Connection

    To execute a specified Shell script on the target server, you must first establish an SSH connection to the server. For details about configuring the SSH data connection, see SSH Data Connection.

    Preparing the Shell Script

    Take the following shell script as an example.

    $0 refers to the shell script name. $1$2, and $3 refer to the parameters (with values assigned in FineDataLink) to be written into the script.

    After executing the script, you get three output parameter values, where $1 returns the data (which is Ketchup in this example) stored at the specified location in the FDL_PRODUCTS table, $2 returns the task execution time, and $3 returns the manually assigned value b3, as shown in the following figure.

    Setting Parameters

    Use the Parameter Assignment node to fetch the $1 parameter from the data table.

    Drag in a Parameter Assignment node and fetch the data to be used as the parameter (which is Ketchup in this example) from the data table, as shown in the following figure.

    Data in the FDL_PRODUCTS table is shown in the following figure.

    Add $2 and $3 parameters in Parameter List, as shown in the following figure.

    Setting the Shell Script Node

    Drag in a Shell Script node, connect it to the Parameter Assignment node, select the SSH connection with the target remote server where the task is executed, enter the script path, click Add Parameter, and add b1b2, and b3 parameters set in the section "Setting Parameters," as shown in the following figure.

    iconNote:
    When a Shell script in a loop container loops through a parameter, the value fetched corresponds to that in the ongoing loop.


    Executing the Task

    Click Run to execute the task, as shown in the following figure.

    You can set Log Level Setting to INFO to get the detailed log, which is shown in the following figure.


    After successful execution, you can see the output parameter values in the output file set in the script, as shown in the following figure.

    Judging the Shell Script Execution Status

    For details, see Description of Shell Script Execution Results.

    Subsequent Operations

    You can set a scheduling plan for the task to execute the shell script periodically. For details, see Scheduling Plan.

    Further Reading

    You can use the following three methods in shell scripts to output information. 

    • Use the echo command: Output information to standard output.

    • Use the printf command: Output formatted text. It is similar to the print function in other programming languages.

    • Use the echo command with redirection: Redirect the output result to a file or another program.

    Here are example codes of each command. 

    Use the echo command to output text:

    echo "Hello, World!"

    Use the printf command to output formatted text:

    printf "Name: %s\nAge: %d\n" "Alice" 30

    Redirect the output to save it to a file:

    echo "This will be saved to a file" > output.txt

    The echo and printf commands output information in the log, as shown in the following figure.

    The information is written to a file without being shown in the log when the echo command with redirection is used.

    附件列表


    主题: Data Development - Scheduled Task
    Previous
    Next
    • Helpful
    • Not helpful
    • Only read

    滑鼠選中內容,快速回饋問題

    滑鼠選中存在疑惑的內容,即可快速回饋問題,我們將會跟進處理。

    不再提示

    10s後關閉

    Get
    Help
    Online Support
    Professional technical support is provided to quickly help you solve problems.
    Online support is available from 9:00-12:00 and 13:30-17:30 on weekdays.
    Page Feedback
    You can provide suggestions and feedback for the current web page.
    Pre-Sales Consultation
    Business Consultation
    Business: international@fanruan.com
    Support: support@fanruan.com
    Page Feedback
    *Problem Type
    Cannot be empty
    Problem Description
    0/1000
    Cannot be empty

    Submitted successfully

    Network busy