Python Script

  • Last update: October 24, 2024
  • Overview

    Version

    FineDataLink VersionFunctional Change
    4.0.28Added a Python Script node.
    4.1.0Displayed configured parameters in the drop-down list of Script Parameter.
    4.1.3Allowed specifying the Python version.

    Application Scenario

    You want to call Python scripts in FineDataLink, as many of your data processing tasks are implemented through Python scripts.

    Function Description

    FineDataLink provides a Python Script node for you to call Python scripts, as shown in the following figure.

    Restriction on Use

    1. The single quotation mark (') is forbidden in the script path and the parameter value if the Python script is stored on a Linux or a macOS system.

    2. The double quotation mark (") is forbidden in the script path and the parameter value if the Python script is stored on a Windows system.

    Differences Between the Python Script Node and the Python Operator

    DifferencePython ScriptPython
    Application ScenarioCall an existing Python script in a scheduled task.Complex data processing that is difficult to implement using visual operators or SparkSQL can be done with Python scripts in the Data Transformation node during data development.
    Where to UseIt belongs to the step flow and can be used with the Data Transformation, Data Synchronization, and other nodes.It belongs to a data flow and is used in the Data Transformation node.
    Usage

    Call Python scripts via an SSH data connection.

    Write Python code in the Python operator after configuring the Python environment.

    Example

    Scenario Description

    The Python script holds the grade information.

    Provide the student's name (with the argument being John in this example) as an input parameter for the Python script to fetch the student's grade. The fetched data is output to an automatically created file named test.txt, located in the Python script's path.

    You can use the example data. Python Script.zip Student Name.xls

    iconNote:
    The remote server in this article uses Python 2.7.5. Modify the script according to actual conditions.


    New SSH Data Connection

    Before using the Python Script node to run a Python script, you need to configure the SSH data connection to connect to the remote target server, as shown in the following figure.

    Fetching the Student's Name and Output It as a Parameter

    1. Fetch the student's name.

    2. Output the fetched data as an argument.

    Set the parameter name to w, as this is the parameter received by the example Python script. Fetch data from all rows in the first column. The default value is only for previewing at downstream nodes and won't be used during actual execution. You can copy the value from Parameter Preview on the right.

    Setting the Python Script Node

    Drag in the Python Script node, set the Python script to be run, and add the script parameter set in the section "Fetching the Student's Name and Output It as a Parameter," as shown in the following figure.

    The following table describes each setting item.

    Setting ItemDescription
    Python CommandSelect the script execution command from the drop-down list. Options include PythonPython2, and Python3.
    SSH ConnectionSelect the SSH connection from the drop-down list. 
    Script Path

    Enter the complete path of the Python script to be executed on the target server of the SSH connection, where:

    • The script file should have the .py extension.

    • Referencing parameters is supported.

    • The script path must be secure against shell injection.

    Script Parameter

    1. Define the parameter to be passed to the script, where:

    • You can select existing parameters only. Parameters with the same name are not supported.

    • The script parameter must be secure against shell injection.

    • Unconfigured script parameters are not allowed.

    • Parameters are passed to the script in a top-to-bottom order.

    2. From version 4.1.0 onwards, the drop-down list displays the configured parameters, which are divided into task parameters and global parameters: 

    • Parameters with identical names are also included, among which the effective ones are arranged by priority from high to low as follows: parameters set in Parameter Assignment nodes, parameters of the task containing the invocation task, custom parameters, global parameters, and built-in parameters.

    • Unavailable parameters are shown in gray.

    Command Timeout Configuration

    Set the maximum execution time for the command. It is set to Unlimited by default.

    The command will be terminated if execution exceeds X second(s), with X ranging from 0 to 1,000,000.

    Character Encoding

    Specify the output character encoding, which defaults to UTF-8

    Options in the drop-down list include:

    • UTF-8 

    • GBK 

    • BIG5

    • ISO-8859-1

    Running Result

    1. Click Run to execute the task. The running result in Log upon successful execution is shown in the following figure.

    2. Check the content of the test.txt file automatically created in the directory of the Python script, which is shown in the following figure.

    附件列表


    主题: Data Development - Scheduled Task
    Previous
    Next
    • Helpful
    • Not helpful
    • Only read

    滑鼠選中內容,快速回饋問題

    滑鼠選中存在疑惑的內容,即可快速回饋問題,我們將會跟進處理。

    不再提示

    10s後關閉

    Get
    Help
    Online Support
    Professional technical support is provided to quickly help you solve problems.
    Online support is available from 9:00-12:00 and 13:30-17:30 on weekdays.
    Page Feedback
    You can provide suggestions and feedback for the current web page.
    Pre-Sales Consultation
    Business Consultation
    Business: international@fanruan.com
    Support: support@fanruan.com
    Page Feedback
    *Problem Type
    Cannot be empty
    Problem Description
    0/1000
    Cannot be empty

    Submitted successfully

    Network busy