FineDataLink Version
Functional Change
3.6.2
Added a Spark SQL operator to the Data Transformation node to achieve flexible data transformations.
4.0.17
Allowed using encryption-related functions and variables in the Spark SQL operator.
Allowed using the Spark SQL operator as a data source to input parameters or constants.
Supported TO_JSON and STRUCT functions.
This document introduces the commonly used Spark SQL syntax to help you use the Spark SQL operator for data processing and development.
1. For details about the usage of the Spark SQL operator, see Spark SQL.
2. Processing escape sequences using Spark SQL is not supported currently.
Type
Document
Commonly used operators and syntax in Spark SQL
Common Spark SQL Syntax
Supported conditional functions in Spark SQL
Spark SQL Conditional Functions
Supported aggregate functions in Spark SQL
Spark SQL Aggregate Functions
Supported window functions in Spark SQL, which can be used to process and calculate grouped data
Spark SQL Window Functions
Supported encoding functions in Spark SQL, which can be used to generate Tokens for fetching API data
Supported string functions in Spark SQL, which can be used to process strings
Date functions in Spark SQL, which can be used to process date data
Spark SQL Date Functions
Mathematical functions in Spark SQL, which can be used for mathematical calculations
Spark SQL Mathematical Functions
For details about more functions and usage, see Spark SQL Syntax. (This website is not maintained by the FDL help document team and may be inaccessible sometimes.)
Problem:
An error occurred when the query statement SELECT split('apple,banana,orange', ',') FROM $[DB table input] was used in the Spark SQL operator, indicating "Unknown data type:array - Unknown data type:array."
Cause:
Spark SQL supports the calculation of array-type data, but cannot be used to output array-type data.
滑鼠選中內容，快速回饋問題
滑鼠選中存在疑惑的內容，即可快速回饋問題，我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy