Spark SQL Mathematical Function

This document introduces the syntax of the mathematical functions in Spark SQL.

Description: Rrounds a number to a specified number of digits.

Syntax: ROUND(Expression, Decimal Place). 

• The Expression value is the number to be rounded. 

• The Decimal Place is the specified number of the decimal place to round to. If Decimal Place is left blank, the value will be rounded to the nearest integer. For example, round(3.1415926) and round(3.1415926,0) return the same result.

Example one: select round(3.1415926) Result: 3

Example two: select round(3.1415926,2) Result: 3.14

You are advised to use the ROUND function in New Calculation Column of FineDataLink.

bround(DOUBLE a): Banker's rounding (1~4: round down, 6~9: round up, 5-> round down if the preceding digit is even, 5-> round up if the preceding digit is odd).

bround(DOUBLE a, INT d): banker's rounding. The value is rounded to d decimal places.

Example 1: select bround(4.5) Result: 4

Example 2: select bround(3.145,2) Result: 3.14

floor(DOUBLE a): Rounds a value down to the nearest integer, that is, returns the nearest value on the left of the desired value on the axis.

Example: select floor(3.14)

You are advised to use the Floor function in New Calculation Column of FineDataLink.

ceil(DOUBLE a): Returns the smallest integer greater than or equal to the given real number.

ceiling(DOUBLE a): Returns the smallest integer greater than or equal to the given real number.

Example 1: select ceil(-4.1)

Example 2: select ceiling(-4.1)

You are advised to use the CEILING function in New Calculation Column of FineDataLink.

rand(): Returns a random value between 0 and 1.

rand(INT seed): Returns a random number between 0 and 1. If you specify a seed, a stable sequence of random numbers will be generated.

Example 1: select rand(), rand()

Example 2: select rand(8), rand(8)

You are advised to use the RAND function in New Calculation Column of FineDataLink.

exp(DOUBLE a), exp(DECIMAL a): Returns the value of e raised to the power of a, where a can be a decimal.

Example: select exp(0), exp(1.11)

You are advised to use the EXP function in New Calculation Column of FineDataLink.

ln(DOUBLE a), ln(DECIMAL a): Returns the logarithm of a to the base of the natural constant e, where a can be a decimal.

Example: select ln(2), ln(2.1)

You are advised to use the LN function in New Calculation Column of FineDataLink.

log10(DOUBLE a) or log10(DECIMAL a): Returns the base 10 logarithm of a , where a can be a decimal.

Example: In Data Transformation, you can use the following statement in Spark SQL, as shown in the following figure.

select log10(10)

In Data Transformation, you can use the following statement in Spark SQL, as shown in the following figure.

select log10(10.11)

10. LOG 2

log2(DOUBLE a) or log2(DECIMAL a): Returns the base 2 logarithm of a, where a can be a decimal.

Example: select log2(2)  The result is shown in the following figure.

log(DOUBLE base, DOUBLE a), log(DECIMAL base, DECIMAL a): Returns the logarithm of a to base.

Example: select Column 1, Column 2, log(Column 1, Column 2) from TV

You are advised to use the LOG function in New Calculation Column of FineDataLink.

pow(DOUBLE a, DOUBLE p), power(DOUBLE a, DOUBLE p): Calculates the value of a to the power of p.

Example: select Column 1, Column 2, pow (Column 1, Column 2) from TV

You are advised to use the POWER function in New Calculation Column of FineDataLink.

sqrt(DOUBLE a), sqrt(DECIMAL a): Calculates the square root of a.

Example: select Column 1, Column 2, sqrt(Column 1), sqrt(Column 2) from TV

You are advised to use the SQRT function in New Calculation Column of FineDataLink.

abs(DOUBLE a), abs(DECIMAL a): Calculates the absolute value of a.

Example: select abs(-3.1), abs(0)

You are advised to use the ABS function in New Calculation Column of FineDataLink.

15. PMOD

pmod (INT a, INT b), pmod (DOUBLE a, DOUBLE b): a is divided by b to get the remainder.

Example: select pmod(2,3), pmod(-4,2)

You are advised to use the MOD function in New Calculation Column of FineDataLink.

You are advised to use the functions in New Calculation Column of FineDataLink.

e(): the mathematical constant e.

pi(): the mathematical constant π.

Example: select e(), pi() The result is shown in the following figure.

negative(INT a) or negative(DOUBLE a): Returns the opposite number of a.

Example: select negative(1) The result is shown in the following figure.

factorial(INT a): Calculates the factorial of a.

Example: select factorial(5) The result is shown in the following figure.

cbrt(DOUBLE a): Calculates the cube root of a.

Example: select cbrt(512) The result is shown in the following figure.

greatest(Expression 1, Expression 2, ...): Returns the maximum value from a list of one or more expressions.

least(Expression 1, Expression 2, ...): Returns the minimum value from a list of one or more expressions.

Example 1: select greatest(Column 1, Column 2) from TV

Example 2: select least(Column 1, Column 2) from TV

You are advised to use MAX and MIN in New Calculation Column of FineDataLink.