Version
FineDataLink Version | Functional Change |
|---|---|
| 1.5 | API input is supported. |
| 4.1.6.1 | Pipeline Task supports the writing of data into the DM database. |
| 4.1.6.3 | Scheduled Task supports the reading of data from the GBase 8s database. Data Service supports databases including GBase8s, KingbaseES, ClickHouse, HANA, DM, and Vertica. |
| 4.1.6.4 | Scheduled Task supports the writing of data into MongoDB. |
| 4.1.7.3 | Pipeline Task supports the writing of data into the HANA database. |
4.1.8.2 | Scheduled Task supported data reading from and writing into the Oscar database. Pipeline Task supported data writing into the Oscar database. Data Service supported data reading from the Oscar database. |
| 4.1.8.3 | Supported Apache Impala 3.4 and 4.1. |
| 4.1.10.4 | Data Pipeline supported data reading from the MySQL 5.5 database. |
| 4.1.11.2 | Scheduled Task supported data reading from and writing into the YMatrix database. Pipeline Task supported data writing into the YMatrix database. Data Service supported the YMatrix database. Database Table Management supported the YMatrix database. |
| 4.1.11.3 | Scheduled Task supported data reading from and writing into the PolarDB PostgreSQL database. Pipeline Task supported data writing to the PolarDB PostgreSQL database. Data Service supports data reading from the PolarDB PostgreSQL database. Database Table Management supported the PolarDB PostgreSQL database. Scheduled Task supported data writing into the Elasticsearch data source. |
| 4.1.11.4 | Scheduled Task supported data writing into the MaxCompute data source. Scheduled Task supported data reading from the FineLogDB (FDL) data source. Added data sources supported by Real-Time Task. |
| 4.1.13.2 | Real-Time Task in Data Development supported data reading from the MQTT and the IBM MQ data sources. Scheduled Task in Data Development supported data reading from and writing into the Amazon S3 data source. |
| 4.1.13.3 | Real-Time Task in Data Development supported data writing into the ClickHouse database. |
| 4.2.0.1 | Scheduled Task supported data reading from and writing into the openGauss database. Database Table Management supported the openGauss database. |
| 4.2.0.2 | Scheduled Task supported data reading from the Kudu table in the Impala database. Pipeline Task supported data writing into the Kudu table in the Impala database. Data Service supported the Kudu table in the Impala database. Scheduled Task supported data reading from the Elasticsearch data source. |
| 4.2.0.3 | For the OceanBase (Oracle) data source: Scheduled Task supported data reading from and writing into it. Pipeline Task supported data writing into it. Database Table Management supported it. Data Service supported it. |
| 4.2.1.2 | Scheduled Task supported data reading from InfluxDB and TDengine databases. |
Types of Data Sources Supported by Data Pipeline
The data sources supported by FineDataLink's Data Pipeline for real-time data synchronization are shown in the following table.
Classification | Type | Data Source | Read | Write | Environment Preparation | Log Parsing Method | DDL Support |
|---|---|---|---|---|---|---|---|
Advanced Data Source | MySQL (Binlog) | MySQL | Non-read-only MySQL databases of V5.5 and above | Non-read-only MySQL databases of V5.6 and above | Binlog | DDL Data Sources Supported by Data Pipeline | |
| SQL Server (CDC) | SQL Server | Enterprise Edition: 2008 2012 2014 2016 2017 2019 Standard Edition: 2016 SP1 2017 2019 | 2000 2005 2008 2012 2014 2016 2017 2019 | CDC | |||
| Oracle (Logminer) | Oracle | Oracle 10g 11g 12c 18c 19c Oracle RAC | Oracle 10g 11g 12c 18c 19c Oracle RAC | Oracle Environment Preparation | LogMiner | ||
| PostgreSQL (wal2json) | PostgreSQL | PostgreSQL 9.4 and above | Postgresql 9.4.7 and 9.5.0 | Oracle Environment Preparation | wal2json | ||
Message Queue | Kafka | 0.10.2 to 3.4 | Not supported currently | ||||
| Db2 (CDC) | Db2 | 9.X 10.X 11.X | Not supported currently | Db2 Environment Preparation | CDC | ||
| SAP HANA | SAP HANA | 2.0 SPS00 to SPS07 | 2.0 SPS00 to SPS07 | SAP HANA Environment Preparation | Trigger | ||
Progressive Data Source | Progressive - Big Data | GaussDB 200 | Not supported currently | GaussDB 200 | |||
| Greenplum | Not supported currently | 5.X and 6.X | If Greenplum (parallel processing) is the target database, database users must have CREATE SCHEMA and CREATE TABLE privileges. During data loading, a schema named fdl_temp will be created to store temporary tables. | ||||
| Doris | Not supported currently | 0.14 and above | |||||
| StarRocks | Not supported currently | 2.3 and above | |||||
| TiDB | Not supported currently | V5.2.0 V5.2.1 | |||||
| ClickHouse | Not supported currently | 20.1.8.41 | |||||
| Amazon Redshift | Not supported currently | Standard version, the cluster-provisioned version, and Amazon Redshift Serverless | |||||
| Apache Impala | Not supported currently | Impala 2.2 Impala 2.3 Impala 2.8 Impala 2.9 Impala 2.10 Impala 3.4 Impala 4.1 Impala 2.10 kudu1.5 | |||||
| SeaboxMPP | Not supported currently | 21.0.3.398 | |||||
| PolarDB PostgreSQL | Not supported currently | PolarDB Oracle2.0 PolarDB pg14 | |||||
| YMatrix | Not supported currently | 5.X | |||||
Basic Data Source | Basic Data Source | DM | Not supported currently | 8.1 | |||
| Oscar | Not supported currently | 7.0 | |||||
| OceanBase (Oracle) | Not supported currently | 4.3.3 |
Types of Data Sources Supported by Scheduled Task
The data sources supported by Scheduled Task in Data Development are shown in the following table.
Classification | Type | Data Source | Supported Database Version | Read | Write |
|---|---|---|---|---|---|
Basic Data Source | Basic Data Source | MySQL | MySQL 5.1 MySQL 5.5 MySQL 5.5.46 MySQL 5.5.5 MySQL 5.6.22 MySQL 5.6.28 MySQL 5.6.29 MySQL 5.6.31 MySQL 5.6.34 MySQL 5.6.35MySQL 5.6.37 MySQL 5.7 MySQL 5.7.16 MySQL 6.5 MySQL 8.0 | √ | √ |
| Oracle | Oracle 9i Oracle 9.2 Oracle 10.2.0 Oracle 11g Oracle 11g2 Oracle 11.2.0.3 Oracle 11.2.0.4 Oracle 11.2.3 Oracle 12c Oracle 10g AIX Oracle 19c | √ | √ | ||
| SQL Server | SQL Server 2000 SQL Server 2005 SQL Server 2008 SQL Server 2012 SQL Server 2014 SQL Server 2016 SQL Server 2019 | √ | √ | ||
| PostgreSQL | PostgreSQL 9.4.7 and 9.5.0 | √ | √ | ||
| OceanBase (Oracle) | 4.3.3 | √ | √ | ||
| openGauss | 3.0.0 and above | √ | √ | ||
| SQLite | / | √ | √  Note: | ||
| HSQL | / | √ | Under development | ||
| IBM Db2 | Db2_7.2 Db2_8.1 Db2_8.2 Db2_9.7 Db2_10.5 | √ | √ | ||
| Informix | V11.7, V11.5, and V12.X | √ | √ | ||
| DM | 8.1 | √ | √ | ||
| SAP HANA | V2.0 | √ | √ | ||
| KingbaseES | KingbaseES V8 | √ | √ | ||
| Oscar | 7.0 | √ | √ | ||
Server Dataset | / | √ | Not supported | ||
Public Data | / | √ | √ | ||
SAP ASE | 15.7 (cluster) 16.0 (cluster) 12.5 (standalone) 15.5 (standalone) | √ | Not supported | ||
| GBase 8s | 8.7 | √ | Not supported | ||
| FineLogDB (FDL) | / | √ | Not supported | ||
Progressive Data Source | Big Data | Impala | Impala 2.2 Impala 2.3 Impala 2.8 Impala 2.9 Impala 2.10 Impala 2.10 kudu1.5 Impala 3.4 Impala 4.1 Impala 2.10 kudu1.5 | √ | Supports data writing into the Kudu table in the Impala database. |
| ClickHouse | 20.1 and 22.6 | √ | √ | ||
| Presto | Presto 0.152 Presto 0.169 | √ | Under development | ||
5.X and 6.X | √ | √ | |||
| PolarDB PostgreSQL Data Connection | PolarDB Oracle2.0 PolarDB pg14 | √ | √ | ||
| YMatrix | 5.X | √ | √ | ||
| Hive | Hive 1.1 Hadoop Hive 1.2 Hive 2.3 Hive 2.1.2 Hive 2.1.1 Hive 3.3.1 | √ | √ | ||
| GaussDB 200 | / | √ | √ | ||
| GBase 8a | / | √ | √ | ||
| HP Vertica | V7.2.1 to V10.0 V12.0.3 | √ | √ | ||
| Doris | 0.14 and above | √ | √ | ||
| Transwarp Inceptor | V4.6 V4.9.2 V5.2.1 V6.0.2 | √ | √ | ||
| StarRocks | 1.19 and above | √ | √ | ||
| MaxCompute | MaxCompute JDBC V3.2.7 and above | √ | √ | ||
| TiDB | V5.2.0 V5.2.1 | √ | √ | ||
| Amazon Redshift | Cluster-provisioned version and Amazon Redshift Serverless | √ | √ | ||
| SeaboxMPP | 21.0.3.398 | √ | √ | ||
| NoSQL | MongoDB | V7 and earlier releases | √ | √ | |
| Elasticsearch Data Connection | 7.0 and above cluster and non-cluster versions | ||||
| SAP ERP | / | √ | Under development | ||
API Protocol (Timed) | API | / | √ | √ | |
| WebService | / | √ | Under development | ||
File System | FTP/SFTP | / | √ | √ | |
| Local File | / | √ | √ | ||
| Amazin S3 | / | √ | √ | ||
| Time Series Database | InfluxDB | 1.X and 2.X | √ | Not supported | |
| TDengine | 2.X and 3.X | √ | Not supported |
Data Sources Supported by Data Service
The data sources supported by FineDataLink's Data Service are shown in the following table.
| Classification | Type | Data Source | Supported Database Version |
|---|---|---|---|
Basic Data Source | Basic Data Source | MySQL | MySQL 5.1 MySQL 5.5 MySQL 5.5.46 MySQL 5.5.5 MySQL 5.6.22 MySQL 5.6.28 MySQL 5.6.29 MySQL 5.6.31 MySQL 5.6.34 MySQL 5.6.35 MySQL 5.6.37 MySQL 5.7 MySQL 5.7.16 MySQL 6.5 MySQL 8.0 |
| Oracle | Oracle 9i Oracle 9.2 Oracle 10.2.0 Oracle 11g Oracle 11g2 Oracle 11.2.0.3 Oracle 11.2.0.4 Oracle 11.2.3 Oracle 12c Oracle 10g AIX Oracle 19c | ||
| SQL Server | SQL server 2000 SQL server 2005 SQL server 2008 SQL server 2012 SQL server 2014 SQL server 2016 SQL server 2019 | ||
| PostgreSQL | Postgresql 9.4.7 and 9.5.0 | ||
| OceanBase (Oracle) | 4.3.3 | ||
FineBI Public Data | / | ||
| IBM Db2 | 10 and above | ||
| KingbaseES | KingbaseES V8 | ||
| SAP HANA | V2.0 | ||
| DM | 8.1 | ||
| Oscar | 7.0 | ||
| GBase 8s | 8.7 | ||
| Progressive Data Source | Big Data | Greenplum | 5.X and 6.X |
| Doris | 0.14 and above | ||
| TiDB | 5.2.0 5.2.1 | ||
| Impala | 2.2, 2.3, 2.8, 2.9, 3.4, 4.1, and Impala 2.10 kudu1.5 | ||
| Transwarp Inceptor | V4.6, V4.9.2, V5.2.1, V6.0.2 | ||
| StarRocks | 1.19 and above | ||
| GaussDB 200 | Latest version of DWS | ||
| MaxCompute | MaxCompute JBDC 3.2.7 and above | ||
| ClickHouse | 20.1 and 22.6 | ||
| PolarDB PostgreSQL | PolarDB Oracle2.0 PolarDB pg14 | ||
| HP Vertica | V9.3.1 |
Data Sources Supported by Database Table Management
Classification | Type | Data Source | Supported Database Version |
|---|---|---|---|
Basic Data Source | Basic Data Source | MySQL | MySQL 5.1 MySQL 5.5 MySQL 5.5.46 MySQL 5.5.5 MySQL 5.6.22 MySQL 5.6.28 MySQL 5.6.29 MySQL 5.6.31 MySQL 5.6.34 MySQL 5.6.35MySQL 5.6.37 MySQL 5.7 MySQL 5.7.16 MySQL 6.5 MySQL 8.0 |
| Oracle | Oracle 9i Oracle 9.2 Oracle 10.2.0 Oracle 11g Oracle 11g2 Oracle 11.2.0.3 Oracle 11.2.0.4 Oracle 11.2.3 Oracle 12c Oracle 10g AIX Oracle 19c | ||
| SQL Server | SQL Server 2000 (supported in FineDataLink 4.2.1.1 and later versions) SQL Server 2005 SQL Server 2008 SQL Server 2012 SQL Server 2014 SQL Server 2016 SQL Server 2019 | ||
| PostgreSQL | PostgreSQL 9.4.7 and 9.5.0 | ||
| OceanBase (Oracle) | 4.3.3 | ||
| openGauss | 3.0.0 and above | ||
| SAP HANA | V2.0 | ||
| IBM Db2 | Db2_7.2 Db2_8.1 Db2_8.2 Db2_9.7 Db2_10.5 | ||
| SQLite | / | ||
| HSQL | / | ||
| Informix | V11.7, V11.5, and V12.X | ||
| KingbaseES | KingbaseES V8 | ||
| DM | 8.1 | ||
Progressive Data Source | Big Data | Doris | 0.14 and above |
| StarRocks | 1.19 and above | ||
5.X and 6.X | |||
| YMatrix | 5.X | ||
| Hive | Hive 1.1 Hadoop Hive 1.2 Hive 2.3 Hive 2.1.2 Hive 2.1.1 Hive 3.3.1 | ||
| Transwarp Inceptor | V4.6 V4.9.2 V5.2.1 V6.0.2 | ||
| Impala | Impala 2.2 Impala 2.3 Impala 2.8 Impala 2.9 Impala 2.10 Impala 2.10 kudu1.5 Impala 3.4 Impala 4.1 | ||
| ClickHouse | 20.1 and 22.6 | ||
| PolarDB PostgreSQL Data Connection | PolarDB Oracle2.0 PolarDB pg14 | ||
| Presto | Presto 0.152 Presto 0.169 | ||
| HP Vertica | V7.2.1 to V10.0 V12.0.3 | ||
| MaxCompute | MaxCompute JDBC V3.2.7 and above | ||
| GBase 8a | / | ||
| GaussDB 200 | / | ||
| TiDB | V5.2.0 V5.2.1 | ||
| Amazon Redshift | Cluster-provisioned version and Amazon Redshift Serverless | ||
| SeaboxMPP | 21.0.3.398 |
Data Sources Supported by Real-Time Task
The data sources supported by Real-Time Task in Data Development are shown in the following table.
Classification | Type | Data Source | Supported Database Version | Read | Write |
|---|---|---|---|---|---|
Basic Data Source | Basic Data Source | MySQL | MySQL 5.1 MySQL 5.5 MySQL 5.5.46 MySQL 5.5.5 MySQL 5.6.22 MySQL 5.6.28 MySQL 5.6.29 MySQL 5.6.31 MySQL 5.6.34 MySQL 5.6.35MySQL 5.6.37 MySQL 5.7 MySQL 5.7.16 MySQL 6.5 MySQL 8.0 | √ | √ |
| Oracle | Oracle 9i Oracle 9.2 Oracle 10.2.0 Oracle 11g Oracle 11g2 Oracle 11.2.0.3 Oracle 11.2.0.4 Oracle 11.2.3 Oracle 12c Oracle 10g AIX Oracle 19c | √ | √ | ||
| SQL Server | SQL Server 2000 SQL Server 2005 SQL Server 2008 SQL Server 2012 SQL Server 2014 SQL Server 2016 SQL Server 2019 | √ | √ | ||
| PostgreSQL | PostgreSQL 9.4.7 and 9.5.0 | √ | √ | ||
| IBM Db2 | 10 and above | √ | / | ||
| SAP HANA | V2.0 | Under development | √ | ||
| DM | 8.1 | Under development | √ | ||
| Progressive Data Source | Big Data | 5.X and 6.X | Under development | √ | |
| Doris | 0.14 and above | Under development | √ | ||
| TiDB | V5.2.0 V5.2.1 | Under development | √ | ||
| StarRocks | 1.19 and above | Under development | √ | ||
| GaussDB 200 | Latest version of DWS | Under development | √ | ||
| ClickHouse | 20.1.8.41 | Under development | √ | ||
| Amazon Redshift | Cluster-provisioned version and Amazon Redshift Serverless | √ | √ | ||
| Advanced Data Source | Message Queue | Kafka | / | √ | √ |
| IBM MQ | IBM MQ V9.4.0.5 (p940-005-240823) | √ | Not supported currently | ||
| Pulsar | Pulsar | / | √ | √ | |
API Protocol (Timed) | MQTT | 2.0.18 | √ | Not supported currently |
Previous：Overview of Data Connection
