Overview
You can deploy FineDataLink 4.1 projects via FineOps.
This document provides multiple standard schemes based on the number of scheduled tasks, real-time tasks, and pipeline tasks. Choose the one that best meets your needs.
Scheme Applicability
The schemes in this document are designed for new project deployment scenarios, providing recommended server configuration based on business usage. You can select the scheme according to the task quantity.
The schemes in this document are intended for scenarios where the volume of data (calculated by Row count * Column count) changed in an hour in a single business database does not exceed one million. If the data change volume surpasses this limit, contact FanRuan technical support personnel for further assistance.
These schemes offer resource recommendations based on typical scales of customers’ projects. If the number of tasks exceeds the applicable range, contact FanRuan technical support personnel for further assistance.
This document provides recommended configuration for newly deploying all FineDataLink components. If you have prepared some components such as the configuration database and the file service, you can lower the corresponding server configuration. For details, see notes of each scheme.
As usage increases over time, if your current server resources become insufficient and require expansion, contact FanRuan technical support personnel for assistance.
Introduction
| Scheme | Applicable Condition | Server Configuration |
|---|---|---|
| Minimum Requirement | The number of scheduled tasks is less than 35. Pipeline tasks are not needed. | It is designed for customers with minimal business usage only, where FineOps Basic Edition instead of FineOps Standard Edition should be used. Monitoring indicators in Tracing and Health Inspection are unavailable. Server: 8-core CPU, 16 GB of memory, and 300 GB of disk space It is used to deploy one FineDataLink component, one Nginx component (an internal gateway), and one MySQL component (a configuration database). It is used to deploy all FineOps components excluding Elasticsearch and SkyWalking OAP. |
| Scheme One | The number of scheduled tasks is less than 150. Pipeline tasks are not needed. | Server: 8-core CPU, 32 GB of memory, and 500 GB of disk space It is used to deploy one FineDataLink component, one Nginx component, and one MySQL component. It is used to deploy all FineOps components. |
| Scheme Two | The number of scheduled tasks is less than 300. The number of pipeline tasks is less than 16. The number of real-time tasks is less than 20. | Server: 16-core CPU, 64 GB of memory, and 2 TB of disk space It is used to deploy one FineDataLink component, one Kafka component, one Nginx component, and one MySQL component. It is used to deploy all FineOps components. |
| Scheme Three | The number of scheduled tasks is less than 500. The number of pipeline tasks is less than 16. The number of real-time tasks is less than 40. | 1. FineDataLink main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space It is used to deploy one FineDataLink component. 2. Server for FineDataLink components and FineOps: 8-core CPU, 32 GB of memory, and 2 TB of disk space It is used to deploy one Kafka component, one Nginx component, and one MySQL component. It is used to deploy all FineOps components. |
| Scheme Four | The number of scheduled tasks is less than 500 * N. The number of pipeline tasks is less than 16 * N. The number of real-time tasks is less than 40 * N. | 1. The number of FineDataLink main application servers is N. 1. Each main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space Each one should be deployed with one FineDataLink component. 2. Server for FineDataLink components and FineOps: 8-core CPU, 32 GB of memory, and 2 TB of disk space It is used to deploy one Kafka component, one Nacos component, one Nginx component, and one MySQL component. It is used to deploy all FineOps components. |
Minimum Configuration Scheme
Note:
This scheme is intended for customers with little business usage and limited server resources only.
This scheme is only applicable to deploy FineOps Basic Edition instead of FineOps Standard Edition. Monitoring indicators in Tracing and Health Inspection are unavailable.
It requires one server with 8-core CPU, 16 GB of memory, and 300 GB of disk space.
It is used to deploy one FineDataLink component, one Nginx component, and one MySQL component.
It is used to deploy all FineOps components excluding Elasticsearch and SkyWalking OAP.
| Configuration | Server | |
|---|---|---|
Component to be installed | It is used to install all components of FineDataLink and FineOps. 1. For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components. Note:
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components. Note:
| |
| Basic server requirement | Non-VM (Recommended) | Due to the nature of virtual machines, such as competing for resources, unexpected system failures may occur. Therefore, you are not advised to deploy FanRuan applications on virtual machines. |
| Operating system | Operating system type | Linux |
| Operating system architecture | X86_64 | |
| Operating system kernel | V3.10 and later versions | |
| Operating system software | Recommended: Ubuntu 22 Supported: Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.) CentOS 7.3 to 7.9 Red Hat 7.6 and later releases Rocky Linux 8.8 to 9.4 Note:1. Since CentOS is discontinued, you are advised to use Ubuntu. 2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator). | |
| CPU | CPU core quantity | 8 cores |
CPU clock speed | 2.5 GHz and above | |
| Memory | Remaining free physical memory | 16 GB |
| Disk | Remaining free disk space | The server shall have a partition with remaining free space of more than 300 GB. Note : |
| Mounting directory | Mounting directory preparation | For details, see Preparing the FineDataLink Mounting Directory. For details, see Preparing the FineOps Server. Note: |
| File system auto-mounting | Ensure that the file system of the mounting directory is configured to be automatically mounted during a boot. Otherwise, containers may fail to access these directories, leading to data loss or application startup issues. | |
| Non-shared path | The mounting directory cannot be a shared-use path. Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container. | |
| Permission and command | Tar command | Ensure the server has the tar command installed. The tar command is a commonly used tool for packaging and compressing files. FineOps requires this command for file extraction. |
| Sed command | Ensure the server has the sed command installed. The sed command is used for text processing. FineOps requires this command for text processing. | |
| Sudo privilege | The server user responsible for deploying the project must have the necessary sudo privileges. 1. Using the root user (superuser) for project deployment and operation is preferred. 2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation. Note: | |
| Port and network | Intranet latency | Less than 1 ms |
| Server intranet bandwidth (between the application and the business database) | At least 100 Mbp/s (12.5 MB/s) | |
| Server extranet bandwidth (between the project and the client) | At least 100 Mbps (12.5 MB/s) | |
Component-used port and component port interconnection | For details, see Confirming Server Network of the FineDataLink Project. For details, see Preparing the FineOps Server. | |
Scheme One
It requires one server with 8-core CPU, 32 GB of memory, and 500 GB of disk space.
It is used to deploy one FineDataLink component, one Nginx component, and one MySQL component.
It is used to deploy all FineOps components.
| Configuration | Server | |
|---|---|---|
Component to be installed | It is used to install all components of FineDataLink and FineOps. 1. For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components. Note:
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components. Note :
| |
| Basic server requirement | Non-VM (Recommended) | Due to the nature of virtual machines, such as competing for resources, unexpected system failures may occur. Therefore, you are not advised to deploy FanRuan applications on virtual machines. |
| Operating system | Operating system type | Linux |
| Operating system architecture | X86_64 | |
| Operating system kernel | V3.10 and later versions | |
| Operating system software | Recommended: Ubuntu 22 Supported: Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.) CentOS 7.3 to 7.9 Red Hat 7.6 and later releases Rocky Linux 8.8 to 9.4 Note:1. Since CentOS is discontinued, you are advised to use Ubuntu. 2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator). | |
| CPU | CPU core quantity | 8 cores |
CPU clock speed | 2.5 GHz and above | |
| Memory | Remaining free physical memory | 32 GB |
| Disk | Remaining free disk space | The server shall have a partition with remaining free space of more than 500 GB. Note : 1. There must be a single partition that meets the condition. 2. The required disk space decreases if you use self-prepared components. If you use a self-provided external configuration database component, the required remaining free disk space on the server can be reduced by 50 GB. |
| Mounting directory | Mounting directory preparation | For details, see Preparing the FineDataLink Mounting Directory. For details, see Preparing the FineOps Server. Note: |
| File system auto-mounting | Ensure that the file system of the mounting directory is configured to be automatically mounted during a boot. Otherwise, containers may fail to access these directories, leading to data loss or application startup issues. | |
| Non-shared path | The mounting directory cannot be a shared-use path. Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container. | |
| Permission and command | Tar command | Ensure the server has the tar command installed. The tar command is a commonly used tool for packaging and compressing files. FineOps requires this command for file extraction. |
| Sed command | Ensure the server has the sed command installed. The sed command is used for text processing. FineOps requires this command for text processing. | |
| Sudo privilege | The server user responsible for deploying the project must have the necessary sudo privileges. 1. Using the root user (superuser) for project deployment and operation is preferred. 2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation. Note: | |
| Port and network | Intranet latency | Less than 1 ms |
| Server intranet bandwidth (between the application and the business database) | At least 1000 Mbp/s (125 MB/s) | |
| Server extranet bandwidth (between the project and the client) | At least 10 Mbps (1.25 MB/s) | |
Component-used port and component port interconnection | For details, see Confirming Server Network of the FineDataLink Project. For details, see Preparing the FineOps Server. | |
Scheme Two
It requires one server with 16-core CPU, 64 GB of memory, and 2 TB of disk space.
It is used to deploy one FineDataLink component, one Kafka component, one Nginx component, and one MySQL component.
It is used to deploy all FineOps components.
| Configuration | Server | |
|---|---|---|
Component to be installed | It is used to install all components of FineDataLink and FineOps. 1. For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components.
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components. Note :
| |
| Basic server requirement | Non-VM (Recommended) | Due to the nature of virtual machines, such as competing for resources, unexpected system failures may occur. Therefore, you are not advised to deploy FanRuan applications on virtual machines. |
| Operating system | Operating system type | Linux |
| Operating system architecture | X86_64 | |
| Operating system kernel | V3.10 and later versions | |
| Operating system software | Recommended: Ubuntu 22 Supported: Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.) CentOS 7.3 to 7.9 Red Hat 7.6 and later releases Rocky Linux 8.8 to 9.4 Note:1. Since CentOS is discontinued, you are advised to use Ubuntu. 2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator). | |
| CPU | CPU core quantity | 16 cores |
CPU clock speed | 2.5 GHz and above | |
| Memory | Remaining free physical memory | 64 GB |
| Disk | Remaining free disk space | The server shall have a partition with remaining free space of more than 2 TB. Note : 1. There must be a single partition that meets the condition. 2. The required disk space decreases if you use self-prepared components. If you use a self-provided Kafka component, the required remaining free disk space on the server can be reduced by 1.5 TB. If you use a self-provided external configuration database component, the required remaining free disk space on the server can be reduced by 50 GB. |
| Mounting directory | Mounting directory preparation | For details, see Preparing the FineDataLink Mounting Directory. For details, see Preparing the FineOps Server. Note: |
| File system auto-mounting | Ensure that the file system of the mounting directory is configured to be automatically mounted during a boot. Otherwise, containers may fail to access these directories, leading to data loss or application startup issues. | |
| Non-shared path | The mounting directory cannot be a shared-use path. Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container. | |
| Permission and command | Tar command | Ensure the server has the tar command installed. The tar command is a commonly used tool for packaging and compressing files. FineOps requires this command for file extraction. |
| Sed command | Ensure the server has the sed command installed. The sed command is used for text processing. FineOps requires this command for text processing. | |
| Sudo privilege | The server user responsible for deploying the project must have the necessary sudo privileges. 1. Using the root user (superuser) for project deployment and operation is preferred. 2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation. Note: | |
| Port and network | Intranet latency | Less than 1 ms |
| Server intranet bandwidth (between the application and the business database) | At least 1000 Mbp/s (125 MB/s) | |
| Server extranet bandwidth (between the project and the client) | At least 10 Mbps (1.25 MB/s) | |
Component-used port and component port interconnection | For details, see Confirming Server Network of the FineDataLink Project. For details, see Preparing the FineOps Server. | |
Scheme Three
1. FineDataLink main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
It is used to deploy one FineDataLink component.
2. Server for FineDataLink components and FineOps: 8-core CPU, 32 GB of memory, and 2 TB of disk space
It is used to deploy one Kafka component, one Nginx component, and one MySQL component.
It is used to deploy all FineOps components.
| Configuration | FineDataLink Main Application Server | Server for FineDataLink Components and FineOps | |
|---|---|---|---|
Component to be installed | It is used to install FineDataLink application components. 1. For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components.
| 1. It is used to install the pipeline and the O&M components of FineDataLink.
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components. Note:
| |
| Basic server requirement | Time consistency | The time on all servers must be synchronized, with a maximum difference of no more than 5 seconds. Inconsistent server time can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies. | |
| Time zone consistency | All servers must be set to the same time zone. Inconsistent server time zones can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies. | ||
| Interconnection with intranet | All servers can communicate with each other on the intranet or have open ports for access. For details, see the following content. | ||
| Non-VM (Recommended) | Due to the nature of virtual machines, such as competing for resources, unexpected system failures may occur. Therefore, you are not advised to deploy FanRuan applications on virtual machines. | ||
| Operating system | Operating system type | Linux | |
| Operating system architecture | X86_64 | ||
| Operating system kernel | V3.10 and later versions | ||
| Operating system software | Recommended: Ubuntu 22 Supported: Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.) CentOS 7.3 to 7.9 Red Hat 7.6 and later releases Rocky Linux 8.8 to 9.4 Note:1. Since CentOS is discontinued, you are advised to use Ubuntu. 2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator). | ||
| CPU | CPU core quantity | 16 cores | 8 cores |
CPU clock speed | 2.5 GHz and above | ||
| Memory | Remaining free physical memory | 64 GB | 32 GB |
| Disk | Remaining free disk space | The server shall have a partition with remaining free space of more than 500 GB. Note : There must be a single partition that meets the condition. | The server shall have a partition with remaining free space of more than 2 TB. Note: 1. There must be a single partition that meets the condition. 2. The required disk space decreases if you use self-prepared components. If you use a self-provided Kafka component, the required remaining free disk space on the server can be reduced by 1.5 TB. If you use a self-provided external configuration database component, the required remaining free disk space on the server can be reduced by 50 GB. |
| Mounting directory | Mounting directory preparation | For details, see Preparing the FineDataLink Mounting Directory. Note: | For details, see Preparing the FineDataLink Mounting Directory. For details, see Preparing the FineOps Server. Note: |
| File system auto-mounting | Ensure that the file system of the mounting directory is configured to be automatically mounted during a boot. Otherwise, containers may fail to access these directories, leading to data loss or application startup issues. | ||
| Non-shared path | The mounting directory cannot be a shared-use path. Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container. | ||
| Permission and command | Tar command | Ensure the server has the tar command installed. The tar command is a commonly used tool for packaging and compressing files. FineOps requires this command for file extraction. | |
| Sed command | Ensure the server has the sed command installed. The sed command is used for text processing. FineOps requires this command for text processing. | ||
| Sudo privilege | The server user responsible for deploying the project must have the necessary sudo privileges. 1. Using the root user (superuser) for project deployment and operation is preferred. 2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation. Note: | ||
| Port and network | Intranet latency | Less than 1 ms | |
| Server intranet bandwidth (between the application and the business database) | At least 1000 Mbp/s (125 MB/s) | ||
| Server extranet bandwidth (between the project and the client) | At least 10 Mbps (1.25 MB/s) | ||
Component-used port and component port interconnection | For details, see Confirming Server Network of the FineDataLink Project. | For details, see Confirming Server Network of the FineDataLink Project. For details, see Preparing the FineOps Server. | |
Scheme Four
1. The number of FineDataLink main application servers is N.
Each main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
Each one should be deployed with one FineDataLink component.
2. Server for FineDataLink components and FineOps: 8-core CPU, 32 GB of memory, and 2 TB of disk space
It is used to deploy one Kafka component, one Nacos component, one Nginx component, and one MySQL component.
It is used to deploy all FineOps components.
| Configuration | FineDataLink Main Application Server | Server for FineDataLink Components and FineOps | |
|---|---|---|---|
Component to be installed | It is used to install FineDataLink application components. For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components.
| 1. It is used to install the pipeline, cluster, and the O&M components of FineDataLink.
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components. Note:
| |
| Basic server requirement | Time consistency | The time on all servers must be synchronized, with a maximum difference of no more than 5 seconds. Inconsistent server time can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies. | |
| Time zone consistency | All servers must be set to the same time zone. Inconsistent server time zones can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies. | ||
| Interconnection with intranet | All servers can communicate with each other on the intranet or have open ports for access. For details, see the following content. | ||
| Non-VM (Recommended) | Due to the nature of virtual machines, such as competing for resources, unexpected system failures may occur. Therefore, you are not advised to deploy FanRuan applications on virtual machines. | ||
| Operating system | Operating system type | Linux | |
| Operating system architecture | X86_64 | ||
| Operating system kernel | V3.10 and later versions | ||
| Operating system software | Recommended: Ubuntu 22 Supported: Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.) CentOS 7.3 to 7.9 Red Hat 7.6 and later releases Rocky Linux 8.8 to 9.4 Note:1. Since CentOS is discontinued, you are advised to use Ubuntu. 2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator). | ||
| CPU | CPU core quantity | 16 cores | 8 cores |
CPU clock speed | 2.5 GHz and above | ||
| Memory | Remaining free physical memory | 64 GB | 32 GB |
| Disk | Remaining free disk space | The server shall have a partition with remaining free space of more than 500 GB. Note : There must be a single partition that meets the condition. | The server shall have a partition with remaining free space of more than 2 TB. Note: 1. There must be a single partition that meets the condition. 2. The required disk space decreases if you use self-prepared components. If you use a self-provided Kafka component, the required remaining free disk space on the server can be reduced by 1.5 TB. If you use a self-provided external configuration database component, the required remaining free disk space on the server can be reduced by 50 GB. If you use a self-provided file storage component, the required remaining free disk space on the server can be reduced by 100 GB. |
| Mounting directory | Mounting directory preparation | For details, see Preparing the FineDataLink Mounting Directory. Note: | For details, see Preparing the FineDataLink Mounting Directory. For details, see Preparing the FineOps Server. Note: |
| File system auto-mounting | Ensure that the file system of the mounting directory is configured to be automatically mounted during a boot. Otherwise, containers may fail to access these directories, leading to data loss or application startup issues. | ||
| Non-shared path | The mounting directory cannot be a shared-use path. Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container. | ||
| Permission and command | Tar command | Ensure the server has the tar command installed. The tar command is a commonly used tool for packaging and compressing files. FineOps requires this command for file extraction. | |
| Sed command | Ensure the server has the sed command installed. The sed command is used for text processing. FineOps requires this command for text processing. | ||
| Sudo privilege | The server user responsible for deploying the project must have the necessary sudo privileges. 1. Using the root user (superuser) for project deployment and operation is preferred. 2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation. Note: | ||
| Port and network | Intranet latency | Less than 1 ms | |
| Server intranet bandwidth (between the application and the business database) | At least 1000 Mbp/s (125 MB/s) | ||
| Server extranet bandwidth (between the project and the client) | At least 10 Mbps (1.25 MB/s) | ||
Component-used port and component port interconnection | For details, see Confirming Server Network of the FineDataLink Project. | For details, see Confirming Server Network of the FineDataLink Project. For details, see Preparing the FineOps Server. | |
Annex
Required Resources of FineDataLink Components
The configuration requirements in the above sections must be met because each component of FineDataLink projects has minimum operational requirements, as shown in the following table.
Note:
1. To prevent resource over-provisioning, a CPU-sharing strategy is used, as the components do not run at full load simultaneously. Therefore, the required number of server CPU cores is not the total number of maximum CPU cores used by each container.
Note 2. The server memory size must be at least the total of the maximum memory used by each container, to implement the JAVA memory configuration for each container, including Xmx, off-heap memory, FineIO read/write memory, CodeCache, and Metaspace.
| Component | Maximum CPU Usage | Maximum Memory Usage | Memory Allocation (Xmx) | Disk Space |
|---|---|---|---|---|
FineDataLink | 4 cores | / | Default: Server memory - 8 GB 30 GB at most | 50 GB |
| OPS Agent | 1 core | 1 GB | / | / |
| Nginx | 1 core | 2 GB | / | / |
| MySQL | 2 cores | 6 GB | / | 50 GB |
| Redis | 1 core | 4.5 GB | 4 GB | / |
| MinIO | 1 core | 2 GB | / | 100 GB |
| Kafka | 4 cores | 4 GB | 2 GB | / |
| Nacos | 2 cores | 2 GB | / | / |
| Nacos MySQL | 2 cores | 2 GB | / | / |
Previous：Preparing the FineReport Mounting Directory