You can deploy FineDataLink 4.1 projects via FineOps.
This document provides multiple standard schemes based on the number of scheduled tasks, real-time tasks, and pipeline tasks. Choose the one that best meets your needs.
The schemes in this document are designed for new project deployment scenarios, providing recommended server configuration based on business usage. You can select the scheme according to the task quantity.
The schemes in this document are intended for scenarios where the volume of data (calculated by Row count * Column count) changed in an hour in a single business database does not exceed one million. If the data change volume surpasses this limit, contact FanRuan technical support personnel for further assistance.
These schemes offer resource recommendations based on typical scales of customers’ projects. If the number of tasks exceeds the applicable range, contact FanRuan technical support personnel for further assistance.
This document provides recommended configuration for newly deploying all FineDataLink components. If you have prepared some components such as the configuration database and the file service, you can lower the corresponding server configuration. For details, see notes of each scheme.
As usage increases over time, if your current server resources become insufficient and require expansion, contact FanRuan technical support personnel for assistance.
The number of scheduled tasks is less than 35.
Pipeline tasks are not needed.
It is designed for customers with minimal business usage only, where FineOps Basic Edition instead of FineOps Standard Edition should be used. Monitoring indicators in Tracing and Health Inspection are unavailable.
Server: 8-core CPU, 16 GB of memory, and 300 GB of disk space
It is used to deploy one FineDataLink component, one Nginx component (an internal gateway), and one MySQL component (a configuration database).
It is used to deploy all FineOps components excluding Elasticsearch and SkyWalking OAP.
The number of scheduled tasks is less than 150.
Server: 8-core CPU, 32 GB of memory, and 500 GB of disk space
It is used to deploy one FineDataLink component, one Nginx component, and one MySQL component.
It is used to deploy all FineOps components.
The number of scheduled tasks is less than 300.
The number of pipeline tasks is less than 16.
The number of real-time tasks is less than 20.
Server: 16-core CPU, 64 GB of memory, and 2 TB of disk space
It is used to deploy one FineDataLink component, one Kafka component, one Nginx component, and one MySQL component.
The number of scheduled tasks is less than 500.
The number of real-time tasks is less than 40.
1. FineDataLink main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
It is used to deploy one FineDataLink component.
2. Server for FineDataLink components and FineOps: 8-core CPU, 32 GB of memory, and 2 TB of disk space
It is used to deploy one Kafka component, one Nginx component, and one MySQL component.
The number of scheduled tasks is less than 500 * N.
The number of pipeline tasks is less than 16 * N.
The number of real-time tasks is less than 40 * N.
1. The number of FineDataLink main application servers is N.
1. Each main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
Each one should be deployed with one FineDataLink component.
It is used to deploy one Kafka component, one Nacos component, one Nginx component, and one MySQL component.
Note:
This scheme is intended for customers with little business usage and limited server resources only.
This scheme is only applicable to deploy FineOps Basic Edition instead of FineOps Standard Edition. Monitoring indicators in Tracing and Health Inspection are unavailable.
It requires one server with 8-core CPU, 16 GB of memory, and 300 GB of disk space.
Component to be installed
It is used to install all components of FineDataLink and FineOps.
1. For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components.
FineDataLink
OPS Agent and Nginx
MySQL (The self-provided component is supported.)
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components.
OPS and Nginx
Pushgateway, Prometheus, Grafana, and Alertmanager
Registry (The self-provided component is supported.)
Supported:
Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.)
CentOS 7.3 to 7.9
Red Hat 7.6 and later releases
Rocky Linux 8.8 to 9.4
1. Since CentOS is discontinued, you are advised to use Ubuntu.
2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator).
CPU clock speed
Remaining free physical memory
The server shall have a partition with remaining free space of more than 300 GB.
Note
:
For details, see Preparing the FineDataLink Mounting Directory.
For details, see Preparing the FineOps Server.
Otherwise, containers may fail to access these directories, leading to data loss or application startup issues.
Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container.
Ensure the server has the tar command installed.
The tar command is a commonly used tool for packaging and compressing files.
FineOps requires this command for file extraction.
Ensure the server has the sed command installed.
The sed command is used for text processing.
FineOps requires this command for text processing.
1. Using the root user (superuser) for project deployment and operation is preferred.
2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation.
At least 100 Mbp/s (12.5 MB/s)
At least 100 Mbps (12.5 MB/s)
Component-used port and component port interconnection
For details, see Confirming Server Network of the FineDataLink Project.
It requires one server with 8-core CPU, 32 GB of memory, and 500 GB of disk space.
OPS, Nginx, and OPS Agent
Elasticsearch and SkyWalking OAP
The server shall have a partition with remaining free space of more than 500 GB.
1. There must be a single partition that meets the condition.
2. The required disk space decreases if you use self-prepared components.
If you use a self-provided external configuration database component, the required remaining free disk space on the server can be reduced by 50 GB.
At least 1000 Mbp/s (125 MB/s)
At least 10 Mbps (1.25 MB/s)
It requires one server with 16-core CPU, 64 GB of memory, and 2 TB of disk space.
Kafka (The self-provided component is supported.)
The server shall have a partition with remaining free space of more than 2 TB.
If you use a self-provided Kafka component, the required remaining free disk space on the server can be reduced by 1.5 TB.
It is used to install FineDataLink application components.
OPS Agent
1. It is used to install the pipeline and the O&M components of FineDataLink.
The time on all servers must be synchronized, with a maximum difference of no more than 5 seconds.
Inconsistent server time can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies.
Inconsistent server time zones can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies.
There must be a single partition that meets the condition.
Each main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
For details about the role of FineDataLink components, see Understanding the Roles of O&M Project Components.
1. It is used to install the pipeline, cluster, and the O&M components of FineDataLink.
Nacos and Nacos MySQL
MinIO (The self-provided component is supported.)
Redis (The self-provided component is supported.)
OPS Nginx, and OPS Agent
If you use a self-provided file storage component, the required remaining free disk space on the server can be reduced by 100 GB.
The configuration requirements in the above sections must be met because each component of FineDataLink projects has minimum operational requirements, as shown in the following table.
1. To prevent resource over-provisioning, a CPU-sharing strategy is used, as the components do not run at full load simultaneously. Therefore, the required number of server CPU cores is not the total number of maximum CPU cores used by each container.
Note 2. The server memory size must be at least the total of the maximum memory used by each container, to implement the JAVA memory configuration for each container, including Xmx, off-heap memory, FineIO read/write memory, CodeCache, and Metaspace.
/
Default: Server memory - 8 GB
30 GB at most