You can use FineOps to deploy FineBI projects.
This document offers multiple standard deployment schemes for scenarios with different business availability demands, daily active user counts, data volume for analysis, and table quantity. Choose the one that best meets your needs.
Term Definition
High business availability: There are at least two FineBI - Application Node components, so if one business node crashes, other nodes can complete the tasks instead.
Daily active users (DAU): It refers to the average number of unique users who log into the FineBI system each day.
Maximum rows in an analysis table: It refers to the maximum number of rows in a single self-service dataset used by a single component in a FineBI dashboard.
Number of tables: It refers to the total count of all tables in the FineBI system, including database tables, SQL datasets, Excel datasets, and self-service datasets.
Scheme Applicability
The schemes in this document cater for the server resources of most customers, so in these schemes, some FineBI components and FineOps are deployed on one server. If more servers are available, FineBI and FineOps can be deployed independently to meet disaster recovery and other needs. Contact FanRuan technical support for further assistance.
The schemes in this document are suitable for scenarios primarily involving extracted data. Contact technical support for further assistance if your FineBI project relies entirely on direct-connected data.
These schemes offer resource recommendations based on typical scales of customers’ projects. Contact FanRuan technical support for further assistance if your DAU number, data volume, or the number of tables exceeds the recommended range.
These schemes are suitable for deploying new projects. Contact FanRuan technical support for assistance in capacity expansion if your existing server resources become insufficient as those used by the project grow.
Notes on Project Configuration
The recommended configuration is suitable for completely independently deployed FineOps and FineBI projects. If the FineBI component and FineOps are to be deployed on the same server, the server's configuration should be the combined total of required configurations of the FineBI component server and the FineOps server.
The recommended configuration is for a fresh deployment of all FineBI components. If you have prepared some components, such as the data storage service and the file service, you can lower the corresponding server configurations. For details, see notes of each scheme.
For FineOps-deployed FineBI projects, an Engine - Metadata Node component must be deployed. The FineBI - Application Node and Engine - Calculation Node components can only be deployed in a ratio of 1:1 or 1:2. The node number ratio cannot be customized.
To achieve read-write separation, you can only choose between Scheme Three or Scheme Four as it requires highly available services.
Quick Selection Guide
Brief Scheme Introduction
App node: a single business node (not highly available)
DAU: 0 to 50
Maximum rows in an analysis table: 0 to 5 million
Number of tables: 0 to 500
1. One main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
It is used to deploy one FineBI - Application Node component, one Engine - Calculation Node component, one Engine - Metadata Node component, and one FineBI MinIO component.
It is used to deploy the following components: Nginx, MySQL, Elasticsearch, and the cluster component.
It is used to deploy all FineOps components.
DAU: 0 to 300
Maximum rows in an analysis table: 0 to 10 million
Number of tables: 0 to 2000
1. FineBI main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
2. Server for FineBI components and FineOps: 8-core CPU, 32 GB of memory, and 500 GB of disk space
DAU: 0 to 600
Maximum rows in an analysis table: 0 to 100 million
Number of tables: 0 to 10 thousand
1. FineBI main application server: 32-core CPU, 128 GB of memory, and 2 TB of disk space
2. Server for FineBI components and FineOps: 16-core CPU, 64 GB of memory, and 500 GB of disk space
App node: multiple application nodes (highly available)
1. FineBI main application server one: 16-core CPU, 64 GB of memory, and 500 GB of disk space
2. FineBI main application server two: 16-core CPU, 64 GB of memory, and 200 GB of disk space
It is used to deploy one FineBI - Application Node component and one Engine - Calculation Node component.
3. Server for FineBI components and FineOps: 8-core CPU, 32 GB of memory, and 500 GB of disk space
DAU: 0 to 1200
1. FineBI main application server one: 32-core CPU, 128 GB of memory, and 2 TB of disk space
2. FineBI main application server two: 32-core CPU, 128 GB of memory, and 500 GB of disk space
3. Server for FineBI components and FineOps: 16-core CPU, 64 GB of memory, and 500 GB of disk space
This scheme is intended for customers with little business usage and limited server resources only. All components of FineBI and FineOps are deployed on one server.
Ensure that the number of daily active users ranges from 0 to 50, the maximum number of rows in an analysis table ranges from 0 to 5 million, and the number of tables ranges from 0 to 500.
It includes one main application server with 16-core CPU, 64 GB of memory, and 500 GB of disk space.
Component to be installed
It is used to install all components of FineBI and FineOps.
1. For details about the role of FineBI components, see Understanding the Roles of O&M Project Components.
1. You are advised to prepare and use the S3 service for data storage.
2. If you use S3 to replace the data storage component FineBI MinIO and the file storage component MinIO, prepare at least two buckets to store the content of each component separately.
FineBI - Application Node, Engine - Calculation Node, and Engine - Metadata Node
FineBI MinIO (Self-provided S3 is recommended.)
Filebeat, OPS Agent, and Nginx
Elasticsearch (The self-provided service is recommended.)
MinIO (The self-provided service is recommended.)
MySQL (The self-provided service is recommended.)
Redis (The self-provided service is recommended.)
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components.
OPS, Nginx, and OPS Agent
Pushgateway, Prometheus, Grafana, and Alertmanager
Elasticsearch and SkyWalking OAP
Registry (The self-provided service is recommended.)
Supported:
Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.)
CentOS 7.3 to 7.9
Red Hat 7.6 and later releases
Rocky Linux 8.8 to 9.4
1. Since CentOS is discontinued, you are advised to use Ubuntu.
2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator) and confirm that the disk type is XFS. For details, see notes in the following content.
CPU clock speed
Remaining free physical memory
64 GB
Registration and authentication must be completed after project deployment.
For the intranet environment that can be connected to the FanRuan cloud or the extranet environment, you are advised to authenticate registration via a public cloud. For details, see Public Cloud Authentication.
For the pure intranet environment, you need to install a registration service component on the project component server and authenticate registration via private cloud on FineOps. Therefore, you must ensure at least 2 GB of additional available physical memory on this server. For details, see New Project Registration.
The server shall have a partition with remaining free space of more than 500 GB.
Disk type
If you have not prepared a data storage component, the FineBI MinIO component must be deployed, and the recommended disk type is XFS.
MinIO high-availability mode relies on the O_DIRECT feature, which is natively supported by the XFS file system. If you use an XFS-type disk, no additional operation is required when you scale MinIO for high availability in the future.
1. The default disk type of Ubuntu systems is typically not XFS. Adjust it accordingly.
2. The default disk type of Alibaba Cloud servers is typically not XFS. Adjust it accordingly.
3. This operation may format the disk. FanRuan does not provide related guidance. Follow the instructions from your server vendor and proceed with caution.
The disk read/write speed must be greater than 100 MB/s, meaning the disk can handle at least 100 MB of data per second.
The random IOPS (4K) must be greater than 500, meaning the disk can perform at least 500 read/write operations on 4 KB data blocks per second.
Mounting directory preparation
For details, see FineBI Mounting Directory Preparation.
For details, see FineOps Mounting Directory Preparation.
Otherwise, containers may fail to access these directories, leading to data loss or application startup issues.
Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container.
Ensure the server has the tar command installed.
The tar command is a commonly used tool for packaging and compressing files.
FineOps requires this command for file extraction.
Ensure the server has the sed command installed.
The sed command is used for text processing.
FineOps requires this command for text processing.
Ensure the user you use can connect to the FineOps server via the SSH protocol.
Ensure the password used for SSH connection contains no English single quotation marks ('), or the privilege will fail to be validated during deployment.
1. Using the root user (superuser) for project deployment and operation is preferred.
2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation.
At least 1000 Mbps (125 MB/s)
Component-used port and component port interconnection
For details, see FineBI Server Network Confirmation.
For details, see FineOps Server Preparation.
If the server is running the Kylin operating system, ensure that Podman is not installed on the server.
The version of runc bundled with Podman is too low and conflicts with the Docker installation managed by FanRuan, which may cause deployment failures.
Refer to the relevant document to uninstall Podman. You are advised to verify the impact of uninstallation in a non-production environment first.
Ensure that Kingsoft Antivirus is not installed on the server.
This antivirus software may trigger server connection interruptions when the dockerd command is executed, resulting in deployment failures.
It includes two servers, namely, a main application server and a component server.
Main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space, used to deploy one FineBI - Application Node component, one Engine - Calculation Node component, one Engine - Metadata Node component, and one FineBI MinIO component
Component server: 8-core CPU, 32 GB of memory, and 500 GB of disk space, used to deploy Nginx, MySQL, Elasticsearch, the cluster component, and all FineOps components
It is used to install the BI business and engine components.
For details about the role of components, see Understanding the Roles of O&M Project Components.
2If you use S3 to replace the data storage component FineBI MinIO and the file storage component MinIO, prepare at least two buckets to store the content of each component separately.
FineBI - Application Node
Engine - Calculation Node
Engine - Metadata Node
Filebeat
OPS Agent
1. It is used to install the BI cluster, log, and configuration components.
Nginx
2. It is used to install FineOps.
OPS and Nginx
The time on all servers must be synchronized, with a maximum difference of no more than 5 seconds.
Inconsistent server time can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies.
Time zone consistency
Inconsistent server time zones can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies.
32 GB
1. There must be a single partition that meets the condition.
2. The required disk space decreases if you use self-prepared components. For example, if you use a self-provided data storage component, the required remaining free disk space on the server can be reduced by 300 GB (meaning that 200 GB of free disk space on the server is enough).
2. The required disk space decreases if you use self-prepared components.
If you use the self-provided Elasticsearch, the required remaining free disk space on the server can be reduced by 50 GB.
If you use the self-provided external database, the required remaining free disk space on the server can be reduced by 50 GB.
If you use the self-provided file storage component, the required remaining free disk space on the server can be reduced by 100 GB.
If you have not prepared a file server, the MinIO component must be deployed, and the recommended disk type is XFS.
The disk should at least be a SATA solid-state drive (SSD).
The random IOPS (4K) must be greater than 10,000, meaning the disk can perform at least 10,000 read/write operations on 4 KB data blocks per second.
The sequential IOPS (1024K) must be greater than 315.5 MB, meaning the disk can transfer at least 315.5 MB of blocks per second.
Since the configuration database of this project is deployed on the same machine, the disk performance requirements of this server are consistent with the main application server.
Main application server: 32-core CPU, 128 GB of memory, and 2 TB of disk space, used to deploy one FineBI - Application Node component, one Engine - Calculation Node component, one Engine - Metadata Node component, and one FineBI MinIO component
Component server: 16-core CPU, 64 GB of memory, and 500 GB of disk space, used to deploy Nginx, MySQL, Elasticsearch, the cluster component, and all FineOps components
For details about the role of components, see Understanding the Roles of FineOps Components.
The server shall have a partition with remaining free space of more than 2 TB.
2. The required disk space decreases if you use self-prepared components. For example, if you use a self-provided data storage component, the required remaining free disk space on the server can be reduced by 1.5 TB (meaning that 500 GB of free disk space on the server is enough).
The random IOPS (4K) must be greater than 20,000, meaning the disk can perform at least 20,000 read/write operations on 4 KB data blocks per second.
The sequential IOPS (1024K) must be greater than 625 MB, meaning the disk can transfer at least 625 MB of blocks per second.
Absence of Kingsoft Antivirus
It includes three servers, namely, two main application servers, a component server, and a FineOps server.
Main application server one: 16-core CPU, 64 GB of memory, and 500 GB of disk space, used to deploy one FineBI - Application Node component, one Engine - Calculation Node component, one Engine - Metadata Node component, and one FineBI MinIO component
Main application server two: 16-core CPU, 64 GB of memory, and 200 GB of disk space, used to deploy one FineBI - Application Node component and one Engine - Calculation Node component
The server shall have a partition with remaining free space of more than 200 GB.
It includes three servers, namely, two main application servers and a component server.
Main application server one: 32-core CPU, 128 GB of memory, and 2 TB of disk space, used to deploy one FineBI - Application Node component, one Engine - Calculation Node component, one Engine - Metadata Node component, and one FineBI MinIO component
Main application server two: 32-core CPU, 128 GB of memory, and 500 GB of disk space, used to deploy one FineBI - Application Node component and one Engine - Calculation Node component
You can download the FineBI 6.1 extraction performance report. FineBI 6.1 Extraction Performance Report.pdf
The configuration requirements in the above tables must be met because each component of FineBI projects has minimum operational requirements, as shown in the following table.
1. The component configuration requirements in the following table represent the minimum specifications for running the system. After successful deployment, you are advised to use the System Inspection function to confirm the recommended values if adjustments are needed.
2. To prevent resource over-provisioning, a CPU-sharing strategy is used, as the components do not run at full load simultaneously. Therefore, the required number of server CPU cores is not the sum of the maximum CPU cores used by each container.
3. The server memory size must be at least the sum of the maximum memory used by each container, to implement the JAVA memory configuration for each container, including Xmx, off-heap memory, FineIO read/write memory, CodeCache, and Metaspace.
Server memory/4
64 GB at most
5/8 Container memory - 2
32766 MB at most
Server memory/2/Engine - Calculation Node component count
5/8 Container memory -10/7
Server memory/8
It should scale with the container memory usage limit.
If the container memory usage limit is less than 8 GB, set the Xmx value to 2420 MB.
If the container memory usage limit is less than 16 GB, set the Xmx value to 4100 MB.
If the container memory usage limit is less than 32 GB, set the Xmx value to 9400 MB.
If the container memory usage limit is at least 32 GB, set the Xmx value to 18800 MB.
滑鼠選中內容,快速回饋問題
滑鼠選中存在疑惑的內容,即可快速回饋問題,我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy