You can use FineOps to deploy FineBI 6.1 projects.
This document offers multiple standard deployment schemes for scenarios with different business availability demands, daily active user counts, data volume for analysis, and table quantity. Choose the one that best meets your needs.
Term Definition
High business availability: There are at least two bi-web components, so if one business node crashes, other nodes can complete the tasks instead.
Daily active users (DAU): It refers to the average number of unique users who log into the FineBI system each day.
Maximum rows in an analysis table: It refers to the maximum number of rows in a single self-service dataset used by a single component in a FineBI dashboard.
Number of tables: It refers to the total count of all tables in the FineBI system, including database tables, SQL datasets, Excel datasets, and self-service datasets.
Scheme Applicability
The schemes in this document cater for the server resources of most customers, so in these schemes, some FineBI components and FineOps are deployed on one server. If more servers are available, FineBI and FineOps can be deployed independently to meet disaster recovery and other needs. Contact FanRuan technical support for further assistance.
The schemes in this document are suitable for scenarios primarily involving extracted data. Contact technical support for further assistance if your FineBI project relies entirely on direct-connected data.
These schemes offer resource recommendations based on typical scales of customers’ projects. Contact FanRuan technical support for further assistance if your DAU number, data volume, or the number of tables exceeds the recommended range.
These schemes are suitable for deploying new projects. Contact FanRuan technical support for assistance in capacity expansion if your existing server resources become insufficient as those used by the project grow.
Notes on Project Configuration
The recommended configuration is suitable for completely independently deployed FineOps and FineBI projects. If the FineBI component and FineOps are to be deployed on the same server, the server's configuration should be the combined total of required configurations of the FineBI component server and the FineOps server.
The recommended configuration is for a fresh deployment of all FineBI components. If you have prepared some components, such as the data storage service and the file service, you can lower the corresponding server configurations. For details, see notes of each scheme.
For FineOps-deployed FineBI projects, a master component must be deployed. The bi-web and worker components can only be deployed in a ratio of 1:1 or 1:2. The node number ratio cannot be customized.
To achieve read-write separation, you can only choose between Scheme Three or Scheme Four as it requires highly available services.
Quick Selection Guide
Brief Scheme Introduction
App node: a single business node (not highly available)
DAU: 0 to 50
Maximum rows in an analysis table: 0 to 5 million
Number of tables: 0 to 500
1. One main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
It is used to deploy one bi-web component, one worker component, one master component, and one bi-minio component.
It is used to deploy the following components: nginx, mysql, elasticsearch, and the cluster component.
It is used to deploy all FineOps components.
DAU: 0 to 300
Maximum rows in an analysis table: 0 to 10 million
Number of tables: 0 to 2000
1. FineBI main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space
2. Server for FineBI components and FineOps: 8-core CPU, 32 GB of memory, and 500 GB of disk space
DAU: 0 to 600
Maximum rows in an analysis table: 0 to 100 million
Number of tables: 0 to 10 thousand
1. FineBI main application server: 32-core CPU, 128 GB of memory, and 2 TB of disk space
2. Server for FineBI components and FineOps: 16-core CPU, 64 GB of memory, and 500 GB of disk space
App node: multiple application nodes (highly available)
1. FineBI main application server one: 16-core CPU, 64 GB of memory, and 500 GB of disk space
2. FineBI main application server two: 16-core CPU, 64 GB of memory, and 200 GB of disk space
It is used to deploy one bi-web component and one worker component.
3. Server for FineBI components and FineOps: 8-core CPU, 32 GB of memory, and 500 GB of disk space
DAU: 0 to 1200
1. FineBI main application server one: 32-core CPU, 128 GB of memory, and 2 TB of disk space
2. FineBI main application server two: 32-core CPU, 128 GB of memory, and 500 GB of disk space
3. Server for FineBI components and FineOps: 16-core CPU, 64 GB of memory, and 500 GB of disk space
This scheme is intended for customers with little business usage and limited server resources only. All components of FineBI and FineOps are deployed on one server.
Ensure that the number of daily active users ranges from 0 to 50, the maximum number of rows in an analysis table ranges from 0 to 5 million, and the number of tables ranges from 0 to 500.
It includes one main application server with 16-core CPU, 64 GB of memory, and 500 GB of disk space.
Component to be installed
It is used to install all components of FineBI and FineOps.
1. For details about the role of FineBI components, see Understanding the Roles of O&M Project Components.
1. You are advised to prepare and use the S3 service for data storage.
2. If you use S3 to replace the data storage component bi-minio and the file storage component minio, prepare at least two buckets to store the content of each component separately.
bi-web, bi-engine-worker, and bi-engine-master
bi-minio (Self-provided S3 is recommended.)
filebeat, ops_agent, and nginx
elasticsearch (The self-provided service is recommended.)
minio (The self-provided service is recommended.)
mysql (The self-provided service is recommended.)
redis (The self-provided service is recommended.)
2. For details about the role of FineOps components, see Understanding the Roles of FineOps Components.
ops, nginx, and ops_agent
pushgateway, prometheus, grafana, and alertmanager
elasticsearch and skywalking_oap
registry (The self-provided service is recommended.)
Supported:
Ubuntu 18.04.4 and later releases (Ubuntu 20.04 is not supported.)
CentOS 7.3 to 7.9
Red Hat 7.6 and later releases
Rocky Linux 8.8 to 9.4
1. Since CentOS is discontinued, you are advised to use Ubuntu.
2. When using Ubuntu, verify the user privilege (as the default root user is not a super administrator) and confirm that the disk type is XFS. For details, see notes in the following content.
CPU clock speed
Remaining free physical memory
The server shall have a partition with remaining free space of more than 500 GB.
There must be a single partition that meets the condition.
Disk type
If you have not prepared the data storage component, the bi-minio component must be deployed, and the disk type must be XFS.
A non-XFS disk might result in data write failure even with available disk space.
1. The default disk type of Ubuntu systems is typically not XFS. Adjust it accordingly.
2. The default disk type of Alibaba Cloud servers is typically not XFS. Adjust it accordingly.
3. This operation may format the disk. FanRuan does not provide related guidance. Follow the instructions from your server vendor and proceed with caution.
The disk read/write speed must be greater than 100 MB/s, meaning the disk can handle at least 100 MB of data per second.
The random IOPS (4K) must be greater than 500, meaning the disk can perform at least 500 read/write operations on 4 KB data blocks per second.
Mounting directory preparation
For details, see FineBI Mounting Directory Preparation.
For details, see FineOps Mounting Directory Preparation.
Otherwise, containers may fail to access these directories, leading to data loss or application startup issues.
Sharing the file system may cause performance degradation, file permission issues, and data inconsistency, affecting the reliability and response speed of applications running in the container.
Ensure the server has the tar command installed.
The tar command is a commonly used tool for packaging and compressing files.
FineOps requires this command for file extraction.
Ensure the server has the sed command installed.
The sed command is used for text processing.
FineOps requires this command for text processing.
1. Using the root user (superuser) for project deployment and operation is preferred.
2. To use a non-root user for deployment and operation, see Linux User Privilege Explanation.
At least 1000 Mbps (125 MB/s)
Component-used port and component port interconnection
For details, see FineBI Server Network Confirmation.
For details, see FineOps Server Preparation.
It includes two servers, namely, a main application server and a component server.
Main application server: 16-core CPU, 64 GB of memory, and 500 GB of disk space, used to deploy one bi-web component, one worker component, one master component, and one bi-minio component
Component server: 8-core CPU, 32 GB of memory, and 500 GB of disk space, used to deploy nginx, mysql, elasticsearch, the cluster component, and all FineOps components
It is used to install the BI business and engine components.
For details about the role of components, see Understanding the Roles of O&M Project Components.
2If you use S3 to replace the data storage component bi-minio and the file storage component minio, prepare at least two buckets to store the content of each component separately.
bi-web
bi-engine-worker
bi-engine-master
filebeat
ops_agent
1. It is used to install the BI cluster, log, and configuration components.
nginx
2. It is used to install FineOps.
ops and nginx
The time on all servers must be synchronized, with a maximum difference of no more than 5 seconds.
Inconsistent server time can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies.
Time zone consistency
Inconsistent server time zones can cause issues such as incorrect execution of scheduled tasks, disorganized log records, and data inconsistencies.
1. There must be a single partition that meets the condition.
2. The required disk space decreases if you use self-prepared components. For example, if you use a self-provided data storage component, the required remaining free disk space on the server can be reduced by 300 GB (meaning that 200 GB of free disk space on the server is enough).
2. The required disk space decreases if you use self-prepared components.
If you use the self-provided Elasticsearch, the required remaining free disk space on the server can be reduced by 50 GB.
If you use the self-provided external database, the required remaining free disk space on the server can be reduced by 50 GB.
If you use the self-provided file storage component, the required remaining free disk space on the server can be reduced by 100 GB.
If you have not prepared the file service component, the minio component must be deployed, and the disk type must be XFS.
The disk should at least be a SATA solid-state drive (SSD).
The random IOPS (4K) must be greater than 10,000, meaning the disk can perform at least 10,000 read/write operations on 4 KB data blocks per second.
The sequential IOPS (1024K) must be greater than 315.5 MB, meaning the disk can transfer at least 315.5 MB of blocks per second.
Since the configuration database of this project is deployed on the same machine, the disk performance requirements of this server are consistent with the main application server.
Main application server: 32-core CPU, 128 GB of memory, and 2 TB of disk space, used to deploy one bi-web component, one worker component, one master component, and one bi-minio component
Component server: 16-core CPU, 64 GB of memory, and 500 GB of disk space, used to deploy nginx, mysql, elasticsearch, the cluster component, and all FineOps components
For details about the role of components, see Understanding the Roles of FineOps Components.
The server shall have a partition with remaining free space of more than 2 TB.
2. The required disk space decreases if you use self-prepared components. For example, if you use a self-provided data storage component, the required remaining free disk space on the server can be reduced by 1.5 TB (meaning that 500 GB of free disk space on the server is enough).
The random IOPS (4K) must be greater than 20,000, meaning the disk can perform at least 20,000 read/write operations on 4 KB data blocks per second.
The sequential IOPS (1024K) must be greater than 625 MB, meaning the disk can transfer at least 625 MB of blocks per second.
It includes three servers, namely, two main application servers, a component server, and a FineOps server.
Main application server one: 16-core CPU, 64 GB of memory, and 500 GB of disk space, used to deploy one bi-web component, one worker component, one master component, and one bi-minio component
Main application server two: 16-core CPU, 64 GB of memory, and 200 GB of disk space, used to deploy one bi-web component and one worker component
The server shall have a partition with remaining free space of more than 200 GB.
It includes three servers, namely, two main application servers and a component server.
Main application server one: 32-core CPU, 128 GB of memory, and 2 TB of disk space, used to deploy one bi-web component, one worker component, one master component, and one bi-minio component
Main application server two: 32-core CPU, 128 GB of memory, and 500 GB of disk space, used to deploy one bi-web component and one worker component
You can download the FineBI 6.1 extraction performance report. FineBI 6.1 Extraction Performance Report.pdf
The configuration requirements in the above tables must be met because each component of FineBI projects has minimum operational requirements, as shown in the following table.
1. To prevent resource over-provisioning, a CPU-sharing strategy is used, as the components do not run at full load simultaneously. Therefore, the required number of server CPU cores is not the total number of maximum CPU cores used by each container.
2. The server memory size must be at least the total of the maximum memory used by each container to implement the JAVA memory configuration for each container, including Xmx, off-heap memory, FineIO read/write memory, CodeCache, and Metaspace.
Server memory/4
64 GB at most
5/8 Container memory - 2
32766 MB at most
Server memory/2/Number of the worker component
5/8 Container memory -10/7
Server memory/8