This article describes how to manually configure a standard extraction cluster on Linux.
If you plan to deploy an extraction cluster on FineBI 6.0, you can contact FanRuan technical support or FanRuan's sales personnel first to conduct preliminary evaluation and of environment confirmation.
You can send an email to international@fanruan.com or click at https://intl.finebi.com/.
Deploying the extraction cluster on your own cannot guarantee that you can deal with unknown risks in a timely manner.
Before manually configuring the standard cluster, you need to perform the following procedures in advance.
Procedure
Description
1
Prepare server environment
You need to configure project nodes, load balancing, state servers, file servers, and database deployment environment.
Ensure that the internal network is connected between project nodes and servers where the components are located.
Ensure that each project node has consistent system time with each component server.
2
Deploy a database
Deploy a database for backup as an external database for projects. MySQL 5 database is recommended.
You only need to deploy and start the database.
3
Deploy load balancing
Nginx-based load balancing deployment solution is recommended.
You just need to install Nginx, requiring no startup or modification of the nginx.conf configuration file.
4
Deploy state servers
The Redis standalone server is recommended as a state server.
You need to ensure that Redis standalone is started.
5
Deploy file servers
You are advised to deploy SFTP (SSH File Transfer Protocol) for file servers.
You need to ensure SFTP is started.
6
Deploy project nodes
Deploy and start only one project node first.
For details about how to deploy, see Deploying the FineBI Project Locally in Tomcat Container on Linux.
Three system parameters need to be modified for Linux to ensure that extraction clusters can run properly later.
Parameter
Modification Method
vm.max_map_count
vm.max_map_count is a Linux kernel parameter used to control the maximum number of memory-mapped areas allowed in a process.
If the value of this parameter is too small, memory mapping errors may occur when you use some applications.
You are advised to set vm.max_map_count to 262144.
Set or change the value of vm.max_map_count to 262144 in the etc/sysctl.conf file.
Run the sysctl -p command for the change to take effect.
Maximum number of open files
open_files specifies the maximum number of file descriptors that can be opened by a process at the same time.
A file descriptor is an unique identifier used to access files (including pipes, sockets, and other file operations) in Unix systems.
It is recommended that the open_files parameter of FineBI be set to not less than 131070.
Modify parameter setting.
Open the software and hardware restriction file /etc/security/limits.conf on Linux and add the following codes at the end of the file:
* soft nofile 65536
* hard nofile 65536
In the above command, hard represents the set hard limit on the system and soft represents the set soft limit on users.
Restart the Tomcat project and log in again.
After the configuration is completed, log in again as the terminal user and restart the Tomcat project for the configuration to take effect.
You can use the ulimit -n command to check whether the modification is successful based on the return value.
glibc
This paraemter is used to solve the problem of potential downtime.
1. Verify the glibc version.
Run the ldd --version command.
2. If the glibc version is not 2.12, you need to add the following content to the first line of the startup.sh file:
export MALLOC_ARENA_MAX=1
3. Restart the Tomcat server.
For FineBI6.0 extraction clusters, the state server Redis is a critical component for which logs needs to be configured for future troubleshooting.
In the redis.conf file, the log is configured as logfile by default.
Add the log as or modify it to logfile "redis.log".
After configuration, the log will be automatically generated in the src directory.
In the %BI_HOME%\webapps\webroot\WEB-INF\config path, create the spider.cluster.properties file.
The file contains three lines of content.
Content
spider_grpc_port={variable}
Set an RPC port for the Spider engine. If it is not set, the default port 50051 is used.
Ensure that the port to be set is available.
bi_conf_rpc_port={variable}
Set the RMI of the new configuration framework. If it is not set, the default port 50200 is used.
is_sync_node=true
Set the properties of project nodes. If it is not set, true is used by default.
false: Asynchronous node
true: Synchronous node
Start the node where the FineBI project is located.
1. Verify external database configuration.
Log in to FineBI as admin, choose System Management > System Setting > General > External Database, and verify that the system has been configured with an external database.
2. Enter the cluster configuration interface.
Log in to FineBI as admin, and choose System Management > Intelligent O&M > Cluster Configuration.
Verify that all configuration operations in the following section are completed on this page.
Preparations:
You are advised to use the Redis standalone state server of the standard cluster.
Ensure that Redis standalone has been started. Otherwise, FineBI cannot connect to it successfully.
Prepare a password for Redis standalone. (If not required, you do not need to prepare.)
Prepare the host IP address and port number of the Redis standalone node. (You are advised not to use a domain name. You need to prepare an internal network IP address.)
Procedure:
1. Enable Status Server.
2. Set Cache System to Redis Single Machine.
3. Enter the host IP address and port number of the Redis standalone node.
4. (Optional) Enter the password (if any).
5. After configuration, click Test connection and save. If no exceptions occur, settings are saved successfully.
MQ built-in functions are designed for containerized deployment clusters. You can directly skip this configuration for non-containerized manual deployment clusters.
Projects installed with containerized deployment generally have RocketMQ components configured by default. RocketMQ can replace some functions provided by Redis message queues.
If RocketMQ is not configured, Redis will be used by default.
If RocketMQ is configured, RocketMQ and Redis function together for different functions.
Prerequisite:
You are advised to set file consistency for a standard cluster under File Server Sharing > SFTP.
Ensure that the status server is configured successfully. Otherwise, the file server cannot be configured.
Ensure that the SFTP service is started. Otherwise, FineBI cannot successfully connect to the file server.
1. Copy the WEB-INF folder under the %FR_HOME%\webapps\webroot directory to the file server.
2. Assign permissions (777 in the Linux system) to the WEB-INF folder in the file server.
3. Open port 22 for the firewall of the file server where SFTP is located.
4. Prepare the path of the WEB-INF folder in the file server, such as /home/ftpuser/WEB-INF.
5. Prepare the encoding format, host IP address (namely internal network IP address), port number, username, password, and key file of the file server.
1. Set File Synchronization Setting to Share File Server.
2. Select SFTP for Agreement.
3. Enter the file server information prepared in the previous section.
4. After configuration, click Test connection and save. If no exceptions occurs, the settings are saved successfully.
You are advised to disable the cache mode for standard direct connect clusters.
You are advised to select Cluster Communication Protocol to TCP for standard clusters.
1. Enable Open Cluster.
2. Restart the node project.
3. Verify that the project node appears in Node Management.
1. Deploy other project nodes for clusters.
2. Copy project files.
Copy the webroot folder in the %BI_HOME%\webapps directory of the standalone cluster node to the %BI_HOME%\webapps directory of other project nodes.
3. Restart these project nodes.
Log in to each project node in turn as admin and verify that all project node information appears in Node Management.
Load balancing can distribute users' requests to different nodes through various algorithms for better performance and load balancing effects.
You are advised to select Nginx for load balancing in a standard cluster.
1. Modify nginx.conf.
Use this file as a basis: Extract Cluster Nginx.zip
2. Restart Nginx components.
Enter IP address: Port number for load balancing/status in the browser address box and view the status of each node in the project on the page displayed..
Then, a standard multi-node cluster has been successfully deployed.
The extract cluster takes effect automatically as the highly available cluster takes effect.
Log in to FineBI as admin, choose Manage > System > Extract Cluster, and view node information.
The SYNC and ASYNC values refer to the synchronous node and asynchronous node, respectively. For details, see Extract Cluster Interface Management.
You can try to add, update, and query the extracted basic tables and self-service datasets. If no exceptions occur, the extraction cluster has been successfully deployed.
Operation
Cluster Registration
If the standalone project was originally registered, it needs to be re-registered after upgrading to a multi-node cluster.
滑鼠選中內容，快速回饋問題
滑鼠選中存在疑惑的內容，即可快速回饋問題，我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy