For projects deployed on the operation and management platform, if you select the ElasticSearch component, ElasticSearch will take the place of swift(logdb) and be stored as logs.
If Elasticsearch Connection Configuration is displayed on the General page under System Management > System Setting, ElasticSearch has been enabled to be stored as logs.
Do not use the plugin in this document for log sync. The plugin is only applicable to the situation where swift(logdb) is stored as logs.
If you need to connect the project log service, you can contact our technical support for help. For details about how to contact technical support, see Technical Support Channel Introduction.
6.0
V1.0.0
Currently, LogDB does not support migration. It is inconvenient for you to perform complex analysis on log data. You may hope to migrate log data to your own databases for management and analysis.
You may want to place logs of various systems in the same place for operation and management.
After installing the Sync Platform Logs to Other Databases plugin, you can periodically sync some table data from LogDB to other databases through setting.
You can obtain the plugin at https://market.fanruan.com/plugin/b33bbbb4-575e-4650-b961-f76e81564549.
For details, see Plugin Management.
Since LogDB contains too much log content, you need to prepare a new and dedicated table space for data migration and storage.
You are not advised to place the LogDB data and other business data/configuration data in the same table space.
MySQL
5.1.73, 5.5.31, 5.5.46, 5.5.56, 5.6.22, 5.6.31, and 5.7.16
SQLServer
2000, 2005, 2008, 2012, 2014, and 2016
Oracle
10g/11g/12c
After the plugin is successfully installed, choose System Management > Intelligent O&M > Platform Log. The Log Sync tab is added to the page, as shown in the following figure.
1. Log in to the decision-making platform as the admin, choose System Management > Intelligent O&M > Platform Log, and click Set Log Sync Task on the Log Sync tab page, as shown in the following figure.
2. On the Log Sync Setting page, select Log Sync Start Time, tick the logs that need to be synced, and click Next, as shown in the following figure.
The logs that support sync include:
Report Execution Log: fine_record_execute
Login Log: fine_record_login
Logout Log: fine_record_logout
Log Sent by Email: fine_record_email
Data Entry Log: fine_record_write
Common Parameter Combination: fine_record_param
Log Sent by SMS: fine_record_sms
Management Behavior Log: fine_record_operate
SQL Execution: fine_record_sql
Real-Time App Situation: fine_real_time_usage
GC Log: fine_record_gc
conf_entity Configuration Info: fine_record_conf
User Lock Log: fine_record_lock
Template Attribute Record: fine_tpl_info
Container Info Log: fine_container_entity
fine_function_process
Cloud O&M Log: fine_operation_process_log
Semaphore Info: fine_operation_shutdown_record
Plugin Use Info Record: fine_plugin_usage
Task Schedule Execution Log: fine_schedule_record
Function Use Info: fine_intelli_focus_point
Performance Info Log: fine_intelli_consume_point
Enter the relevant information of the database where the log data will be migrated, and click OK, as shown in the following figure.
The plugin performs sync every three minutes. A maximum of 30,000 records can be synced per table each time. If the data to be synced this time is less than 30,000 records, only data within the last 24 hours can be synced.
For example, if 30 records of data exist during the period from 2021-08-02 13:00:00 to 16:00:00, a maximum of these 30 records can be synced during this sync.
After successful sync, the sync status is as shown in the following figure.
Notifying the Admin If Sync Fails
If Notify Admin If Sync Fails is ticked, the admin will receive a platform message when sync fails, as shown in the following figure.
Pausing Sync
You can click the Pause button to pause the sync, as shown in the following figure.
Editing Sync Setting
When sync is paused or fails, you can click the Edit button to re-edit the sync task, as shown in the following figure.
Instructions for setting Effective Object of Start Time:
If you want all selected tables to start syncing from the previously set Log Sync Start Time, set Effective Object of Start Time to All Logs to Be Synced.
If you want only newly added tables to start syncing from the set value of Log Sync Start Time, and the rest of the tables that are already being synced to continue syncing from where they left off, set Effective Object of Start Time to Only Effective for Log in This Change.
Viewing Sync Details
Click the View Sync Details button to view the sync details, as shown in the following figure.
Only the operation status of the last sync task will be displayed. The following table describes each field.
Synced Table
Selected logs to be synced
Log Sync End Time
The latest time of the stored log in the last executed sync task, indicating that all logs from Log Sync Start Time to this time point have been stored
Last Sync Date
Start time of the last executed sync task
Volume of Data Synced Last Time
Number of log records synced in the last executed sync task
Time Consumed by Last Sync
Time span of the last executed sync task
Operation Result
Successful/Failed
Click the xxx icon to view the specific error if the operation result is Failed.
Log data will be synced to the configured database.
1. syncID (auto-increment primary key) and storageTime (timestamp field) will be added to the synced table, as shown in the following figure.
2. Log cleaning will not affect the log data that has been migrated to external databases.
3. Content in the field can be stored without a character limit, but content exceeding 255 characters will be truncated.
4. For compatibility with various versions, aliases are created for field names that use database-reserved names.
The following table describes the matching between specific fields and aliases.
FINE_RECORD_OPERATE
type
operateType
resource
operateResource
status
operateStatus
FINE_RECORD_EXECUTE
executeType
sql
query
sqlTime
queryTime
FINE_RECORD_WRITE
Sync Logic
Sync Frequency
If the data to be synced this time is less than 30,000 records, data within the last 24 hours (at most) can be synced (Before March 23, 2021, only data within the last three hours can be synced.)
Sync Field Length
Content in the field can be stored without a character limit. However, content exceeding 255 characters will be truncated, and only content within 255 characters will be submitted.
Sync Status
If the sync fails, you can troubleshoot the following database problems.
Checking the Database Version
You can view the supported database types and versions by referring to section “Database Supported for Migration.”
Checking Database Information
Check if the database driver, schema, and password are correct. You can first test connection under System Management > Data Connection to see if the connection works properly.
Database reboot will interrupt the log sync process. You need to restart the task on the platform when the sync terminates.
Checking the Database User Permission
You need to have the create, delete, alter, update, select, and insert permissions.
Checking the Database Unicode
UTF-8 is recommended for MySQL5 database. The utf8mb4 code is not supported.
If you are using ApsaraDB RDS for MySQL, you also need to check whether the engine is InnoDB.
When the sync fails and any of the following three situations occur, you can troubleshoot the following plugin problems.
Problem:
1. Duplicate plugin packages are found in the fanruan.log file.
2. The Log Sync page displays blank content.
3. The Log Sync page displays content in English.
Checking Plugin Installation
Check whether the plugin has been installed repeatedly.
If so, delete all the plugins, restart the server, and reinstall the plugin.
Checking the WebSocket Port
Check whether the WebSocket port is working properly. For details, see WebSocket Connection Failed.
When sync fails, you can check the fanruan.log file and locate the problem through error messages containing "com.fr.plugin.sync".
SyncLog: getConfigInstance failed!, retry to getConfigInstance!
java.lang.AssertionError: com.fr.plugin.sync.config.status.IsStart has the same tableName with com.fr.plugin.sync.config.status.IsStart
2. You may have installed both the original log data sync plugin and the Syncing Platform Logs to Other Databases plugin. You need to delete the log data sync plugin.
Cannot set int field com.fr.plugin.sync.db.entity.Schedule.logType to java.lang.Long
logDB does not support int type. You need to change the int type to the long type. Update the plugin to 1.1.2 or later versions.
SyncLog: get logs failed from XXX
The Syncing Platform Logs to Other Databases plugin cannot support the sync of particularly large amounts of log data. You are advised to clean up the table data appropriately before the sync.
SyncLog: sync failed: com.fr.third.org.hibernate.StaleStateException:
Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
java.sql.SQLException: Incorrect string value: '/xE4/xB8/xB4/xE6/x97/xB6...' for column 'userrole' at row 1
QueryTimeoutException- could not extract ResultSet
If so, handle them before attempting to perform sync again.
could not execute statement
Check whether the Oracle schema is set to thin. If so, change it to the username.
滑鼠選中內容，快速回饋問題
滑鼠選中存在疑惑的內容，即可快速回饋問題，我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy