This document briefly describes each data table and the fields in these tables in the LogDB database.
For platform-related LogDB data tables and fields, see LogDB Table Structure.
fine_pipeline_sync_informationThis table stores synchronization records.
fine_pipeline_log
This table stores pipeline logs.
Log classification information
0-Key task nodes; 2-Data writing; 3-Transmission queue; 4-Data reading
5-DDL capture; 6-DDL application; 7-Dirty data records; 8-Dirty data processing
9-Resynchronization; 10-Heartbeat detection
fine_record_fdl_dataservice_monitor_info
This table stores call records of data services.
fine_dp_work_exec_record
Historical operation record table: New operation records are directly inserted into it.
1. Starting from FineDataLink V4.1.7.3, this table has been migrated from FineDB to LogDB, with the table name unchanged.2. Each execution of a scheduled task generates an execution record.
Statistical information of running instances and the storage path of task snapshots. Example value: dpworks/record/2024-03-13/04c51446-0053-48f3-b507-bad1bdf46592.log
fine_record_fdl_dev_realtime_exec_info
This table stores the basic running information of Real-Time Task (added in FineDataLink V4.2.8.3).
Start Type:
START: Start
RESTART: Reset and start
Running Status:
START_FAILED: Manually paused
RUNNING_ERROR: Startup failed
MANUAL_STOP: Error-induced abort
RUNNING: Running
Detailed Running Information of Tasks:
memoryLimit: Task memory limit during runtime
memoryMAX: The maximum value of memory occupied by this run during its execution period.
memoryAVG: The average value of memory occupied by this run during its execution period.
lagTimeMAX: The maximum value of latency time occupied by this run during its execution period.
lagTimeMIN: The minimum value of latency time occupied by this run during its execution period.
lagTimeAVG: The average value of latency time occupied by this run during its execution period.
fine_record_fdl_dev_realtime_Unit_exec_info
This table stores running statistical information of the execution unit for Real-Time Tasks (added in FineDataLink V4.2.8.3).
It records the detailed execution information of this execution unit.
QPSMAX: It represents the maximum output quantity per second of this execution unit in the record of this run.
QPSAVG: It represents the average output quantity per second of this execution unit in the record of this run.
fine_record_fdl_dev_realtime_Unit_line_exec_info
This table stores the running connection information of the execution unit for the Real-Time Task (added in FineDataLink V4.2.8.3).
The actually running execution units and connections of real-time tasks do not match the task configuration information; therefore, it is necessary for you to collect the runtime connection information separately.
Operation record ID
The relationships between tables are as follows:
This table records the configuration information of the FineDataLink Data Service API.
The main fields are introduced in the following table:
API request method:
GET
POST
Request Body Format (It is -999999 when the API request method is GET.)
application/json
x-www-form-urlencoded
API internal service configuration, which stores configuration information in the form of JSON.
JSON attributes of the Data Publication API records:
For special usages appearing in SQL: when they appear, deduplicate and record them (separated by commas); if none appear, no record is made.
For example: if, join, case when, nvl.
Parameter List (JSON array)
[{"paraName":"", "paraType":"", "isRequired":""}]
paraName: Parameter name
paraType: Parameter type
isRequired: Whether it is required.
JSON attributes for Data Reception API records:
Whether to wait for the service execution result.
Y (Return a response after the write operation is completed.)
N (Return a response immediately upon receiving the data.)
Receive Type:
JDY
COMMON
Different ID values represent different module information. Refer to the following table for details:
globalInfo: represents FineDataLink version information
Task metadata of the scheduled tasks, including task attribute values.
Scheduled task name
Other Configurations of the Scheduled Task, with Values as JSON Objects.
Example：
"timeoutConfig": {
"hour": 1,
"minute": 0,
"enabled": true //Enable or not: true for enabled, false for disabled.
}
"retryConfig" :{
"max": 3, //Retry count
"delayMinute": 2, //Interval time
},
If it is not set separately, it will be set to 0 by default.
"NoticeStore":{
"notification": true, //Enable or not: true for enabled, false for disabled.
"noticeConfig": ["notifyInDetail"] ,//Result notification. If checked, record the value: notify: Task exception notification and task failure alert (checked by default); notifyInDetail: Detailed error information; notifyDirtyData: Dirty data notification; notifyDirtyDataInDetail: Dirty data summary information; notifyDDLChangeEvent: Source table structure change notification
"userGroup": true, //Whether the platform user group is configured: true for configured, false for not configured.
"noticeChannels":["sms"],//Notification channel values are JSON arrays. sms: SMS; platform: Platform; email: Email; Enterprise WeChat app push: Enterprise WeChat app push; Enterprise WeChat robot: Enterprise WeChat group robot; Dingtalk app push: DingTalk app push; Dingtalk group bot: DingTalk group bot; Feishu Push: Lark app push; Feishu Group Robot: Lark group robot
"WorkPriorityStore":{
"taskPriority": 2, //Task priority. Lowest: -2; Low: -1; Medium: 0; High: 1; Highest: 2
"customLogLevel": true, //Whether the log level is enabled: true for enabled, false for disabled
"logLevel": "ERROR" //Log level: Includes ERROR, WARN, INFO
Scheduling plan ID
Scheduling Type
time：Timed Scheduling
event：Event-Based Scheduling
Plan Type
Single：Single-task plan
Batch：Batch plan
If the scheduling type is Event-Based Scheduling, the default values are set to Batch.
Time Scheduling:"schedule_detail_time": { "task_quantity": 1, // Scheduled task count "schedule_status": "open", // Scheduling status: open = enabled; close = disabled; ended = completed "frequency": 1, // Execution frequency configuration: 0 = disabled; // 1 : execute once only; 2 : simple recurring execution; 3 : detailed frequency setting; // 4 : expression-based setting "calendar": true // Whether to reference scheduling calendar: true = referenced; false = not referenced}Event-based Scheduling："schedule_detail_event": [ { "group_name": "Group Name 1", // Event scheduling group name "group_task_count": 4, // Number of tasks in current group "judgment_type": "regular", // regular : scheduled judgment; immediate : real-time judgment "judgment_detail": { // Condition details "limit_count": 2, "time_unit": "weeks", // Value options: weeks : Week; days : Day; hours : Hour; minutes : Minute "type": "success" // Value options: success : execute after success; unconditional : execute unconditionally; // Value is fixed as success for scheduled judgment }, "repeat_judgment": true // Whether upstream records triggering current task group execution will be excluded from subsequent condition judgment: // true : enabled; false : disabled }, { "group_name": "Group Name 2", // Event scheduling group name ... } ...]
Operator information of the data flow of the scheduled task
Operator name
Operator type
Other connection configurations
Pipeline task ID
Task Status
IDLE - Idle
RUNNING - Running
ERROR - Error
PAUSED - Paused
Synchronization Type:
FULL_AND_INCREMENTAL - Full + Incremental
Enable DDL or not
Dirty data threshold (If it is not enabled, it will be set to 0 by default.)
Notification channels, value is a JSON array, where:
SMS: Indicates SMS notification channel
Platform: Indicates the platform message notification channel
Email: Indicates Email notification channel
API name
API status
ONLINE（Online）
OFFLINE（Offline）
API request method
Request Body Format (It is empty when the API request method is GET.)
SQL parameter list (JSON array)
paraName：Parameter name
paraType：Parameter type
isRequired：Whether it is required.
Definition
Authentication method
EmptyAuth（No authenticatio）
DigestSignatureAuth（Digest authentication）
AppCodeAuth（AppCode authentication）
It represents the binding relationship between the Data Service App and APIs.
It records the Real-Time Task configuration information.
Field
It stores the basic task execution information.
execId: Latest execution record ID (It is empty if the record is deleted.)
status: Task status
lastStartTime: Last start time of the task
firstStartTime:First start time of the task
It records the Real-Time Task operator configuration information.
It stores complete operator configurations, with the following exceptions:
When the SQL configuration method is selected for dimension table input, it does not record the SQL statement, but only records SQL keywords and SQL size.
It stores the Real-Time Task connection information.
It stores the Real-Time Task parameter information.
It represents the attribute configuration of storage parameters.
taskId: It indicates which real-time task it belongs to.
paramType: Fixed as "static_param"
type: Field type of the parameter
valueSize: Parameter value size
Task/node/operator execution records.
Time imported to the cloud
This table stores the daily operation records of pipeline tasks.
Pipeline task name
This table records the daily API call information of the FineDataLink data service; it has a granularity down to a single API call, and the call record ID serves as the primary key.
SUCCESS
FAILED
It stores the API internal service execution details.
API updated data volume (rows)
API deleted data volume (rows)
滑鼠選中內容，快速回饋問題
滑鼠選中存在疑惑的內容，即可快速回饋問題，我們將會跟進處理。
不再提示
10s後關閉
Submitted successfully
Network busy