Data Lineage

Version

Application Scenarios

Data Lineage in FineBI records the kinship among data tables and components.

If you have a data table Banking Institution Indicator Data, you can see its parent tables and its child resources (including child tables, components, and dashboards) in Data Lineage.

Data Lineage can help us:

• Track the data source: Track the original source table of the current table to check data accuracy and credibility.

• Troubleshoot: If quality problems, abnormal analysis results, or update errors occur in the current table, you can track the source and update status of the data through Data Lineage, locate the root cause of the problem, and take appropriate measures.

• View the scope of impact of the current table: If the current table has numerous child resources, it has an extensive impact. Modify or delete this kind of table with caution.

Functions

In the Data Lineage tab page, you can:

• Learn the scope of impact of the current table by viewing its parent tables and child resources (including child tables, components, and dashboards).

• Click  to directly jump to the editing page of the corresponding data table or dashboard.

Viewing Data Lineage

Click Public Data, select a table, and click Data Lineage.

You can hover your cursor over the tag of a table/component/dashboard to know more information like Creator, Location, and Creation Time.

In the detailed information of the data table, the lineage level of the data table is displayed, which represents the level of the data table in the lineage.

• If the lineage level of the data table is 1, the data table does not have any parent tables and is the ancestor or one of the ancestors of this lineage.

• If the lineage level of the data table is 2, the data table has one level of parent tables.

• If the lineage level of the data table is 13, the data table has 12 levels of parent tables. All the tables on these 12 levels are its source tables, to which any changes can affect this data table. You need to avoid such deep lineage levels. For details, see section "Avoiding the Creation of Data Tables with Deep Lineage Levels".

Data lineage can be further expanded if the parent table has its own source table, the child table has its own child table/component/dashboard, and the component is used in a dashboard. The lineage of sublevels is folded by default, which can be unfolded manually.

Clicking to Jump to the Corresponding Location

You can click to jump to the corresponding table.

You can click to jump to the editing page of the corresponding component or dashboard.

Risks Caused by Deep Lineage Levels

The following figure shows that the current table has countless parent tables and each parent table has countless parent tables, forming deep lineage levels. If a problem occurs in the current table, you may find it difficult to track the root cause and locate the problem from the parent tables, which complicates troubleshooting and tracking.

Even worse, if a problem occurs in any of these nume parent tables, the current table may become unavailable. Such dependency significantly reduces the stability and availability of the entire data table.

The above-mentioned extreme case warns you to avoid creating and using data tables with deep lineage levels. In addition, other problems may occur as follows.

• Difficult to understand and maintain: If the table structure has deep levels, the relationships between data elements become complex, making it difficult to read and understand the logic of the table structure. Therefore, data maintainers and developers may find it hard to accurately understand the source and destination of data in the table, which complicates the maintenance.

• Easy to cause errors and risks: Deep-level tables are easily affected by thousands of data tables simultaneously. The current table cannot be used if there is an error in even one parent table.

• Inefficient: The operations of querying and manipulating data may become inefficient in a deep-level table. If data retrieval needs to traverse multiple levels, query performance and data access efficiency may be affected. Data table update is also impacted significantly, thus consuming unnecessary update resources.

Ways to Optimizing Lineage

You can simplify the lineage relationship in FineBI through the following ways:

Data in the Public Database Prepared by IT Personnel

Roles of Public Data and My Analysis:

• Public Data: It is a public place to store public data, rather than a private area. In public places like parks and airports, the facilities are prepared by specialized municipal institutions. Similarly, the public data should also be prepared and managed by professional IT personnel, ensuring an organized and efficient public data space and reliable data.

• My Analysis: It is your personal space for analysis, similar to your own home. You can arrange it freely, and will have better usage experience if you keep it organized.

Why data lineage can be optimzied if public data is prepared by IT personnel?

• IT personnel often use professional ETL (Extract, Transform, Load) tools (such as FineDataLink) for data governance, and then add data to Public Data in FineBI through SQL datasets or database tables. As a result, the tables obtained in this way have high quality and do not have parent tables. Data analysis based on these tables does not have complex lineage relationships.

• IT personnel handle and plan data better, reducing the probability of problems that may affect the child resources that are created using these tables.

Avoiding the Publishing of Data with Deep Lineage Levels

You are not advised to publish tables where data is editing to Public Data. While you are advised to publish tables with lower levels to Public Data if necessary.

For example, if a data table with a level of 12 is published to Public Data, the lineage level of new analysis resources created by other users based on the data table is greater than 12.

In contrast, if a data table with a level of 2 is published to Public Data, the lineage level of new analysis resources created by other users based on the data table starts from 3, which is much better than the previous choice.

Trying to Use Data from Public Data for Analysis

Data in public data is cleaner and lower in level, so using data from Public Data to conduct analysis can greatly optimize lineage relationships.

In contrast, data analysis based on data from My Analysis will cause coupling between different subjects and confusion in the lineage of data tables. Therefore, try not to reference data across subjects to avoid such problems.

Unable to Click Dataset in Data Lineage Due to Lack of Permission

• No permission of the current dataset: You have no permission to use this dataset, which needs to be assigned by the admin.

• No permission of the current dashboard/component: You have no permission of the dashboard/component which is not created by you.

• Unavailable current dataset/dashboard/component: The dataset/dashboard/component cannot be used because of update failure. You need to check the update status.

• Lost current dataset/dashboard/component: The dataset/dashboard/component is deleted, unpublished, or unshared.

Data Linage Exceeding the Limit

The maximum lineage level of data tables defaults to 16 in FineBI. If the level of a data table exceeds 16, an error message saying "The current dataset exceeds the lineage level limit." is displayed.

The parent table of the current table has a high level. You can select a lower-level table as the parent table and re-edit the data.