反馈已提交

网络繁忙

Resource Migration

  • Last update:  2023-06-30
  • Overview

    Version

    FineBI Version

    Functional Change

    6.0

    /

    6.0.11
    Limited the size of uploaded files dynamically to prevent memory overflow.

    Application Scenarios

    You are admins, and you want to carry out data development and deployment in the testing environment system before migrating all the resources to FineBI system.

    FineBI supports the migration of resources like directories, datasets, data connection, and dashboards from the testing system to the official one.

    Functions

    After being installed with the resource migration plugin, FineBI supports the incremental migration of resources migration of directories, datasets, data connection and dashboards from the testing system to the official one.

    After the plugin installation, log into FineBI, click Manage > Intelligent Operations > Resource Migration and you can find that this function has two parts: Resource Export and Resource Import.

    • Resource export: export resources from the server and migrate them to other systems.

    • Resource import: import the migrated resources to the new system.

     QQ图片20230424105418.png

    Notes

    1. In terms of the official project, you need to buy the function point named Intelligent Operations before using Resource Migration.

    2. Make sure that the super admin account is only used by the one operating the resource migration, or some migration requests may be rejected.

    3. You are not recommended to synchronize the templates developed on PC with the company project through resource migration.

    Because the projects on PC are not maintained by the admin, the resource migration may fail due to mismatch of JAR package versions, resource mismatch or configuration mismatch, rendering the O&M cost rather high.

    Preparations

    Plugin Installation

    To use the resource migration function, you need to install the resource migration plugin.

    For plugin download, you can send an email to international@fanruan.com or click 微信图片_20230512093844.png at https://intl.finebi.com/en.

    For details about plugin download, see Plugin Management.

    Version Confirmation

    Make sure that the imported and exported FineBI projects are of the same version.

    You can click Manage > Registration > Version Information to check the version.

    If the versions are different, you can upgrade FineBI. For details, see [Standalone] Upgrade from 5.1 to 6.0.5.

     QQ图片20230424110519.png

    Note: You cannot import resources with the project version in conflict with the current one.

    Error message:

    Unable to import!

    The project version required by the resource to be imported:{}

    Current project version:{}

    Please make sure the project version is consistent!

    Backup and Restore

    You are recommended to perform Backup and Restore of both the testing system and the official system.

    Note: You need to use the utf8_bin order rule when backing up data in MySQL database.

     QQ图片20230424113923.png

    Database Case Sensitivity Confirmation

    If the system is configured with the external database, make sure that before the resource migration, the default database character verification rule in both the testing and official systems is case-sensitive.

    Take MySQL database as an example.

    1.     Settings

    Setting method: If FineBI is configured with the external database, link to the FineDB database. Open the FINE_CONF_ENTITY table, set the character verification rule as utf8_bin, and enter the following command:

     

    ALTER TABLE fine_conf_entity MODIFY id VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_bin;  #Set the verification rule of the column

    ALTER TABLE fine_conf_entity DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;  #Set the verification rule of the table


    2.     Effects

     微信图片_20230512092425.png

    Plugin and Driver Confirmation

    1. Data connection needs the installation of a third-party plugin. FineBI only supports the migration of JSON dataset and MySQL data connection. Using other incompatible plugins will lead to resource import failure.

    2. The server dataset generated by the permission export plugin does not support resource migration, thus unable to be imported.

    3. Data connection migration only migrate the configuration. You need to configure / place JAR package, Kerberos authentication files and JNDI configuration files, as they cannot be migrated.

    • Check whether you have installed relevant data connection plugins before the data connection migration, or the resource import will fail.

    • Check whether you have installed relevant data connection drivers before the data connection migration, or the imported data connection will be unusable.

     微信图片_20230512093321.jpg

    Data Connection Confirmation

    • If you need to carry out the data connection migration, create the data connection in the official system before the resource import. The data connection name should be identical to the one in the testing system, or the resource migration will fail.

    • If you want to migrate the dashboard created by the server dataset, make sure to create a server dataset of the same name in the official system before the resource migration.

    • If the server dataset is based on certain data connection, check whether you have imported the related data connection into the official project, or the imported server dataset will be unavailable.

    User Confirmation

    1. User

    The owner of the resource package should be in the system when you import the resource.

    For example, if the name of the table creator, Anna, is in the testing system, then there must be a user named Anna in the official system. Or you need to modify the resource owner before importing the resource.

    2. User Type

    The creator of the FineBI table should be of the same type as the user during the resource migration.

    For example, the creator of the table in the testing system is an analysis user, then in the official system, he/she should be classified as an analysis user.

    Migration of Custom Map

    During the resource migration, if you have added a custom map to the template, you need to copy the map's JSON file under %FineBI%/webroot/WEB-INF/assets/map/image to the corresponding path in the official project.

    Resource Export in the Testing System

    Selecting Resources

    Log into FineBI, click Manage > Intelligent Operation > Resource Migration > Resource Export.

    You can select the Resource Type and then choose the specific content for resource export. Multiple selection is supported.

     QQ图片20230511151418.png

    Resources that allow being exported:

    Resource Type

    Explanation

    Catalog

    The smallest unit supporting selection is dashboard.

    • The exported resources will include: dashboard configuration, ID, escape name, owner name, physical path (folder and subject), platform publishing path, dashboard style (only the current dashboard style), dashboard watermark (excluding the global watermark), sharing status, collaboration status.

    Note: Not all dashboards/data in the system would be exported. Unpublished data will not be exported. Therefore, publish the dashboard before exporting it.

    My Analysis

    The smallest unit supporting selection is dataset, component and dashboard.

    • When you export the table, the exported resources will include: dataset configuration, association, ID, escape name, owner name, physical path (folder and subject), platform publishing path (folder), set field grouping, sharing status, collaboration status.

    • When you export the component, the exported resources will include: component configuration, ID, escape name, owner name, physical path (folder and subject), collaboration status.

    • When you export the dashboard, the exported resources will include: dashboard configuration, ID, escape name, owner name, physical path (folder and subject), platform publishing path, sharing status, collaboration status.

    Note: When you export resources, all these resources will be imported into My Analysis of the related user based on the export path. If the related user does not exist, you need to change the resource owner.

    Public Data

    The smallest unit supporting selection is dataset.

    • When you export the table, the exported resources will include: dataset configuration, association, ID, escape name, owner name, physical path (folder and subject), platform publishing path (folder), set field grouping.

    • When you export the folder, the exported resources will include: folder configuration, ID, escape name, owner name, physical path (folder).

    Note:  

    1.   When you export resources, all the associations will be exported. When you import resources, if the associated table does not exist, the relevant associations will be cleared automatically.

    2.   You need to apply for table publishing again after importing the table if the table is applying for publishing when you export it.

    Data Connection

    You can select the data connection in the FineBI system.

    When you export the data connection, the exported resources include all configurations of data connection, basic one and high-level one alike.

    Note: During the resource export, the password of data connection will be encrypted. The keytab key path does not allow being exported, and you need to manually enter the password after the migration.

    Server Dataset

    You can select server datasets in the FineBI system.

    Datasets allowing being exported: SQL datasets, file datasets, tree datasets, JSON datasets, built-in datasets, program datasets, association datasets, stored   procedures.

    • File dataset

    If the file dataset source is URL, only the file configuration will be exported, and files will not be exported.

    If the file dataset source is file, and the file path includes parameters, the files will not be exported.

    • Tree dataset

    When you export the tree dataset without exporting the original dataset, you can still import the tree dataset afterwards, but the imported dataset cannot be used.

    • Association dataset

    When you export the association dataset without exporting the original dataset, you can still import the association dataset afterwards, but the imported dataset cannot be used.

    • Permission export plugin

    The server dataset generated by the permission export plugin does not support resource migration, thus unable to be imported.

    • Third-Party plugin dataset

    Server Dataset needs the installation of a third-party plugin. FineBI only supports the migration of JSON dataset and MySQL data connection. Using other incompatible plugins will lead to resource import failure.

    Exporting All Dependent Resources

    For example, dataset A is created based on dataset B. If you want to migrate dataset A to the official system and use it, you need to import dataset B as well.

    Therefore, when selecting resources of Catalog, My Analysis and Public Data, you need to select Export all dependent resources.

    This function allows exporting all the resources that datasets, components and dashboards of these three types are dependent on.

    This function only works for the current resource type, so you need to set different resource types one by one.

     QQ图片20230511151717.png

    Dependent resources that allow being exported:

    Resources

     

    Dependent Resources that Allow Being Exported

    Datasets

    Rules:

    Tables directly used in datasets can be exported.

    Tables related to the datasets can be exported.

    Example:

    Dataset B is created based on dataset A, and dataset C is created based on dataset B.

    Dataset C's dependent resources are dataset A and dataset B.

    Note: If you click Export all dependent resources, the dataset’s dependent resources in My Analysis will be imported into the corresponding directory in My Analysis based on the export path.

    Component

    Rules:

    Datasets directly used in the component can be exported.

    Datasets related to tables used in the component can be exported.

    Example:

    Dataset B is created based on dataset A, dataset C is created based on dataset B, and component X is created based on dataset C.

    Component X's dependent resources are dataset A, dataset B and dataset C.

    Note: If you click Export all dependent resources, the dataset’s dependent resources in My Analysis will be imported into the corresponding directory in   My Analysis based on the export path.

    Dashboard

    Rules:   

    Components used in the dashboard can be exported.

    Datasets directly used in the dashboard can be exported.

    Datasets related to the tables used in the dashboard can be exported.

    Only datasets directly related to component associations and widgets’ filtering of components can be exported.

    Example:

    Dataset B is created based on dataset A, dataset C is created based on dataset B, and component D is created based on dataset C.

    Dataset Y is created based on dataset X, dataset Z is created based on dataset Y, and filter component W is created based on dataset Z.

    Dashboard A is created based on component D and filter component W.

    Dashboard A's dependent resources are component D, dataset A, B, C, and dataset X, Y, Z.

    Note: If you click Export all dependent resources, the dataset’s dependent resources in My Analysis will be imported into the corresponding directory in My Analysis based on the export path.

    Simultaneous Export

    For example, dataset has timed update tasks, you may need to import the timed update tasks into the official system during the resource migration.

    During the Export, you can select Original excel attachment, Setting items for public links, Timed update tasks and Export permission configuration at the same time to export all of them simultaneously.

    This function works for all resource types.

     QQ图片20230511151808.png

    Resources that allow being exported simultaneously:

    Exported Resources

    Explanation

    Original excel attachment

     

    If you want to export resources or dependent resources with Excel datasets, select Original excel attachment to export Excel attachments along with the resources, so as to avoid importing Excel files afterwards.

    Note: If you import table A into the official system, and in the same placeis a table named table A as well, then the original one would be covered. Therefore, you need to check the table name before importing a new table.

    Setting items for public links

    If the resources to be exported have dashboards, you can select Setting items for public links to export all the public links of the dashboard.

    • Select the option:

    Whether to overwrite the original dashboard or add a new dashboard during the import   will be determined by the public link settings of the imported dashboard.

    • Unselect the option:

    If you choose to overwrite the original dashboard, the public links of the original dashboard will not be overwritten.

    If you choose to add a new dashboard, the public links of the imported dashboard will be disabled by default.

    Timed update tasks

    If you want to export resources or dependent datasets with times update tasks, select Timed update tasks, and these tasks would be exported along with the resources. Only their own update tasks will be exported, update tasks in their parent node will not be exported.

    • Select the option:

    Whether to overwrite the original dataset or add a new dataset during the import will be determined by the update task settings of the imported dataset.

    • Unselect the option:

    If you choose to overwrite the original dataset, the update tasks of the original dataset will not be overwritten.

    If you choose to add a new dataset, the imported dataset will not have update tasks.

    Export permission configuration at the same time

    Rules:

    You can export the permission configuration of the resources after selecting this option.

    The configured permissions include: usage permission (viewing, editing, authorization and etc.), row and column permission.

    Only resources' own permissions will be exported, and permissions inherited from the parent node will not be exported.

    When you export permissions of resources like folders, subjects, dashboards and datasets, inherited permission will not be exported.

    Example:

    Folder X has a dataset A, and both the file and the dataset have been configured with the usage permission.

    • Select Export permission configuration at the same time. If you export dataset A only, only dataset A's configured permissions will be exported, and folder X's configured permissions will not.  

    • If you export both folder X and dataset A, configured permissions of dataset A and folder X will all be exported.

    Exported File

    A file named resources.zip will be generated.

     QQ图片20230511152121.png

    Importing Resources in the Official System

    Importing Files

    Log into the FineBI system, click Manage > Intelligent Operations > Resource Migration > Resource Import > Upload Files.

    Note:

    1. You are recommended to perform Backup and Restoration before importing resources. You also need to select the utf8_bin order rule for data processing when migrating the backup data to MySQL database.

    2. You can only import one compressed file for one time.

     QQ图片20230511152445.png

    The system allows you to upload files smaller than ( the available memory obtained automatically minus 256 MB). If the file size is larger than the available memory obtained automatically minus 256 MB, an error message is displayed due to restriction triggering, indicating that upload is disallowed because the current available memory is xxx MB while the file size is xxx MB, as shown in the following figure.

    The system will get all the file resources after the upload, including Resource Type, Resource Name, Resource Owner, Paths, Import Mode and Resource information comparison.

     QQ图片20230511152535.png

    Setting

    Explanation

    Resource Type

    Include catalog, server dataset, data connection, dashboard, component and dataset.

    Resource Name

    Names of all types of resources.

    Resource Owner

    If the resource type is catalog, server dataset or data connection, the resource owner will be empty.

    If the resource type is dashboard, component or dataset, the resource owner will be the resource creator.

    Creator of the resource

    Paths

    If the resource type is catalog, server dataset or data connection, the path will be empty.

    If the resource type is dashboard, component or dataset, the path would be the location of resources in Public Data / My Analysis.

    Import Mode

    The system will detect the imported resource to check whether they will conflict with the existing resource.

    Based on conflict types, import modes include New   import, Override import and Import is not allowed.

    • Add import: If the imported resources' IDs and types are not identical to the existing ones', you can import these resources directly.

    Note: If those with the same IDs and types as the existing ones are placed in the recycle bin, you can also import them directly.

    • Override import: If the imported resources' IDs, types and owners are the same as the existing ones', the   existing resources will be overwritten.

    • Import is not allowed: If the imported resources' IDs, types are the same as the existing one's, but their owners can not be found in the system, you need to modify their owners before importing them.

    Resource information comparison

    When the import mode is Override Import, click 微信图片_20230512094224.png , and you can check whether the imported resource information is consistent with the original one.

    QQ图片20230511152627.png

    Note:

    1. If the imported resources' escape name is the same as the existing ones' under the given path, the system will automatically modify the imported resources' escape name as {escape name1}.

    2. You cannot import empty folders.

    3. If the resources are not allowed to be imported, and their owners are unknown, you can modify the owners to the existing owners in the system, and click OK to make it take effect.

    • If resources are datasets in Public Data, modifying the resource owners will only change the user IDs of the resources.

    • If resources are subjects in My Analysis, modifying the resource owners will change the subjects' paths to the ones in the corresponding users' My Analysis and refresh user IDs.

    Importing Files

    Select resources as needed, click Import and then click OK.

     QQ图片20230511152805.png

    If the import fails, the reason will be displayed. If the import is successful, the resources will take effect immediately.

     QQ图片20230511152835.png

    Note: Because IP port numbers will change according to the corresponding environment configuration, you need to re-share the public links of dashboards after the resource migration.

    Data Upgrade

    If the imported basic tables' resources are extracted data, you need to update data manually, or perform Global Update directly.

    You cannot view the imported tables' data normally without updating them.

    Note: If the imported tables are Excel datasets or datasets based on basic Excel datasets, you have to perform Folder Update or Global Update.

    If the imported tables are database tables or SQL datasets, you can update the table sheet.

    附件列表


    主题: Admin Guide
    Previous
    Next
    • Helpful
    • Not helpful
    • Only read

    feedback

    鼠标选中内容,快速反馈问题

    鼠标选中存在疑惑的内容,即可快速反馈问题,我们将会跟进处理。

    不再提示

    10s后关闭