Common Issues and Solutions for Large Dataset Export

This document describes common issues and solutions for large dataset export. It also provides troubleshooting guidance to help you resolve similar issues more efficiently.

For details about large dataset export, see Large Dataset Export by JS and Large Dataset Export Plugin.

Unavailable Function

Large Dataset Export with JavaScript

1. JavaScript cannot obtain parameters from the URL. This is because JavaScript-based export applies directly to templates or data and thus can be performed without a template preview or relying on sessions. However, URL parameters are passed only during preview and are not attributes of the template itself or the server environment.

2. If you add the &_filename_= parameter to the URL in JavaScript to specify the exported file name, the dataset name is automatically appended to that file name. The appended dsname cannot be removed.

3. Error 414 occurs when there are too many parameters.

4. If no value is selected in a multi-select drop-down tree widget, JavaScript obtains an empty array instead of an empty string. As a result, the dataset query cannot identify the parameter as an empty filter condition, causing the query for all data to fail.

5. The function is not supported on mobile terminals.

Data Export with the Large Dataset Export Plugin

1. Data exported to Excel is retrieved directly from the database through SQL statements, rather than from the report. Therefore, data formats configured in the report are not exported.

2. Exporting to CSV files is not supported.

3. If a field for large dataset export to Excel contains slashes (/), the button configuration cannot be saved.

4. The number of exports recorded by the decision-making platform does not include exports performed by the Large Dataset Export plugin.

5. Configuring multiple events for the same button is not supported.

6. Filtered table fields cannot be exported if their field names contain braces ({}).

7. Spider datasets are not supported.

8. The function is not supported on mobile terminals.

Troubleshooting Method

General Limits

1. If the export takes more than five seconds, a progress bar is displayed.

2. Data formats may change. For example, 100.0 is displayed as 100 after the export.

3. A new sheet is created for every 1,000,000 rows during large dataset export to Excel.

4. The data to be exported should not exceed 10 million rows by 20 columns. If the data volume is extremely high, only part of the data may be exported.

Large Dataset Export with JavaScript

When issues occur during large dataset exports with JavaScript, troubleshoot them according to the following figure.

Data Export with the Large Dataset Export Plugin

When issues occur during exports with the Large Dataset Export plugin, troubleshoot them according to the following figure.

Empty Exported Excel File

Issue description

For large dataset export with JavaScript, a drop-down tree widget can be used to filter data. When a parameter value is selected, data can be exported normally. If no value is selected, clicking Query still returns results, but the exported file is empty.

Cause analysis

If no value is selected in a multi-select drop-down tree widget, JavaScript obtains an empty array instead of an empty string. As a result, the dataset query cannot identify the parameter as an empty filter condition, causing the query for all data to fail.

Solution

Manually assign an empty value to the parameter in JavaScript.

Sheet Pagination Every 1,000,000 Rows During Large Dataset Export to Excel

A new sheet is created for every 1,000,000 rows during large dataset export to Excel.

Small Row Height in Exported Excel Files with the Row-based Engine Enabled

Issue description

After the row-based engine is enabled, the row height in the exported Excel files is too small. The same issue occurs when the Streaming Export to Excel (Row-based Engine) plugin is used.

Solution

If this issue occurs in FineReport 8.0 or FineReport 9.0, increase the row height as needed to resolve it. If this issue occurs in FineReport 10.0, use the Large Dataset Export plugin or switch to the new calculation engine to resolve it.

Incomplete Large Dataset Export via a Custom Button

Issue description

For a data volume of 13,000,000 rows by 26 columns, only part of the data is exported.

Cause analysis

For large dataset export with JavaScript, the data to be exported should not exceed 10 million rows by 20 columns. If the data volume is extremely high, only part of the data may be exported.

JSON Array Support for Large Dataset Export API Parameter

Issue description

During large dataset export, when a single parameter is to be assigned multiple values, can the parameter strings be concatenated as a JSON array? To test this, a JSON array is used in the API parameter to pass multiple values for the same parameter, but the exported file is empty, although the console displays a successful export message.

The strings are concatenated in the following formats:

{val:[\"Region A\"]} == The exported file is empty.
{val:[\"Region A\",\"Region B\"]} == The exported file is empty.
{val:\"Region A\"} == Data for Region A is exported.
{val:['Region A','Region B']} == The exported file is empty.{
val:[Region B','Region A]} == An export error is reported: Unexpected End: at character 15 of {val:[Region B','Region A]}.
{val:\"Region A\",val:\"Region B\"}; == Data for Region B is exported.

Cause analysis

The parameters are concatenated incorrectly.

Solution

Region A','Region B needs to be enclosed in quotation marks ("") and the quotation marks need to be escaped to prevent them from being parsed. Use the following code:

var paramStr = encodeURIComponent("{val:\"Region A','Region B\"}")
// Dataset parameters
var colNames = encodeURIComponent("Region,Salesperson,Product Type,Product,Sales Volume") // Specifies the fields to be exported and the export order. If this value is empty, all fields are exported by default.
_g().directExportToExcel("ds5", “Export Test”, paramStr, colNames)

Missing Excel Data Caused by ORDER BY

Issue description

The dataset contains 130,000 records in total, but only 65,535 records are exported to Excel. After ORDER BY is commented out in the SQL statement, all records are exported successfully. The same issue occurs when you export data via the &format=excel parameter, the Excel export function, and the large dataset export function.

Cause analysis

The issue is caused by the database configuration.

Solution

Comment out ORDER BY before using the large dataset export function.

Configured Export Columns Not Applied

Issue description

For data export with the Large Dataset Export plugin, if the name of a selected export field contains slashes (/), the field selection will not take effect.

Cause analysis

Special characters are not supported currently because they may cause HTML parsing issues.

Field Names Exported Without Data

Issue description

After the JAR files of both the system and the designer are upgraded to the version released on November 30, 2021, exporting templates via the Large Dataset Export plugin works normally in a local designer. However, when the designer is connected to a remote server or when the template is mounted to the platform directory, the exported file contains only field names and no data.

Solution

Upgrade the JAR file to a version released on or after December 15, 2021.

Database Support Error During the Export of New Adaptive FRM Templates

Issue description

When you use the Large Dataset Export plugin to export FRM templates with the new adaptive layout and datasets created from MySQL 5.7.23 to Excel, the error message is displayed: "Databases of this type are not supported." FRM templates without the new adaptive layout can be exported normally.

Solution

Upgrade the Large Dataset Export plugin to V1.0.9.

Empty Export Result When Parameter Names Contain Underscores

Solution

Upgrade the Large Dataset Export plugin to V1.1.1 or a later version.

Empty Export Result When a Widget Shares a Name with a Dataset Parameter

Solution

Upgrade the Large Dataset Export plugin to V1.1.5 or a later version.

Parameter Default Affected by Plugin Parameter Configuration

Solution

Upgrade the Large Dataset Export plugin to V1.1.6 or a later version.

URL Parameters Not Obtainable via the Large Dataset Export API

Cause analysis

FineReport with JAR files released on or after April 26, 2020 does not allow obtaining parameters from the URL.

Solution

Add parameters to the SQL dataset of the template, as shown in the following figure.

Connection Timeout During Large Dataset Export

Cause analysis

The data volume is large, causing the export to exceed the timeout period configured for the container.

Solution

Adjust the timeout period for the container.

For example, for a Tomcat container, you can adjust the value of the connectionTimeout parameter in the server.xml file in the path Tomcat installation directory\conf, as shown in the following figure.

The default connection timeout of Tomcat is 20 seconds. You can increase it to 60 seconds to prevent disconnection after long backend processing. Modify the configuration as follows: connectionTimeout="60000".

Large Dataset Export Error: Insufficient Disk Space

Issue description

In FineReport of V11.5.1 and later versions, the error message is displayed in a pop-up window: "An exception occurred during large dataset export."

Click Display Error Stack. The following error message is displayed:

• Before the export: "Since the available disk space is less than xx GB, you cannot start export. Clean the disk or contact the administrator and try again."

• During the export: "Since the available disk space is less than xx GB, the system has terminated the export. Free up enough space and try again."

Cause analysis

The remaining disk space is insufficient, causing the export to fail to start or stop during export.

Solution

Free up the disk space.

Spider Datasets Not Supported by Large Dataset Export

Issue description

You have created a detail table from a Spider dataset of about 4,200,000 records by dragging data columns into cells and setting borders as the only format. The template data can be previewed correctly, but cannot be exported to Excel via the built-in export function. Can the template data be exported via the large dataset export function?

Cause analysis

The large dataset export function only supports datasets created from relational databases. Spider datasets and class datasets are not supported.

Solution

Use a regular dataset created from a relational database.

Progress Bar for Large Dataset Export

Issue description

When a button is configured with a JavaScript event for large dataset export and the data volume is high, the page may appear unresponsive for several seconds after you click the button.

Cause analysis

Although a progress bar is provided during large dataset export, it appears only five seconds after the export begins by default.