chrome浏览器导出80万条数据，浏览器崩溃，Safari可以 #2816

New Issue

jialei-sun · 2022-10-26T09:35:56Z

jialei-sun commented

2022-10-26 09:35:56 +00:00

导出数据量较大100万行，每行21列，每行大概300个字符
版本：0.16.9

尝试1：每20万行追加sheet1中，数据请求完后统一下载
尝试2：每20万行放入1个sheet中，数据请求完后统一下载
2种方式，都是chorme不能完成，Safari可以完成
Chrome：
在writeFile的时候直接崩溃，XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true, type: 'binary'})，其中加opts也不行
Safari：
可以下载成功，{compression: true, type: 'binary'}添加后文件下载的大小相差十几倍

        let o = this
        let sheet = "Sheet"+o._option.sheetNum
 
        XLSX.utils.book_append_sheet(o.wb, XLSX.utils.aoa_to_sheet(o.content), sheet)
        o.content = []

        if (down) {
            XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true,  type: 'binary'})
        }else{
            o._option.sheetNum ++
        }

问题： chrome怎样才能下载成功，什么导致的下载失败

错误：有时报错
RangeError: Invalid string length
at Array.join

	定位到xlsx.full.min.js 中，this.data.join("")位置报错
 有时直接Chrome崩溃

导出数据量较大100万行，每行21列，每行大概300个字符版本：0.16.9 尝试1：每20万行追加sheet1中，数据请求完后统一下载尝试2：每20万行放入1个sheet中，数据请求完后统一下载 2种方式，都是chorme不能完成，Safari可以完成 Chrome：在writeFile的时候直接崩溃，XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true, type: 'binary'})，其中加opts也不行 Safari：可以下载成功，{compression: true, type: 'binary'}添加后文件下载的大小相差十几倍 let o = this let sheet = "Sheet"+o._option.sheetNum XLSX.utils.book_append_sheet(o.wb, XLSX.utils.aoa_to_sheet(o.content), sheet) o.content = [] if (down) { XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true, type: 'binary'}) }else{ o._option.sheetNum ++ } 问题： chrome怎样才能下载成功，什么导致的下载失败错误：有时报错 RangeError: Invalid string length at Array.join 定位到xlsx.full.min.js 中，this.data.join("")位置报错有时直接Chrome崩溃

image.png

24 KiB

image.png

24 KiB

sheetjs commented

2022-10-26 15:22:41 +00:00

https://docs.sheetjs.com/docs/miscellany/errors#aw-snap-or-oops-an-error-has-occurred

Please try upgrading to 0.19.0 and calling writeFile with the option dense: true. This won't resolve the "Invalid string length" issue but it should avoid other crashes in Chrome.

To fix the "Invalid string length" we need to change how the writer works. Currently a large XML string is generated. In Chrome, the string length limit is 0x1FFFFFE8 (536870888) characters, so the writer will need to generate the file without assembling the entire string.

https://docs.sheetjs.com/docs/miscellany/errors#aw-snap-or-oops-an-error-has-occurred Please try upgrading to 0.19.0 and calling `writeFile` with the option `dense: true`. This won't resolve the "Invalid string length" issue but it should avoid other crashes in Chrome. To fix the "Invalid string length" we need to change how the writer works. Currently a large XML string is generated. In Chrome, the string length limit is 0x1FFFFFE8 (536870888) characters, so the writer will need to generate the file without assembling the entire string.

jialei-sun commented

2022-10-28 02:17:40 +00:00

多谢大佬回复。可是目前还有一定几率下载失败

请问有没有办法减少内存使用，80万行数据，每20万1个sheet，内存占用到了1.3G，如下：
XLSX.utils.book_append_sheet(o.wb, XLSX.utils.aoa_to_sheet(o.content), sheet)
但是到调用下载时，内存飙升到3.4G，如下
XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true})

XLSX.writeFile()
XLSX.utils.aoa_to_sheet()
XLSX.utils.book_append_sheet()

上面3个方法，有没有对应的opts来减少内存的占用

多谢大佬回复。可是目前还有一定几率下载失败请问有没有办法减少内存使用，80万行数据，每20万1个sheet，内存占用到了1.3G，如下： XLSX.utils.book_append_sheet(o.wb, XLSX.utils.aoa_to_sheet(o.content), sheet) 但是到调用下载时，内存飙升到3.4G，如下 XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true}) 1. XLSX.writeFile() 2. XLSX.utils.aoa_to_sheet() 3. XLSX.utils.book_append_sheet() 上面3个方法，有没有对应的opts来减少内存的占用

jialei-sun commented

2022-10-28 02:19:30 +00:00

XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true})

这个方法，加了opts {compression: true} 之后，80万行下载后文件大小为110多M，没加这个opts之前，下载80万行数据的文件大小为800多M

XLSX.writeFile(o.wb, o._option.filename + '.xlsx' , {compression: true}) 这个方法，加了opts {compression: true} 之后，80万行下载后文件大小为110多M，没加这个opts之前，下载80万行数据的文件大小为800多M

sheetjs commented

2022-10-28 02:32:41 +00:00

You will need to update to the latest version. https://docs.sheetjs.com/docs/getting-started/#installation has instruction links for standalone script tag and for bundlers like React projects.

Using the latest version, try:

XLSX.utils.book_append_sheet(o.wb, XLSX.utils.aoa_to_sheet(o.content, {dense: true}));
XLSX.writeFile(o.wb, o._option.filename + '.xlsx');

800K rows x 10 columns live demo https://jsfiddle.net/3ksfq7n9/ runs in 15 seconds (Macbook Pro 2018)

Fiddle Code (click to show)

var data = Array.from({length: 800000}, (_,i) => Array.from({length:10}, (_,j) => j == 0 ? "" + i : i + j));
console.time("build");
var ws = XLSX.utils.aoa_to_sheet(data, {dense: true});
var wb = XLSX.utils.book_new();
XLSX.utils.book_append_sheet(wb, ws, "Sheet1");
console.timeEnd("build");
console.time("write");
XLSX.writeFile(wb, "issue2816.xlsx");
console.timeEnd("write");

You will need to update to the latest version. https://docs.sheetjs.com/docs/getting-started/#installation has instruction links for standalone script tag and for bundlers like React projects. Using the latest version, try: ```js XLSX.utils.book_append_sheet(o.wb, XLSX.utils.aoa_to_sheet(o.content, {dense: true})); XLSX.writeFile(o.wb, o._option.filename + '.xlsx'); ``` 800K rows x 10 columns live demo https://jsfiddle.net/3ksfq7n9/ runs in 15 seconds (Macbook Pro 2018) <details><summary><b>Fiddle Code</b> (click to show)</summary> ```js var data = Array.from({length: 800000}, (_,i) => Array.from({length:10}, (_,j) => j == 0 ? "" + i : i + j)); console.time("build"); var ws = XLSX.utils.aoa_to_sheet(data, {dense: true}); var wb = XLSX.utils.book_new(); XLSX.utils.book_append_sheet(wb, ws, "Sheet1"); console.timeEnd("build"); console.time("write"); XLSX.writeFile(wb, "issue2816.xlsx"); console.timeEnd("write"); ``` </details>

abhiram11 referenced this issue

2023-02-26 02:09:11 +00:00

Pull Requests Fail CI/CD Tests #2894

Sign in to join this conversation.