From 3461ce7537cfb0021eeaf9490916a6ac6eb8158f Mon Sep 17 00:00:00 2001 From: SheetJS Date: Mon, 24 Oct 2022 12:49:28 -0400 Subject: [PATCH] dense --- docz/docs/07-csf/03-sheet.md | 42 ++++++++++++++++++++++++++++ docz/docs/09-miscellany/02-errors.md | 20 +++++++++++++ 2 files changed, 62 insertions(+) diff --git a/docz/docs/07-csf/03-sheet.md b/docz/docs/07-csf/03-sheet.md index b6f1845..fc203fd 100644 --- a/docz/docs/07-csf/03-sheet.md +++ b/docz/docs/07-csf/03-sheet.md @@ -20,6 +20,8 @@ Generic sheets are plain JavaScript objects. Each key that does not start with By default, the parsers and utility functions generate "sparse-mode" worksheet objects. `sheet[address]` returns the cell object for the specified address. +#### Dense Mode + When the option `dense: true` is passed, parsers will generate a "dense-mode" worksheet where cells are stored in an array of arrays. `sheet["!data"][R][C]` returns the cell object at row `R` and column `C` (zero-indexed values). @@ -28,6 +30,46 @@ When processing small worksheets in older environments, sparse worksheets are more efficient than dense worksheets. In newer browsers, when dealing with very large worksheets, dense sheets use less memory and tend to be more efficient. +
Migrating to Dense Mode (click to show) + +`read`, `readFile`, `write`, `writeFile`, and the various API functions support +sparse and dense worksheets. Functions that accept worksheet or workbook objects +(e.g. `writeFile` and `sheet_to_json`) will detect dense sheets. + +The option `dense: true` should be used when creating worksheet or book objects: + +```diff +-var workbook = XLSX.read(data, {...opts}); ++var workbook = XLSX.read(data, {...opts, dense: true}); + +-var sheet = XLSX.utils.aoa_to_sheet([[1,2,3],[4,5,6]], {...opts}); ++var sheet = XLSX.utils.aoa_to_sheet([[1,2,3],[4,5,6]], {...opts, dense: true}); + +-var sheet = XLSX.utils.json_to_sheet([{x:1,y:2}], {...opts}); ++var sheet = XLSX.utils.json_to_sheet([{x:1,y:2}], {...opts, dense: true}); +``` + +Code that manually loops over worksheet objects should test for `"!data"` key: + +```js +const { decode_range, encode_cell } = XLSX.utils; + +function log_all_cells(ws) { + var range = decode_range(ws["!ref"]); + // highlight-next-line + var dense = ws["!data"] != null; // test if sheet is dense + for(var R = 0; R <= range.e.r; ++R) { + for(var C = 0; C <= range.e.c; ++C) { + // highlight-next-line + var cell = dense ? ws["!data"]?.[R]?.[C] : ws[encode_cell({r:R, c:C})]; + console.log(R, C, cell); + } + } +} +``` + +
+ ### Sheet Properties Each key starts with `!`. The properties are accessible as `sheet[key]`. diff --git a/docz/docs/09-miscellany/02-errors.md b/docz/docs/09-miscellany/02-errors.md index 8afabc5..279af11 100644 --- a/docz/docs/09-miscellany/02-errors.md +++ b/docz/docs/09-miscellany/02-errors.md @@ -29,6 +29,26 @@ workarounds and solutions! Browsers have strict memory limits and large spreadsheets can exceed the limits. +
Technical Limitations (click to show) + +V8 (Node/Chrome) have a maximum string length that has changed over the years. +Node 16 and Chrome 106 enforce a limit of 536870888 characters. This issue will +manifest with error messages such as `Invalid string length`. + +There are memory bottlenecks associated with string addresses. A number of bugs +have been reported to the V8 and Chromium projects on this subject. While those +bugs are being resolved, for sheets containing >100K rows, dense mode worksheets +should be used. + +
+ +The API functions support [dense mode](../csf/sheet#dense-mode): + +```js +var wb = XLSX.read(data, {dense: true}); // creates a dense-mode sheet +XLSX.writeFile(data, "large.xlsx"); // writeFile can handle dense-mode sheets +``` + When processing very large files is a must, consider running processes in the server with NodeJS or some other server-side technology.