2017-06-10 01:47:42 +00:00
|
|
|
### Streaming Read
|
|
|
|
|
|
|
|
<details>
|
2017-09-24 23:40:09 +00:00
|
|
|
<summary><b>Why is there no Streaming Read API?</b> (click to show)</summary>
|
2017-04-16 04:32:13 +00:00
|
|
|
|
|
|
|
The most common and interesting formats (XLS, XLSX/M, XLSB, ODS) are ultimately
|
|
|
|
ZIP or CFB containers of files. Neither format puts the directory structure at
|
|
|
|
the beginning of the file: ZIP files place the Central Directory records at the
|
2017-08-01 05:50:53 +00:00
|
|
|
end of the logical file, while CFB files can place the storage info anywhere in
|
2017-04-16 04:32:13 +00:00
|
|
|
the file! As a result, to properly handle these formats, a streaming function
|
|
|
|
would have to buffer the entire file before commencing. That belies the
|
2017-06-10 01:47:42 +00:00
|
|
|
expectations of streaming, so we do not provide any streaming read API.
|
|
|
|
|
|
|
|
</details>
|
|
|
|
|
|
|
|
When dealing with Readable Streams, the easiest approach is to buffer the stream
|
|
|
|
and process the whole thing at the end. This can be done with a temporary file
|
|
|
|
or by explicitly concatenating the stream:
|
|
|
|
|
|
|
|
<details>
|
2017-09-24 23:40:09 +00:00
|
|
|
<summary><b>Explicitly concatenating streams</b> (click to show)</summary>
|
2017-06-10 01:47:42 +00:00
|
|
|
|
|
|
|
```js
|
|
|
|
var fs = require('fs');
|
|
|
|
var XLSX = require('xlsx');
|
|
|
|
function process_RS(stream/*:ReadStream*/, cb/*:(wb:Workbook)=>void*/)/*:void*/{
|
2017-09-24 23:40:09 +00:00
|
|
|
var buffers = [];
|
|
|
|
stream.on('data', function(data) { buffers.push(data); });
|
|
|
|
stream.on('end', function() {
|
|
|
|
var buffer = Buffer.concat(buffers);
|
|
|
|
var workbook = XLSX.read(buffer, {type:"buffer"});
|
2017-06-10 01:47:42 +00:00
|
|
|
|
2017-09-24 23:40:09 +00:00
|
|
|
/* DO SOMETHING WITH workbook IN THE CALLBACK */
|
|
|
|
cb(workbook);
|
|
|
|
});
|
2017-06-10 01:47:42 +00:00
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
More robust solutions are available using modules like `concat-stream`.
|
|
|
|
|
|
|
|
</details>
|
|
|
|
|
|
|
|
<details>
|
2017-09-24 23:40:09 +00:00
|
|
|
<summary><b>Writing to filesystem first</b> (click to show)</summary>
|
2017-06-10 01:47:42 +00:00
|
|
|
|
2017-09-24 23:40:09 +00:00
|
|
|
This example uses [`tempfile`](https://npm.im/tempfile) to generate file names:
|
2017-06-10 01:47:42 +00:00
|
|
|
|
|
|
|
```js
|
|
|
|
var fs = require('fs'), tempfile = require('tempfile');
|
|
|
|
var XLSX = require('xlsx');
|
|
|
|
function process_RS(stream/*:ReadStream*/, cb/*:(wb:Workbook)=>void*/)/*:void*/{
|
2017-09-24 23:40:09 +00:00
|
|
|
var fname = tempfile('.sheetjs');
|
|
|
|
console.log(fname);
|
|
|
|
var ostream = fs.createWriteStream(fname);
|
|
|
|
stream.pipe(ostream);
|
|
|
|
ostream.on('finish', function() {
|
|
|
|
var workbook = XLSX.readFile(fname);
|
|
|
|
fs.unlinkSync(fname);
|
2017-06-10 01:47:42 +00:00
|
|
|
|
2017-09-24 23:40:09 +00:00
|
|
|
/* DO SOMETHING WITH workbook IN THE CALLBACK */
|
|
|
|
cb(workbook);
|
|
|
|
});
|
2017-06-10 01:47:42 +00:00
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
</details>
|
2017-04-16 04:32:13 +00:00
|
|
|
|