docs.sheetjs.com/docz/docs/03-demos/03-net/01-network.mdx
2023-10-16 22:45:09 -04:00

779 lines
21 KiB
Plaintext

---
title: HTTP Network Requests
pagination_next: demos/net/server/index
---
<head>
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
<script src="https://unpkg.com/superagent@7.1.1/dist/superagent.min.js"></script>
</head>
import current from '/version.js';
import CodeBlock from '@theme/CodeBlock';
`XMLHttpRequest` and `fetch` browser APIs enable binary data transfer between
web browser clients and web servers. Since this library works in web browsers,
server conversion work can be offloaded to the client! This demo shows a few
common scenarios involving browser APIs and popular wrapper libraries.
:::caution Third-Party Hosts and Binary Data
Some services like AWS will corrupt raw binary uploads / downloads by encoding
requests and responses in UTF-8. Typically, these services have options for
disabling this behavior.
For AWS, in the "Binary Media Types" section of the API Gateway console, the
following types should be added to ensure smooth uploads and downloads:
- `"multipart/form-data"` (for Lambda functions to receive files from clients)
- `"application/vnd.ms-excel"` (for Lambda functions to send files to clients)
:::
## Downloading Binary Data
Most interesting spreadsheet files are binary data that contain byte sequences
that represent invalid UTF-8 characters.
The APIs generally have a way to control the interpretation of the downloaded
data. The `arraybuffer` response type usually forces the data to be presented
as a pure `ArrayBuffer` which can be parsed directly with `XLSX.read`.
For example, with `fetch`:
```js
const res = await fetch("https://sheetjs.com/pres.numbers");
const ab = await res.arrayBuffer(); // recover data as ArrayBuffer
const wb = XLSX.read(ab);
```
## Uploading Binary Data
`FormData` objects can hold `File` blobs generated from `XLSX.write`:
```js
/* generate XLSX file bytes */
var data = XLSX.write(workbook, {bookType: 'xlsx', type: 'array'});
/* build FormData with the generated file */
var fdata = new FormData();
fdata.append('data', new File([data], 'sheetjs.xlsx'));
// field name ^^^^ file name ^^^^^^^^^^^^
```
The `FormData` object can be passed along to the POST request. For example:
```js
var req = new XMLHttpRequest();
req.open("POST", "/upload", true);
req.send(fdata);
```
## Browser Demos
The included demos focus on an editable table. There are two separate flows:
- When the page is accessed, the browser will attempt to download <https://sheetjs.com/pres.numbers>
and read the workbook. The old table will be replaced with an editable table
whose contents match the first worksheet. The table is generated using the
`sheet_to_html` utility with `editable:true` option
- When the upload button is clicked, the browser will generate a new worksheet
using `table_to_book` and build up a new workbook. It will then attempt to
generate a file, upload it to <https://s2c.sheetjs.com> and show the response.
### XMLHttpRequest
For downloading data, the `arraybuffer` response type generates an `ArrayBuffer`
that can be viewed as an `Uint8Array` and fed to `XLSX.read` using `array` type:
```js
/* set up an async GET request */
var req = new XMLHttpRequest();
req.open("GET", url, true);
req.responseType = "arraybuffer";
req.onload = function(e) {
/* parse the data when it is received */
var data = new Uint8Array(req.response);
var workbook = XLSX.read(data, {type:"array"});
/* DO SOMETHING WITH workbook HERE */
};
req.send();
```
<details><summary><b>Live Download demo</b> (click to show)</summary>
This demo uses `XMLHttpRequest` to download <https://sheetjs.com/pres.numbers>
and show the data in an HTML table.
```jsx live
function SheetJSXHRDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
/* Fetch file */
const req = new XMLHttpRequest();
req.open("GET", "https://sheetjs.com/pres.numbers", true);
req.responseType = "arraybuffer";
req.onload = e => {
/* Parse file */
const wb = XLSX.read(new Uint8Array(req.response));
const ws = wb.Sheets[wb.SheetNames[0]];
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
};
req.send();
}, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
```
</details>
For uploading data, this demo populates a `FormData` object with an ArrayBuffer
generated with the `array` output type:
```js
/* generate XLSX as array buffer */
var data = XLSX.write(workbook, {bookType: 'xlsx', type: 'array'});
/* build FormData with the generated file */
var fd = new FormData();
fd.append('data', new File([data], 'sheetjs.xlsx'));
/* send data */
var req = new XMLHttpRequest();
req.open("POST", url, true);
req.send(fd);
```
<details><summary><b>Live Upload demo</b> (click to show)</summary>
This demo uses `XMLHttpRequest` to upload data to <https://s2c.sheetjs.com>. It
will parse the workbook and return an HTML table.
```jsx live
function SheetJSXHRUL() {
const [__html, setHTML] = React.useState("");
const [sz, setSz] = React.useState(0);
const csv = "a,b,c\n1,2,3";
/* Fetch and update HTML */
const xport = React.useCallback(async() => {
/* Make Workbook from CSV */
const wb = XLSX.read(csv, { type: "string" });
/* Make FormData */
const data = XLSX.write(wb, {bookType: 'xlsx', type: 'array'});
setSz(data.length || data.byteLength);
const fdata = new FormData();
fdata.append('file', new File([data], 'sheetjs.xlsx'));
/* Upload */
const url = "https://s2c.sheetjs.com";
const req = new XMLHttpRequest();
req.open("POST", url, true);
req.onload = (e) => setHTML(req.responseText);
req.send(fdata);
});
return (<pre>
<b>CSV Data</b>
<div>{csv}</div>
{sz ? ( <>
<b>Generated file size: {sz} bytes</b>
<div dangerouslySetInnerHTML={{ __html }}/>
</> ) : (<button onClick={xport}><b>Export and Upload!</b></button>)}
</pre>);
}
```
</details>
### fetch
For downloading data, `Response#arrayBuffer` resolves to an `ArrayBuffer` that
can be converted to `Uint8Array` and passed to `XLSX.read`:
```js
fetch(url).then(function(res) {
/* get the data as a Blob */
if(!res.ok) throw new Error("fetch failed");
return res.arrayBuffer();
}).then(function(ab) {
/* parse the data when it is received */
var data = new Uint8Array(ab);
var workbook = XLSX.read(data, {type:"array"});
/* DO SOMETHING WITH workbook HERE */
});
```
<details><summary><b>Live Download demo</b> (click to show)</summary>
This demo uses `fetch` to download <https://sheetjs.com/pres.numbers> and show
the data in an HTML table.
```jsx live
function SheetJSFetchDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
/* Fetch file */
const res = await fetch("https://sheetjs.com/pres.numbers");
const ab = await res.arrayBuffer();
/* Parse file */
const wb = XLSX.read(ab);
const ws = wb.Sheets[wb.SheetNames[0]];
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
}, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
```
</details>
`fetch` takes a second parameter which allows for setting POST request body:
```js
// assuming `fdata` is a FormData object from "Uploading Binary Data" section
fetch("/upload", { method: "POST", body: fdata });
```
<details><summary><b>Live Upload demo</b> (click to show)</summary>
This demo uses `fetch` to upload data to <https://s2c.sheetjs.com>. It will parse
the workbook and return an HTML table.
```jsx live
function SheetJSFetchUL() {
const [__html, setHTML] = React.useState("");
const [sz, setSz] = React.useState(0);
const csv = "a,b,c\n1,2,3";
/* Fetch and update HTML */
const xport = React.useCallback(async(e) => {
/* Make Workbook from CSV */
const wb = XLSX.read(csv, { type: "string" });
const data = XLSX.write(wb, {bookType: 'xlsx', type: 'array'});
/* Make FormData */
setSz(data.length || data.byteLength);
const fdata = new FormData();
fdata.append('file', new File([data], 'sheetjs.xlsx'));
/* Upload */
const url = "https://s2c.sheetjs.com";
const res = await fetch(url, {method:"POST", body: fdata});
/* Set HTML */
setHTML((await res.text()));
});
return (<pre>
<b>CSV Data</b>
<div>{csv}</div>
{sz ? ( <>
<b>Generated file size: {sz} bytes</b>
<div dangerouslySetInnerHTML={{ __html }}/>
</> ) : (<button onClick={xport}><b>Export and Upload!</b></button>)}
</pre>);
}
```
</details>
### jQuery
[jQuery](https://jquery.com/) is a JavaScript library that includes helpers for
performing "Ajax" network requests. `jQuery.ajax` (`$.ajax`) does not support
binary data out of the box[^1]. A custom `ajaxTransport` can add support.
SheetJS users have reported success with `jquery.binarytransport.js`[^2] in IE10.
After including the main `jquery.js` and `jquery.binarytransport.js` scripts,
`$.ajax` will support `dataType: "binary"` and `processData: false`.
_Download Files_
**[Live Download Demo](pathname:///jquery/index.html)**
In a GET request, the default behavior is to return a `Blob` object. Passing
`responseType: "arraybuffer"` returns a proper `ArrayBuffer` object in IE10:
```js
$.ajax({
type: "GET", url: "https://sheetjs.com/pres.numbers",
/* suppress jQuery post-processing */
// highlight-next-line
processData: false,
/* use the binary transport */
// highlight-next-line
dataType: "binary",
/* pass an ArrayBuffer in the callback */
// highlight-next-line
responseType: "arraybuffer",
success: function (ab) {
/* at this point, ab is an ArrayBuffer */
// highlight-next-line
var wb = XLSX.read(ab);
/* do something with workbook here */
var ws = wb.Sheets[wb.SheetNames[0]];
var html = XLSX.utils.sheet_to_html(ws);
$("#out").html(html);
}
});
```
### Wrapper Libraries
Before `fetch` shipped with browsers, there were various wrapper libraries to
simplify `XMLHttpRequest`. Due to limitations with `fetch`, these libraries
are still relevant.
#### axios
[`axios`](https://axios-http.com/) presents a Promise based interface. Setting
`responseType` to `arraybuffer` ensures the return type is an ArrayBuffer:
```js
async function workbook_dl_axios(url) {
const res = await axios(url, {responseType:'arraybuffer'});
const workbook = XLSX.read(res.data);
return workbook;
}
```
<details><summary><b>Live Download demo</b> (click to show)</summary>
This demo uses `axios` to download <https://sheetjs.com/pres.numbers> and show
the data in an HTML table.
:::caution pass
If the live demo shows a message
```
ReferenceError: axios is not defined
```
please refresh the page. This is a known bug in the documentation generator.
:::
```jsx live
function SheetJSAxiosDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
if(typeof axios != "function") return setHTML("ReferenceError: axios is not defined");
/* Fetch file */
const res = await axios("https://sheetjs.com/pres.numbers", {responseType: "arraybuffer"});
/* Parse file */
const wb = XLSX.read(res.data);
const ws = wb.Sheets[wb.SheetNames[0]];
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
}, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
```
</details>
Uploading form data is nearly identical to the `fetch` example:
```js
axios("/upload", { method: "POST", data: fdata });
```
<details><summary><b>Live Upload demo</b> (click to show)</summary>
This demo uses `axios` to upload data to <https://s2c.sheetjs.com>. It will parse
the workbook and return an HTML table.
```jsx live
function SheetJSAxiosUL() {
const [__html, setHTML] = React.useState("");
const [sz, setSz] = React.useState(0);
const csv = "a,b,c\n1,2,3";
/* Fetch and update HTML */
const xport = React.useCallback(async() => {
/* Make Workbook from CSV */
const wb = XLSX.read(csv, { type: "string" });
/* Make FormData */
const data = XLSX.write(wb, {bookType: 'xlsx', type: 'array'});
setSz(data.length || data.byteLength);
const fdata = new FormData();
fdata.append('file', new File([data], 'sheetjs.xlsx'));
/* Upload */
const url = "https://s2c.sheetjs.com";
const res = await axios(url, {method:"POST", data: fdata});
/* Set HTML */
setHTML(res.data);
});
return (<pre>
<b>CSV Data</b>
<div>{csv}</div>
{sz ? ( <>
<b>Generated file size: {sz} bytes</b>
<div dangerouslySetInnerHTML={{ __html }}/>
</> ) : (<button onClick={xport}><b>Export and Upload!</b></button>)}
</pre>);
}
```
</details>
#### superagent
[`superagent`](https://github.com/visionmedia/superagent) is a network request
library with a "Fluent Interface"[^3]. Calling the `responseType` method with
`"arraybuffer"` will ensure the final response object is an `ArrayBuffer`:
```js
/* set up an async GET request with superagent */
superagent
.get(url)
.responseType('arraybuffer')
.end(function(err, res) {
/* parse the data when it is received */
var data = new Uint8Array(res.body);
var workbook = XLSX.read(data, {type:"array"});
/* DO SOMETHING WITH workbook HERE */
});
```
<details><summary><b>Live Download demo</b> (click to show)</summary>
This demo uses `superagent` to download <https://sheetjs.com/pres.numbers> and
show the data in an HTML table.
:::caution pass
If the live demo shows a message
```
ReferenceError: superagent is not defined
```
please refresh the page. This is a known bug in the documentation generator.
:::
```jsx live
function SheetJSSuperAgentDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
if(typeof superagent == "undefined" || typeof superagent.get != "function")
return setHTML("ReferenceError: superagent is not defined");
/* Fetch file */
superagent
.get("https://sheetjs.com/pres.numbers")
.responseType("arraybuffer")
.end((err, res) => {
/* Parse file */
const wb = XLSX.read(res.body);
const ws = wb.Sheets[wb.SheetNames[0]];
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
});
}, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
```
</details>
The upload portion only differs in the actual request command:
```js
/* send data (fd is the FormData object) */
superagent.post("/upload").send(fd);
```
<details><summary><b>Live Upload demo</b> (click to show)</summary>
This demo uses `superagent` to upload data to <https://s2c.sheetjs.com>. It will
parse the workbook and return an HTML table.
```jsx live
function SheetJSSuperAgentUL() {
const [__html, setHTML] = React.useState("");
const [sz, setSz] = React.useState(0);
const csv = "a,b,c\n1,2,3";
/* Fetch and update HTML */
const xport = React.useCallback(async() => {
/* Make Workbook from CSV */
const wb = XLSX.read(csv, { type: "string" });
/* Make FormData */
const data = XLSX.write(wb, {bookType: 'xlsx', type: 'array'});
setSz(data.length || data.byteLength);
const fdata = new FormData();
fdata.append('file', new File([data], 'sheetjs.xlsx'));
/* Upload */
const url = "https://s2c.sheetjs.com";
superagent.post(url).send(fdata).end((err, res) => {
/* Set HTML */
setHTML(res.text);
});
});
return (<pre>
<b>CSV Data</b>
<div>{csv}</div>
{sz ? ( <>
<b>Generated file size: {sz} bytes</b>
<div dangerouslySetInnerHTML={{ __html }}/>
</> ) : (<button onClick={xport}><b>Export and Upload!</b></button>)}
</pre>);
}
```
</details>
## NodeJS Demos
These examples show how to download data in NodeJS.
### HTTPS GET
The `https` module provides a low-level `get` method for HTTPS GET requests:
```js title="SheetJSHTTPSGet.js"
var https = require("https"), XLSX = require("xlsx");
https.get('https://sheetjs.com/pres.numbers', function(res) {
var bufs = [];
res.on('data', function(chunk) { bufs.push(chunk); });
res.on('end', function() {
var buf = Buffer.concat(bufs);
var wb = XLSX.read(buf);
/* print the first worksheet to console */
var ws = wb.Sheets[wb.SheetNames[0]];
console.log(XLSX.utils.sheet_to_csv(ws));
});
});
```
<details><summary><b>Complete Example</b> (click to show)</summary>
:::note
This demo was last tested on 2023 August 29 against NodeJS `20.5.1`
:::
1) Install the [NodeJS module](/docs/getting-started/installation/nodejs)
<CodeBlock language="bash">{`\
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`}
</CodeBlock>
2) Copy the `SheetJSHTTPSGet.js` code snippet to a file `SheetJSHTTPSGet.js`
3) Run `node SheetJSHTTPSGet.js`. It will print CSV contents of the test file.
</details>
### fetch
The `fetch` implementation has the same return types as the browser version:
```js
async function parse_from_url(url) {
const res = await fetch(url);
if(!res.ok) throw new Error("fetch failed");
const ab = await res.arrayBuffer();
const workbook = XLSX.read(ab);
return workbook;
}
```
<details><summary><b>Complete Example</b> (click to show)</summary>
:::note
This demo was last tested on 2023 August 29 against NodeJS `20.5.1`
:::
1) Install the [NodeJS module](/docs/getting-started/installation/nodejs)
<CodeBlock language="bash">{`\
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`}
</CodeBlock>
2) Save the following to `SheetJSFetch.js`:
```js title="SheetJSFetch.js"
var XLSX = require("xlsx");
async function parse_from_url(url) {
const res = await fetch(url);
if(!res.ok) throw new Error("fetch failed");
const ab = await res.arrayBuffer();
const workbook = XLSX.read(ab);
return workbook;
}
(async() => {
const wb = await parse_from_url('https://sheetjs.com/pres.numbers');
/* print the first worksheet to console */
var ws = wb.Sheets[wb.SheetNames[0]];
console.log(XLSX.utils.sheet_to_csv(ws));
})();
```
3) Run `node SheetJSFetch.js`. It will print CSV contents of the test file.
</details>
### Wrapper Libraries
The latest releases of NodeJS support `fetch` natively. Before `fetch` support
was added to the platform, third party modules wrapped the native APIs.
#### request
:::warning pass
`request` has been deprecated and should only be used in legacy deployments.
:::
Setting the option `encoding: null` passes raw buffers:
```js title="SheetJSRequest.js"
var XLSX = require('xlsx'), request = require('request');
var url = 'https://sheetjs.com/pres.numbers';
/* call `request` with the option `encoding: null` */
// highlight-next-line
request(url, {encoding: null}, function(err, res, data) {
if(err || res.statusCode !== 200) return;
/* if the request was successful, parse the data */
// highlight-next-line
var wb = XLSX.read(data);
/* print the first worksheet to console */
var ws = wb.Sheets[wb.SheetNames[0]];
console.log(XLSX.utils.sheet_to_csv(ws));
});
```
<details><summary><b>Complete Example</b> (click to show)</summary>
:::note
This demo was last tested on 2023 August 29 against request `2.88.2`
:::
1) Install the [NodeJS module](/docs/getting-started/installation/nodejs)
<CodeBlock language="bash">{`\
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz request@2.88.2`}
</CodeBlock>
2) Copy the `SheetJSRequest.js` code snippet to a file `SheetJSRequest.js`
3) Run `node SheetJSRequest.js`. It will print CSV contents of the test file.
</details>
#### axios
When the `responseType` is `"arraybuffer"`, `axios` actually captures the data
in a NodeJS Buffer. `XLSX.read` will transparently handle Buffers:
```js title="SheetJSAxios.js"
const XLSX = require("xlsx"), axios = require("axios");
async function workbook_dl_axios(url) {
const res = await axios(url, {responseType:'arraybuffer'});
/* at this point, res.data is a Buffer */
const workbook = XLSX.read(res.data);
return workbook;
}
```
<details><summary><b>Complete Example</b> (click to show)</summary>
:::note
This demo was last tested on 2023 August 29 against Axios `1.5.0`
:::
1) Install the [NodeJS module](/docs/getting-started/installation/nodejs)
<CodeBlock language="bash">{`\
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz axios@1.5.0`}
</CodeBlock>
2) Save the following to `SheetJSAxios.js`:
```js title="SheetJSAxios.js"
const XLSX = require("xlsx"), axios = require("axios");
async function workbook_dl_axios(url) {
const res = await axios(url, {responseType:'arraybuffer'});
/* at this point, res.data is a Buffer */
const workbook = XLSX.read(res.data);
return workbook;
}
(async() => {
const wb = await workbook_dl_axios('https://sheetjs.com/pres.numbers');
/* print the first worksheet to console */
var ws = wb.Sheets[wb.SheetNames[0]];
console.log(XLSX.utils.sheet_to_csv(ws));
})();
```
3) Run `node SheetJSAxios.js`. It will print CSV contents of the test file.
</details>
## Other Platforms
Other demos show network operations in special platforms:
- [React Native "Fetching Remote Data"](/docs/demos/mobile/reactnative#fetching-remote-data)
- [NativeScript "Fetching Remote Files"](/docs/demos/mobile/nativescript#fetching-remote-files)
[^1]: See [`dataType` in `jQuery.ajax`](https://api.jquery.com/jQuery.ajax/) in the official jQuery documentation.
[^2]: See [the official `jquery.binarytransport.js` repo](https://github.com/henrya/js-jquery/tree/master/BinaryTransport) for more details.
[^3]: See ["Fluent interface"](https://en.wikipedia.org/wiki/Fluent_interface) on Wikipedia.