docs.sheetjs.com/docz/docs/03-demos/30-cloud/14-cloudflare.md

467 lines
14 KiB
Markdown
Raw Normal View History

2024-04-03 00:12:07 +00:00
---
title: Cloudflare
pagination_prev: demos/local/index
pagination_next: demos/extensions/index
---
import current from '/version.js';
import CodeBlock from '@theme/CodeBlock';
[Cloudflare](https://www.cloudflare.com/) is a cloud services
platform which includes "Serverless
Functions" and cloud storage.
[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing
data from spreadsheets.
This demo explores two key Cloudflare offerings:
- ["Cloudflare Workers"](#cloudflare-workers) explores the serverless
computing offering. The demo creates a JavaScript function that can process
user-submitted files and generate spreadsheets.
- ["Cloudflare R2"](#cloudflare-r2) explores the cloud storage ("R2") offering. The
demo uses the NodeJS connection library to read spreadsheets from R2 and write
spreadsheets to an R2 bucket.
:::note Tested Deployments
This demo was last tested on 2024 April 02.
:::
## Cloudflare Workers
Cloudflare offers NodeJS runtimes for running JavaScript serverless functions.[^1]
The [SheetJS NodeJS module](/docs/getting-started/installation/nodejs) can be
2024-04-03 02:05:56 +00:00
required in Cloudflare Workers. Cloudflare provides a cli called C3 [^2] in order to setup your worker project and another cli called Wrangler [^3]
which can streamline development and deployment,
2024-04-03 00:12:07 +00:00
:::note pass
2024-04-03 00:16:16 +00:00
In this demo, Wrangler is used as that is Cloudflare's recommended way of developing and deploying to
Workers
2024-04-03 00:12:07 +00:00
:::
2024-04-03 04:42:25 +00:00
### Overview
Cloudflare Workers exposes a handler in src/index.(js|ts) when you create a worker project using C3 as per above,
this handler exposes a Request object which can be used to expose the formdata or json of the request to the worker
```ts
export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
const data = await request.formData()
// const data = await request.json() if you are receiving a json request
//...
},
};
2024-04-03 00:12:07 +00:00
```
2024-04-03 04:42:25 +00:00
### Reading Data
2024-04-03 00:12:07 +00:00
2024-04-03 04:42:25 +00:00
The SheetJS `read` method[^4] can read ArrayBuffer objects and generate SheetJS
workbook objects[^5] which can be processed with other API functions.
For example, a handler can use `sheet_to_csv`[^6] to generate CSV text:
```ts
import XLSX from "xlsx";
export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
if (request.method === 'POST') {
const data = await request.formData()
// const data = await request.json() if you are doing a json request
const file = data.get('file') as File;
const buffer = await file.arrayBuffer();
const wb = XLSX.read(buffer);
const ws = wb.Sheets[wb.SheetNames[0]];
var b64 = XLSX.write(wb, { type: "base64", bookType: "xlsx" });
return new Response(XLSX.utils.sheet_to_csv(ws))
}
return new Response('Hello World!');
},
2024-04-03 00:12:07 +00:00
};
```
### Writing Data
2024-04-03 04:42:25 +00:00
```ts
import XLSX from "xlsx";
export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
var wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"});
var data = XLSX.write(wb, { type: "buffer", bookType: "xlsx" });
return new Response(data,{
status: 200,
headers: {
"Content-Disposition": 'attachment; filename="SheetJSCFWorker.xlsx"' //Content disposition will make the browser download the file
}
})
},
2024-04-03 00:12:07 +00:00
};
```
2024-04-03 04:42:25 +00:00
### Cloudflare Workers Demo
2024-04-03 00:12:07 +00:00
:::note pass
2024-04-03 04:42:25 +00:00
At the time of writing, the Cloudflare Workers Free Tier included an allowance of 100,000
free requests per day and 10 milliseconds of CPU time per invocation.
2024-04-03 00:12:07 +00:00
:::
2024-04-03 04:42:25 +00:00
1) If you do not have an account, create a new account[^7].
2024-04-03 00:12:07 +00:00
2024-04-03 04:56:14 +00:00
### Setting up the Project and Testing
2024-04-03 00:12:07 +00:00
2024-04-03 04:56:14 +00:00
[The Cloudflare Docs provide a simple step by step guide to setup, develop, test, build and deploy your project](https://developers.cloudflare.com/workers/get-started/guide/)
2024-04-03 00:12:07 +00:00
## Cloudflare R2
The main NodeJS module for S3 and all Cloudflare services is `Cloudflare-sdk`[^8].
The [SheetJS NodeJS module](/docs/getting-started/installation/nodejs) can be
required in NodeJS scripts.
### Connecting to R2
The `Cloudflare-sdk` module exports a function `S3` that performs the connection. The
function expects an options object that includes an API version and credentials.
Access keys for an IAM user[^9] must be used:
```js
/* credentials */
var accessKeyId = "...", secretAccessKey = "..."";
/* file location */
var Bucket = "...", Key = "pres.numbers";
/* connect to s3 account */
var Cloudflare = require('Cloudflare-sdk');
var s3 = new Cloudflare.S3({
apiVersion: '2006-03-01',
credentials: { accessKeyId, secretAccessKey }
});
```
### Downloading Data
#### Fetching Files from S3
The `s3#getObject` method returns an object with a `createReadStream` method.
`createReadStream` returns a NodeJS stream:
```js
/* open stream to the file */
var stream = s3.getObject({ Bucket: Bucket, Key: Key }).createReadStream();
```
#### Concatenating NodeJS Streams
Buffers can be concatenated from the stream into one unified Buffer object:
```js
/* array of buffers */
var bufs = [];
/* add each data chunk to the array */
stream.on('data', function(data) { bufs.push(data); });
/* the callback will be called after all of the data is collected */
stream.on('end', function() {
/* concatenate */
var buf = Buffer.concat(bufs);
/* AT THIS POINT, `buf` is a NodeJS Buffer */
});
```
#### Parsing NodeJS Buffers
The SheetJS `read` method[^10] can read the final object and generate SheetJS
workbook objects[^11] which can be processed with other API functions.
For example, a callback can use `sheet_to_csv`[^12] to generate CSV text:
```js
stream.on('end', function() {
/* concatenate */
var buf = Buffer.concat(bufs);
/* parse */
var wb = XLSX.read(Buffer.concat(bufs));
/* generate CSV from first worksheet */
var first_ws = wb.Sheets[wb.SheetNames[0]];
var csv = XLSX.utils.sheet_to_csv(first_ws);
console.log(csv);
});
```
### Uploading Data
The SheetJS `write` method[^13] with the option `type: "buffer"` will generate
NodeJS Buffers. `S3#upload` directly accepts these Buffer objects.
This example creates a sample workbook object, generates XLSX file data in a
NodeJS Buffer, and uploads the data to S3:
```js
/* generate sample workbook */
var wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"});
/* write to XLSX file in a NodeJS Buffer */
var Body = XLSX.write(wb, {type: "buffer", bookType: "xlsx"});
/* upload buffer */
s3.upload({ Bucket, Key, Body }, function(err, data) {
if(err) throw err;
console.log("Uploaded to " + data.Location);
});
```
### S3 Demo
:::note pass
At the time of writing, the Cloudflare Free Tier included 5GB of Cloudflare R2 with 20,000
Get requests and 2000 Put requests per month.
:::
This sample fetches a buffer from S3 and parses the workbook.
0) If you do not have an account, create a new Cloudflare free tier account[^14].
#### Create S3 Bucket
1) Sign into the [Cloudflare Management Console](https://Cloudflare.amazon.com/console/) with
a root user account.
2) Type "S3" in the top search box and click S3 (under Services).
3) Open "Buckets" in the left sidebar.
If the left sidebar is not open, click the `≡` icon in the left edge of the page.
4) Click the "Create bucket" button in the main panel.
5) Select the following options:
- Type a memorable "Bucket Name" ("sheetjsbouquet" when last tested)
- In the "Object Ownership" section, select "ACLs disabled"
- Check "Block *all* public access"
- Look for the "Bucket Versioning" section and select "Disable"
6) Click "Create bucket" to create the bucket.
#### Create IAM User
7) Type "IAM" in the top search box and click IAM (under Services).
8) Open "Users" in the left sidebar.
If the left sidebar is not open, click the `≡` icon in the left edge of the page.
9) Click the "Create user" button in the main panel.
10) In step 1, type a memorable "Bucket Name" ("sheetjs-user" when last tested).
Click "Next".
11) In step 2, click "Next"
12) In step 3, click "Create user" to create the user.
#### Add Permissions
13) Click the new user name in the Users table.
14) Select the "Permissions" tab
15) Click the "Add permissions" dropdown and select "Add permissions".
16) Select "Attach policies directly".
17) In the "Permissions policies" section, search for "AmazonS3FullAccess".
There should be one entry.
18) Check the checkbox next to "AmazonS3FullAccess" and click the "Next" button.
19) In the "Review" screen, click "Add permissions"
#### Generate Keys
20) Click "Security credentials", then click "Create access key".
21) Select the "Local code" option. Check "I understand the above recommendation
and want to proceed to create an access key." and click "Next"
22) Click "Create Access Key" and click "Download .csv file" in the next screen.
In the generated CSV:
- Cell A2 is the "Access key ID" (`accessKeyId` in the Cloudflare API)
- Cell B2 is the "Secret access key" (`secretAccessKey` in the Cloudflare API)
#### Set up Project
23) Create a new NodeJS project:
```bash
mkdir SheetJSS3
cd SheetJSS3
npm init -y
```
24) Install dependencies:
<CodeBlock language="bash">{`\
mkdir -p node_modules
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz Cloudflare-sdk@2.1467.0`}
</CodeBlock>
#### Write Test
:::note pass
This sample creates a simple workbook, generates a NodeJS buffer, and uploads
the buffer to S3.
```
| A | B | C | D | E | F | G |
---+---|---|---|---|---|---|---|
1 | S | h | e | e | t | J | S |
2 | 5 | 4 | 3 | 3 | 7 | 9 | 5 |
```
:::
25) Save the following script to `SheetJSWriteToS3.js`:
```js title="SheetJSWriteToS3.js"
var XLSX = require("xlsx");
var Cloudflare = require('Cloudflare-sdk');
/* replace these constants */
// highlight-start
var accessKeyId = "<REPLACE WITH ACCESS KEY ID>";
var secretAccessKey = "<REPLACE WITH SECRET ACCESS KEY>";
var Bucket = "<REPLACE WITH BUCKET NAME>";
// highlight-end
var Key = "test.xlsx";
/* Create a simple workbook and write XLSX to buffer */
var ws = XLSX.utils.aoa_to_sheet(["SheetJS".split(""), [5,4,3,3,7,9,5]]);
var wb = XLSX.utils.book_new(); XLSX.utils.book_append_sheet(wb, ws, "Sheet1");
var Body = XLSX.write(wb, {type: "buffer", bookType: "xlsx"});
/* upload buffer */
var s3 = new Cloudflare.S3({
apiVersion: '2006-03-01',
credentials: {
accessKeyId: accessKeyId,
secretAccessKey: secretAccessKey
}
});
s3.upload({ Bucket: Bucket, Key: Key, Body: Body }, function(err, data) {
if(err) throw err;
console.log("Uploaded to " + data.Location);
});
```
26) Edit `SheetJSWriteToS3.js` and replace the highlighted lines:
- `accessKeyId`: access key for the Cloudflare account
- `secretAccessKey`: secret access key for the Cloudflare account
- `Bucket`: name of the bucket
The keys are found in the CSV from step 22. The Bucket is the name from step 5.
27) Run the script:
```bash
node SheetJSWriteToS3.js
```
This file will be stored with the object name `test.xlsx`. It can be manually
downloaded from the S3 web interface.
#### Read Test
This sample will download and process the test file from "Write Test".
28) Save the following script to `SheetJSReadFromS3.js`:
```js title="SheetJSReadFromS3.js"
var XLSX = require("xlsx");
var Cloudflare = require('Cloudflare-sdk');
/* replace these constants */
// highlight-start
var accessKeyId = "<REPLACE WITH ACCESS KEY ID>";
var secretAccessKey = "<REPLACE WITH SECRET ACCESS KEY>";
var Bucket = "<REPLACE WITH BUCKET NAME>";
// highlight-end
var Key = "test.xlsx";
/* Get stream */
var s3 = new Cloudflare.S3({
apiVersion: '2006-03-01',
credentials: {
accessKeyId: accessKeyId,
secretAccessKey: secretAccessKey
}
});
var f = s3.getObject({ Bucket: Bucket, Key: Key }).createReadStream();
/* collect data */
var bufs = [];
f.on('data', function(data) { bufs.push(data); });
f.on('end', function() {
/* concatenate and parse */
var wb = XLSX.read(Buffer.concat(bufs));
console.log(XLSX.utils.sheet_to_csv(wb.Sheets[wb.SheetNames[0]]));
});
```
29) Edit `SheetJSReadFromS3.js` and replace the highlighted lines:
- `accessKeyId`: access key for the Cloudflare account
- `secretAccessKey`: secret access key for the Cloudflare account
- `Bucket`: name of the bucket
The keys are found in the CSV from Step 22. The Bucket is the name from Step 5.
30) Run the script:
```bash
node SheetJSReadFromS3.js
```
The program will display the data in CSV format.
```
S,h,e,e,t,J,S
5,4,3,3,7,9,5
```
[^1]: See ["Node.js compatibility"](https://developers.cloudflare.com/workers/runtime-apis/nodejs/) in the Cloudflare documentation
2024-04-03 02:05:56 +00:00
[^2]: See ["Get started guide"](https://developers.cloudflare.com/workers/get-started/guide/#1-create-a-new-worker-project)
[^3]: [Wrangler documentation](https://developers.cloudflare.com/workers/wrangler/)
2024-04-03 04:42:25 +00:00
[^4]: See [`read` in "Reading Files"](/docs/api/parse-options)
[^5]: See ["Workbook Object" in "SheetJS Data Model"](/docs/csf/book) for more details.
[^6]: See [`sheet_to_csv` in "CSV and Text"](/docs/api/utilities/csv#delimiter-separated-output)
[^7]: Registering for a free account [on the Cloudflare Free Tier](https://dash.cloudflare.com/sign-up).
2024-04-03 00:12:07 +00:00
[^8]: The `Cloudflare-sdk` module is distributed [on the public NPM registry](https://npm.im/Cloudflare-sdk)
[^9]: See ["Managing access keys for IAM users"](https://docs.Cloudflare.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) in the Cloudflare documentation
[^10]: See [`read` in "Reading Files"](/docs/api/parse-options)
[^11]: See ["Workbook Object" in "SheetJS Data Model"](/docs/csf/book) for more details.
[^12]: See [`sheet_to_csv` in "CSV and Text"](/docs/api/utilities/csv#delimiter-separated-output)
[^13]: See [`write` in "Writing Files"](/docs/api/write-options)
[^14]: Registering for a free account [on the Cloudflare Free Tier](https://Cloudflare.amazon.com/free/) requires a valid phone number and a valid credit card.