docs.sheetjs.com/docz/docs/03-demos/30-cloud/14-cloudflare.md
2024-04-03 18:22:39 +11:00

14 KiB

title pagination_prev pagination_next
Cloudflare demos/local/index demos/extensions/index

import current from '/version.js'; import CodeBlock from '@theme/CodeBlock';

Cloudflare is a cloud services platform which includes "Serverless Functions" and cloud storage.

SheetJS is a JavaScript library for reading and writing data from spreadsheets.

This demo explores two key Cloudflare offerings:

  • "Cloudflare Workers" explores the serverless computing offering. The demo creates a JavaScript function that can process user-submitted files and generate spreadsheets.

  • "Cloudflare R2" explores the cloud storage ("R2") offering. The demo uses the NodeJS connection library to read spreadsheets from R2 and write spreadsheets to an R2 bucket.

:::note Tested Deployments

This demo was last tested on 2024 April 02.

:::

Cloudflare Workers

Cloudflare offers NodeJS runtimes for running JavaScript serverless functions.1

The SheetJS NodeJS module can be required in Cloudflare Workers. Cloudflare provides a cli called C3 2 in order to setup your worker project and another cli called Wrangler 3 which can streamline development and deployment,

:::note pass

In this demo, Wrangler is used as that is Cloudflare's recommended way of developing and deploying to Workers

:::

Overview

Cloudflare Workers exposes a handler in src/index.(js|ts) when you create a worker project using C3 as per above, this handler exposes a Request object which can be used to expose the formdata or json of the request to the worker

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    const data = await request.formData()
    // const data = await request.json() if you are receiving a json request
    //...
  },
};

Reading Data

The SheetJS read method4 can read ArrayBuffer objects and generate SheetJS workbook objects5 which can be processed with other API functions.

For example, a handler can use sheet_to_csv6 to generate CSV text:

import XLSX from "xlsx";
export default {
	async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
		if (request.method === 'POST') {
			const data = await request.formData()
			// const data = await request.json() if you are doing a json request
			const file = data.get('file') as File;
			const buffer = await file.arrayBuffer();
			const wb = XLSX.read(buffer);
			const ws = wb.Sheets[wb.SheetNames[0]];
			var b64 = XLSX.write(wb, { type: "base64", bookType: "xlsx" });
			return new Response(XLSX.utils.sheet_to_csv(ws))
		}
		return new Response('Hello World!');
	},
};

Writing Data

import XLSX from "xlsx";
export default {
	async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
		var wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"});
		var data = XLSX.write(wb, { type: "buffer", bookType: "xlsx" });
		return new Response(data,{ 
			status: 200,
			headers: {
				"Content-Disposition": 'attachment; filename="SheetJSCFWorker.xlsx"' //Content disposition will make the browser download the file
			}
		})
	},
};

Cloudflare Workers Demo

:::note pass

At the time of writing, the Cloudflare Workers Free Tier included an allowance of 100,000 free requests per day and 10 milliseconds of CPU time per invocation.

:::

  1. If you do not have an account, create a new Cloudflare account7.

Setting up the Project and Testing

We will be using the C3 CLI which means we do not have to interact with the Cloudflare dashboard as much.

  1. Create a Worker project with C3 CLI. It will ask you to authenticate into cloudflare. It will also give you some options but for this example we will be using JS,
npm create cloudflare@latest
  1. Install Required Dependencies
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz @aws-sdk/client-s3@3.540.0
  1. Navigate to src/index.js, then replace the default async fetch function there with this
const XLSX = require("xlsx");
export default {
  async fetch(request, env, ctx) {		
    const wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"});
    /* write to XLSX file in a NodeJS Buffer */
    const data = XLSX.write(wb, {type: "buffer", bookType: "xlsx"});
    return new Response(data,{
			headers: {
				"Content-Disposition": 'attachment; filename="SheetJSCFWorker.xlsx"'
			}
		})
  }
}
  1. Start the development server
npx wrangler dev

Then open the browser at the port wrangler opened, it should download a file called SheetJSCFWorker.xlsx with some test data in it

  1. Deploy the worker and configure live enviromnent secrets
npx wrangler deploy
  1. Go to your Cloudflare dashboard, go to Workers & Pages, click on your worker, then go to Settings -> Triggers and then click on your worker route, it should download SheetJSCFWorker.xlsx with the same test data

Cloudflare R2

Due to R2's S3 compatability, the main NodeJS module for Cloudflare R2 is actually the AWS S3 SDK @aws-sdk/client-s38.

The SheetJS NodeJS module can be required in NodeJS scripts.

:::note pass

As the time of writing, Cloudflare R2 is not 100% compatible with the S3 API, please refer to the S3 API Compatibility section of the official Cloudflare R2 Docs to know more about what S3 Features are and are not supported.

:::

Connecting to R2

The @aws-sdk/client-s3 module exports a class S3Client that performs the connection. The function expects an options object that includes a region, credentials and in the case of R2 an endpoint. this client is used to interact with R2.

import { S3Client } from "@aws-sdk/client-s3";
/* credentials */
const accessKeyId = "...", secretAccessKey = "...";

/* file location */
const Bucket = "...", Key = "pres.numbers";

/* connect to s3 account */
const Cloudflare = require('Cloudflare-sdk');
const s3 = new S3Client({
  region: "REGION", //For cloudflare, if the cloudflare region is Auto then you put "auto"
  credentials: {
    accessKeyId,
    secretAccessKey
  },
  endpoint: 'R2-ENDPOINT' //Note that the cloudflare dashboard will include the bucket prefix to the endpoint
  //Make sure the endpoint ends with .r2.cloudflarestorage.com not .r2.cloudflarestorage.com/bucket-name
});

Downloading Data

Fetching Files from R2

The S3 SDK's client mentioned above when instantiated does needs to be fed commands through its .send() method to interact with R2.

import { GetObjectCommand, S3Client } from "@aws-sdk/client-s3";
/*
 ... Client instantiation code from before
 ...
 ...
*/
const command = new GetObjectCommand({
  Bucket: 'bucket-name',
  Key: 'key'
})  //Create the command object, we are using GetObjectCommand to fetch an item from R2
const data = await client.send(command); //Send the command
const bytes = await data.Body?.transformToByteArray(); //After receiving the item, transform it to a byte array
const wb = XLSX.read(bytes); //Use SheetJS as normal
const first_ws = wb.Sheets[wb.SheetNames[0]];
const csv = XLSX.utils.sheet_to_csv(first_ws);

Uploading Data

The SheetJS write method9 with the option type: "buffer" will generate NodeJS Buffers. R2 directly accepts these Buffer objects.

This example creates a sample workbook object, generates XLSX file data in a NodeJS Buffer, and uploads the data to R2:

Here we are using the PutObjectCommand object to upload the data written by write into the R2 bucket

import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
/*
 ... Client instantiation code from before
 ...
 ...
*/
/* generate sample workbook */
var wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"});

/* write to XLSX file in a NodeJS Buffer */
var Body = XLSX.write(wb, {type: "buffer", bookType: "xlsx"});
const command = new PutObjectCommand({
  Bucket: 'bucket-name',
  Key: 'key',
  Body
})
await client.send(command);

R2 Demo

:::note pass

At the time of writing, the Cloudflare Free Tier included 10GB of Cloudflare R2 with 1 million Class A operations and 10 million Class B operations per month

Please visit the official pricing model for R2 for more info

:::

This sample fetches a buffer from R2 and parses the workbook.

  1. If you do not have an account, create a new Cloudflare account 7

  2. Once in the dashboard, click on R2 , you will need a valid credit card to get access to R2

  3. Click on Create Bucket. Select your bucket name and region. Then click "Create Bucket" at the bottom

  4. Click on R2 again on the dashboard, then click Manage R2 API Tokens

  5. Click on Create API Token, then make sure to set the correct permissions for what you want to do, then click "Create API Token" at the bottom

Set up Project

  1. Create a new NodeJS project and install dependencies:
mkdir SheetJSR2 
cd SheetJSR2
npm init -y
# Install Dependencies
mkdir -p node_modules
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz @aws-sdk/client-s3@3.540.0

Write Test

:::note pass

This sample creates a simple workbook, generates a NodeJS buffer, and uploads the buffer to S3.

   | A | B | C | D | E | F | G |
---+---|---|---|---|---|---|---|
 1 | S | h | e | e | t | J | S |
 2 | 5 | 4 | 3 | 3 | 7 | 9 | 5 |

:::

  1. Save the following script to SheetJSWriteToR2.js:
const { PutObjectCommand, S3Client } = require("@aws-sdk/client-s3");
const XLSX = require("xlsx");

/* replace these constants */
// highlight-start
var accessKeyId = "accessKeyId";
var secretAccessKey = "secretAccessKey";
var Bucket = "Bucket";
var region = "REGION";
var endpoint = "ENDPOINT";
// highlight-end

var Key = "sheetjstest.xlsx";

/* Create a simple workbook and write XLSX to buffer */
var ws = XLSX.utils.aoa_to_sheet(["SheetJS".split(""), [5,4,3,3,7,9,5]]);
var wb = XLSX.utils.book_new(); XLSX.utils.book_append_sheet(wb, ws, "Sheet1");
var Body = XLSX.write(wb, {type: "buffer", bookType: "xlsx"});

/* upload to R2 */
var s3 = new S3Client({
  credentials: {
    accessKeyId,
    secretAccessKey
  },
  region,
  endpoint
});

const command = new PutObjectCommand({
    Body,
    Bucket,
    Key
})

s3.send(command).then(data => {
    console.log("Entity Tag: " + data.ETag);
}).catch(e => {
    throw e
});

:::note pass

Although this minimal example is in regular js and therefore cannot use async await syntax at top level, be aware that the S3 SDK is async and thus you can use async await if you wish

:::

  1. Edit SheetJSWriteToR2.js and replace the highlighted lines:
  • accessKeyId: access key for the AWS account
  • secretAccessKey: secret access key for the AWS account
  • Bucket: name of the bucket
  • REGION: region of the bucket
  • ENDPOINT: endpoint of your cloudflare R2, should end with .r2.cloudflarestorage.com , To get this endpoint go to the settings of your bucket and copy the "S3 API" in Bucket details, remove the /bucket-name at the end of the url

The keys are found in the CSV from step 22. The Bucket is the name from step 5.

  1. Run the script:
node SheetJSWriteToR2.js

This file will be stored with the object name test.xlsx. It can be manually downloaded from the S3 web interface.

Read Test

This sample will download and process the test file from "Write Test".

  1. Save the following script to SheetJSReadFromR2.js:
const { GetObjectCommand, S3Client } = require("@aws-sdk/client-s3");
const XLSX = require("xlsx");

/* replace these constants */
// highlight-start
var accessKeyId = "accessKeyId";
var secretAccessKey = "secretAccessKey";
var Bucket = "Bucket";
var region = "REGION";
var endpoint = "ENDPOINT";
// highlight-end

var Key = "sheetjstest.xlsx";

/* upload to R2 */
var s3 = new S3Client({
  credentials: {
    accessKeyId,
    secretAccessKey
  },
  region,
  endpoint
});

const command = new GetObjectCommand({
    Bucket,
    Key
})

s3.send(command).then(async (data) => {
    var wb = XLSX.read(await data.Body?.transformToByteArray());
    console.log(XLSX.utils.sheet_to_csv(wb.Sheets[wb.SheetNames[0]]));
}).catch(e => {
    throw e
});
  1. Edit SheetJSReadFromR2.js and replace the highlighted lines:
  • accessKeyId: access key for the AWS account
  • secretAccessKey: secret access key for the AWS account
  • Bucket: name of the bucket
  • REGION: region of the bucket
  • ENDPOINT: endpoint of your cloudflare R2, should end with .r2.cloudflarestorage.com , To get this endpoint go to the settings of your bucket and copy the "S3 API" in Bucket details, remove the /bucket-name at the end of the url

The keys are found in the CSV from Step 22. The Bucket is the name from Step 5.

  1. Run the script:
node SheetJSReadFromR2.js

The program will display the data in CSV format.

S,h,e,e,t,J,S
5,4,3,3,7,9,5