diff --git a/docz/docs/03-demos/30-cloud/14-cloudflare.md b/docz/docs/03-demos/30-cloud/14-cloudflare.md new file mode 100644 index 0000000..ce0d5a4 --- /dev/null +++ b/docz/docs/03-demos/30-cloud/14-cloudflare.md @@ -0,0 +1,465 @@ +--- +title: Cloudflare +pagination_prev: demos/local/index +pagination_next: demos/extensions/index +--- + +import current from '/version.js'; +import CodeBlock from '@theme/CodeBlock'; + +[Cloudflare](https://www.cloudflare.com/) is a cloud services +platform which includes "Serverless +Functions" and cloud storage. + +[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing +data from spreadsheets. + +This demo explores two key Cloudflare offerings: + +- ["Cloudflare Workers"](#cloudflare-workers) explores the serverless + computing offering. The demo creates a JavaScript function that can process + user-submitted files and generate spreadsheets. + +- ["Cloudflare R2"](#cloudflare-r2) explores the cloud storage ("R2") offering. The + demo uses the NodeJS connection library to read spreadsheets from R2 and write + spreadsheets to an R2 bucket. + +:::note Tested Deployments + +This demo was last tested on 2024 April 02. + +::: + +## Cloudflare Workers + +Cloudflare offers NodeJS runtimes for running JavaScript serverless functions.[^1] + +The [SheetJS NodeJS module](/docs/getting-started/installation/nodejs) can be +required in Cloudflare Workers. Cloudflare provides a cli called C3 [^2] in order to setup your worker project and another cli called Wrangler [^3] +which can streamline development and deployment, + +:::note pass + +In this demo, Wrangler is used as that is Cloudflare's recommended way of developing and deploying to +Workers + +::: + +### Overview + +Cloudflare Workers exposes a handler in src/index.(js|ts) when you create a worker project using C3 as per above, +this handler exposes a Request object which can be used to expose the formdata or json of the request to the worker + +```ts +export default { + async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise { + const data = await request.formData() + // const data = await request.json() if you are receiving a json request + //... + }, +}; +``` + +### Reading Data + +The SheetJS `read` method[^4] can read ArrayBuffer objects and generate SheetJS +workbook objects[^5] which can be processed with other API functions. + +For example, a handler can use `sheet_to_csv`[^6] to generate CSV text: + +```ts +import XLSX from "xlsx"; +export default { + async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise { + if (request.method === 'POST') { + const data = await request.formData() + // const data = await request.json() if you are doing a json request + const file = data.get('file') as File; + const buffer = await file.arrayBuffer(); + const wb = XLSX.read(buffer); + const ws = wb.Sheets[wb.SheetNames[0]]; + var b64 = XLSX.write(wb, { type: "base64", bookType: "xlsx" }); + return new Response(XLSX.utils.sheet_to_csv(ws)) + } + return new Response('Hello World!'); + }, +}; +``` + +### Writing Data + +```ts +import XLSX from "xlsx"; +export default { + async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise { + var wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"}); + var data = XLSX.write(wb, { type: "buffer", bookType: "xlsx" }); + return new Response(data,{ + status: 200, + headers: { + "Content-Disposition": 'attachment; filename="SheetJSCFWorker.xlsx"' //Content disposition will make the browser download the file + } + }) + }, +}; +``` + +### Cloudflare Workers Demo + +:::note pass + +At the time of writing, the Cloudflare Workers Free Tier included an allowance of 100,000 +free requests per day and 10 milliseconds of CPU time per invocation. + +::: + +1) If you do not have an account, create a new Cloudflare account[^7]. + +#### Setting up the Project and Testing + +We will be using the C3 CLI which means we do not have to interact with the Cloudflare dashboard as much. + +2) Create a Worker project with C3 CLI. It will ask you to authenticate into cloudflare. It will also give you some options but for this example we will be using JS, + +```bash +npm create cloudflare@latest +``` + +3) Install Required Dependencies + +```bash +npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz @aws-sdk/client-s3@3.540.0 +``` + + +4) Navigate to src/index.js, then replace the default `async fetch` function there with this + +```js +const XLSX = require("xlsx"); +export default { + async fetch(request, env, ctx) { + const wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"}); + /* write to XLSX file in a NodeJS Buffer */ + const data = XLSX.write(wb, {type: "buffer", bookType: "xlsx"}); + return new Response(data,{ + headers: { + "Content-Disposition": 'attachment; filename="SheetJSCFWorker.xlsx"' + } + }) + } +} +``` + +5) Start the development server + +```bash +npx wrangler dev +``` +Then open the browser at the port wrangler opened, it should download a file called SheetJSCFWorker.xlsx with some test data in it + +6) Deploy the worker and configure live enviromnent secrets + +```bash +npx wrangler deploy +``` + +7) Go to your Cloudflare dashboard, go to Workers & Pages, click on your worker, then go to Settings -> Triggers and then click on your worker route, +it should download SheetJSCFWorker.xlsx with the same test data + +## Cloudflare R2 + +Due to R2's S3 compatability, the main NodeJS module for Cloudflare R2 is actually the AWS S3 SDK `@aws-sdk/client-s3`[^8]. + +The [SheetJS NodeJS module](/docs/getting-started/installation/nodejs) can be +required in NodeJS scripts. + +:::note pass + +As the time of writing, Cloudflare R2 is not 100% compatible with the S3 API, please refer to the [S3 API Compatibility section of the official Cloudflare R2 Docs](https://developers.cloudflare.com/r2/api/s3/api/) to know more about what S3 Features are and are not supported. + +::: + +### Connecting to R2 + +The `@aws-sdk/client-s3` module exports a class `S3Client` that performs the connection. The +function expects an options object that includes a region, credentials and in the case of R2 an endpoint. +this client is used to interact with R2. + +```ts +import { S3Client } from "@aws-sdk/client-s3"; +/* credentials */ +const accessKeyId = "...", secretAccessKey = "..."; + +/* file location */ +const Bucket = "...", Key = "pres.numbers"; + +/* connect to s3 account */ +const Cloudflare = require('Cloudflare-sdk'); +const s3 = new S3Client({ + region: "REGION", //For cloudflare, if the cloudflare region is Auto then you put "auto" + credentials: { + accessKeyId, + secretAccessKey + }, + endpoint: 'R2-ENDPOINT' //Note that the cloudflare dashboard will include the bucket prefix to the endpoint + //Make sure the endpoint ends with .r2.cloudflarestorage.com not .r2.cloudflarestorage.com/bucket-name +}); +``` + +### Downloading Data + +#### Fetching Files from R2 + +The S3 SDK's client mentioned above when instantiated does needs to be fed commands through its ```.send()``` method +to interact with R2. + +```ts +import { GetObjectCommand, S3Client } from "@aws-sdk/client-s3"; +/* + ... Client instantiation code from before + ... + ... +*/ +const command = new GetObjectCommand({ + Bucket: 'bucket-name', + Key: 'key' +}) //Create the command object, we are using GetObjectCommand to fetch an item from R2 +const data = await client.send(command); //Send the command +const bytes = await data.Body?.transformToByteArray(); //After receiving the item, transform it to a byte array +const wb = XLSX.read(bytes); //Use SheetJS as normal +const first_ws = wb.Sheets[wb.SheetNames[0]]; +const csv = XLSX.utils.sheet_to_csv(first_ws); +``` + +### Uploading Data + +The SheetJS `write` method[^9] with the option `type: "buffer"` will generate +NodeJS Buffers. `R2` directly accepts these Buffer objects. + +This example creates a sample workbook object, generates XLSX file data in a +NodeJS Buffer, and uploads the data to R2: + +Here we are using the PutObjectCommand object to upload the data written by +`write` into the R2 bucket + +```js +import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3"; +/* + ... Client instantiation code from before + ... + ... +*/ +/* generate sample workbook */ +var wb = XLSX.read("S,h,e,e,t,J,S\n5,4,3,3,7,9,5", {type: "binary"}); + +/* write to XLSX file in a NodeJS Buffer */ +var Body = XLSX.write(wb, {type: "buffer", bookType: "xlsx"}); +const command = new PutObjectCommand({ + Bucket: 'bucket-name', + Key: 'key', + Body +}) +await client.send(command); +``` + +### R2 Demo + +:::note pass + +At the time of writing, the Cloudflare Free Tier included 10GB of Cloudflare R2 with 1 million Class A operations +and 10 million Class B operations per month + +Please visit the [official pricing model for R2 for more info](https://developers.cloudflare.com/r2/pricing/) + +::: + +This sample fetches a buffer from R2 and parses the workbook. + +1) If you do not have an account, create a new Cloudflare account [^7] + +2) Once in the dashboard, click on R2 , you will need a valid credit card to get access to R2 + +3) Click on Create Bucket. Select your bucket name and region. Then click "Create Bucket" at the bottom + +4) Click on R2 again on the dashboard, then click Manage R2 API Tokens + +5) Click on Create API Token, then make sure to set the correct permissions for what you want to do, then click "Create API Token" at the bottom + +#### Set up Project + +6) Create a new NodeJS project and install dependencies: + +```bash +mkdir SheetJSR2 +cd SheetJSR2 +npm init -y +# Install Dependencies +mkdir -p node_modules +npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz @aws-sdk/client-s3@3.540.0 +``` + +#### Write Test + +:::note pass + +This sample creates a simple workbook, generates a NodeJS buffer, and uploads +the buffer to S3. + +``` + | A | B | C | D | E | F | G | +---+---|---|---|---|---|---|---| + 1 | S | h | e | e | t | J | S | + 2 | 5 | 4 | 3 | 3 | 7 | 9 | 5 | +``` + +::: + +7) Save the following script to `SheetJSWriteToR2.js`: + +```js title="SheetJSWriteToR2.js" +const { PutObjectCommand, S3Client } = require("@aws-sdk/client-s3"); +const XLSX = require("xlsx"); + +/* replace these constants */ +// highlight-start +var accessKeyId = "accessKeyId"; +var secretAccessKey = "secretAccessKey"; +var Bucket = "Bucket"; +var region = "REGION"; +var endpoint = "ENDPOINT"; +// highlight-end + +var Key = "sheetjstest.xlsx"; + +/* Create a simple workbook and write XLSX to buffer */ +var ws = XLSX.utils.aoa_to_sheet(["SheetJS".split(""), [5,4,3,3,7,9,5]]); +var wb = XLSX.utils.book_new(); XLSX.utils.book_append_sheet(wb, ws, "Sheet1"); +var Body = XLSX.write(wb, {type: "buffer", bookType: "xlsx"}); + +/* upload to R2 */ +var s3 = new S3Client({ + credentials: { + accessKeyId, + secretAccessKey + }, + region, + endpoint +}); + +const command = new PutObjectCommand({ + Body, + Bucket, + Key +}) + +s3.send(command).then(data => { + console.log("Entity Tag: " + data.ETag); +}).catch(e => { + throw e +}); +``` + +:::note pass + +Although this minimal example is in regular js and therefore cannot use async await syntax at top level, be aware that the S3 SDK is async +and thus you can use async await if you wish + +::: + +1) Edit `SheetJSWriteToR2.js` and replace the highlighted lines: + +- `accessKeyId`: access key for the AWS account +- `secretAccessKey`: secret access key for the AWS account +- `Bucket`: name of the bucket +- `REGION`: region of the bucket +- `ENDPOINT`: endpoint of your cloudflare R2, should end with .r2.cloudflarestorage.com , + To get this endpoint go to the settings of your bucket and copy the "S3 API" in Bucket details, remove the /bucket-name at the end of the url + +The keys are found in the CSV from step 22. The Bucket is the name from step 5. + +1) Run the script: + +```bash +node SheetJSWriteToR2.js +``` + +This file will be stored with the object name `test.xlsx`. It can be manually +downloaded from the S3 web interface. + +#### Read Test + +This sample will download and process the test file from "Write Test". + +10) Save the following script to `SheetJSReadFromR2.js`: + +```js title="SheetJSReadFromR2.js" +const { GetObjectCommand, S3Client } = require("@aws-sdk/client-s3"); +const XLSX = require("xlsx"); + +/* replace these constants */ +// highlight-start +var accessKeyId = "accessKeyId"; +var secretAccessKey = "secretAccessKey"; +var Bucket = "Bucket"; +var region = "REGION"; +var endpoint = "ENDPOINT"; +// highlight-end + +var Key = "sheetjstest.xlsx"; + +/* upload to R2 */ +var s3 = new S3Client({ + credentials: { + accessKeyId, + secretAccessKey + }, + region, + endpoint +}); + +const command = new GetObjectCommand({ + Bucket, + Key +}) + +s3.send(command).then(async (data) => { + var wb = XLSX.read(await data.Body?.transformToByteArray()); + console.log(XLSX.utils.sheet_to_csv(wb.Sheets[wb.SheetNames[0]])); +}).catch(e => { + throw e +}); +``` + +1) Edit `SheetJSReadFromR2.js` and replace the highlighted lines: + +- `accessKeyId`: access key for the AWS account +- `secretAccessKey`: secret access key for the AWS account +- `Bucket`: name of the bucket +- `REGION`: region of the bucket +- `ENDPOINT`: endpoint of your cloudflare R2, should end with .r2.cloudflarestorage.com , + To get this endpoint go to the settings of your bucket and copy the "S3 API" in Bucket details, remove the /bucket-name at the end of the url + +The keys are found in the CSV from Step 22. The Bucket is the name from Step 5. + +12) Run the script: + +```bash +node SheetJSReadFromR2.js +``` + +The program will display the data in CSV format. + +``` +S,h,e,e,t,J,S +5,4,3,3,7,9,5 +``` + +[^1]: See ["Node.js compatibility"](https://developers.cloudflare.com/workers/runtime-apis/nodejs/) in the Cloudflare documentation +[^2]: See ["Get started guide"](https://developers.cloudflare.com/workers/get-started/guide/#1-create-a-new-worker-project) +[^3]: [Wrangler documentation](https://developers.cloudflare.com/workers/wrangler/) +[^4]: See [`read` in "Reading Files"](/docs/api/parse-options) +[^5]: See ["Workbook Object" in "SheetJS Data Model"](/docs/csf/book) for more details. +[^6]: See [`sheet_to_csv` in "CSV and Text"](/docs/api/utilities/csv#delimiter-separated-output) +[^7]: Registering for a free account [on the Cloudflare Free Tier](https://dash.cloudflare.com/sign-up). +[^8]: The `@aws-sdk/client-s3` module is distributed [on the public NPM registry](https://www.npmjs.com/package/@aws-sdk/client-s3) +[^9]: See [`write` in "Writing Files"](/docs/api/write-options) \ No newline at end of file