LankyMoose/docs.sheetjs.com
1
Fork 0
forked from sheetjs/docs.sheetjs.com
Code Pull Requests Activity
b571592055
docs.sheetjs.com/docz/docs/03-demos/30-cloud/12-azure.md

18 KiB
Raw Blame History

title pagination_prev pagination_next
Azure Cloud Services demos/local/index demos/extensions/index

import current from '/version.js'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock';

Azure Cloud Services is a cloud services platform which includes traditional virtual machine support, "Serverless Functions" and cloud storage.

SheetJS is a JavaScript library for reading and writing data from spreadsheets.

This demo explores two key AWS offerings:

  • "Azure Functions" ("Lambda") explores the serverless computing offering. The demo creates a JavaScript function that can process user-submitted files and generate spreadsheets.

  • "Blob Storage" explores the cloud storage offering. The demo uses the NodeJS connection library to read spreadsheets from storage and write spreadsheets back to cloud storage.

:::caution pass

Azure iterates quickly and there is no guarantee that the referenced services will be available in the future.

:::

:::note Tested Deployments

This demo was last tested on 2023 October 06.

:::

Telemetry

:::danger Telemetry

Each command-line tool related to Azure embeds telemetry.

Azure tools embed telemetry without proper disclaimer.

:::

It is strongly recommended to disable telemetry before working with Azure.

Azure Functions Core Tools

Azure Functions Core Tools (func) telemetry is controlled through the FUNCTIONS_CORE_TOOLS_TELEMETRY_OPTOUT environment variable.

Add the following line to .profile, .bashrc and .zshrc:

export FUNCTIONS_CORE_TOOLS_TELEMETRY_OPTOUT=1

Close and restart the Terminal to load the changes.

Type env in the search bar and select "Edit the system environment variables".

In the new window, click the "Environment Variables..." button.

In the new window, look for the "System variables" section and click "New..."

Set the "Variable name" to FUNCTIONS_CORE_TOOLS_TELEMETRY_OPTOUT and the value to 1.

Click "OK" in each window (3 windows) and restart your computer.

Azure CLI

Azure CLI (az) telemetry can be disabled using a subcommand (after installing the CLI tool)1:

az configure -d collect_telemetry=false

Azure Functions

The SheetJS NodeJS module can be required in Azure Functions that use the NodeJS runtime.

This discussion focuses on the "HTTP Trigger" function type.

:::note pass

In earlier tests, to enable binary data processing, function.json required a dataType option:

{
  "bindings": [
    {
      "type": "httpTrigger",
      "direction": "in",
//highlight-next-line
      "dataType": "binary",
      "name": "req",

In the most recent test, the template did not create a function.json and the option was not required.

:::

Reading Data

Using @azure/functions, the handler callback receives a Request object. Uploaded files can be pulled into ArrayBuffer objects.

Code Snippet (click to show)

This function returns a promise that resolves to an ArrayBuffer object:

const { Blob } = require('buffer');

async function get_file_from_request(request, form_field_name) {
  /* parse the request body */
  const formData = await request.formData();

  /* pull the specified field */
  const file = formData.get(form_field_name);

  /* if a file was submitted, `file` will be a Blob */
  if(!(file instanceof Blob)) throw new Error(`File is missing!`);

  /* pull data into an ArrayBuffer object */
  const ab = await file.arrayBuffer();
  return ab;
}

The SheetJS read method2 can read the ArrayBuffer objects and generate SheetJS workbook objects3 which can be processed with other API functions.

For example, a handler can use sheet_to_csv4 to generate CSV text from user-submitted spreadsheets:

const { Blob } = require('buffer');
const { app } = require('@azure/functions');
const XLSX = require('xlsx');

app.http('SheetJSAzure', {
  methods: ['POST'],
  handler: async (req, context) => {
    /* grab the file at form key `upload` */
    const formData = await req.formData();
    const f = formData.get("upload");

    /* if a file was submitted, `f` will be a Blob object */
    if(!(f instanceof Blob)) return { status: 400, body: "Must submit a file" };

    /* parse file */
    const ab = await f.arrayBuffer();
    const wb = XLSX.read(ab);

    /* generate CSV from first sheet */
    const ws = wb.Sheets[wb.SheetNames[0]];
    const csv = XLSX.utils.sheet_to_csv(ws);
    return { status: 200, body: csv };
  }
});

Writing Data

The SheetJS write method5 with the option type: "buffer" will generate NodeJS buffers which can be sent in the callback handler response.

The following example generates a sample worksheet using the aoa_to_sheet6 method, generates a sample workbook using worksheet helper methods7, writes the workbook to XLSX format in a Buffer, and sends the Buffer in the response:

const { app } = require('@azure/functions');
const XLSX = require('xlsx');

app.http('SheetJSAzure', {
  methods: ['GET'],
  handler: async (req, context) => {
    /* generate sample worksheet */
    var ws = XLSX.utils.aoa_to_sheet(["SheetJS".split(""), [5, 4, 3, 3, 7, 9, 5]]);
    /* generate workbook */
    var wb = XLSX.utils.book_new(); XLSX.utils.book_append_sheet(wb, ws, "Data");
    /* write to XLSX, returning a NodeJS Buffer */
    var buf = XLSX.write(wb, { type: "buffer", bookType: "xlsx" });
    /* send Buffer to client */
    return {
      status: 200,
      /* Content-Disposition header */
      headers: { "Content-Disposition": `attachment; filename="SheetJSAzure.xlsx";` },
      /* data */
      body: buf
    };
  }
});

Functions Demo

:::note pass

At the time of writing, the Azure Free Tier included an allowance of 1 million free requests per month.

:::

  1. If you do not have an account, create a new Azure free tier account8.

Local Setup

  1. Disable Azure Functions Core Tools Telemetry.

  2. Install the CLI tool using npm:

npm i -g azure-functions-core-tools@4 --unsafe-perm true

:::note pass

On macOS and Linux, sudo may be required:

sudo npm i -g azure-functions-core-tools@4 --unsafe-perm true

:::

  1. Install Azure CLI

  2. Disable Azure CLI telemetry:

az configure -d collect_telemetry=false

Start Project

  1. Create a new JavaScript HTTP Trigger project:
mkdir SheetJSAzure
cd SheetJSAzure
func new --template httpTrigger --language JavaScript --name SheetJSAzure

:::danger pass

When the demo was last tested, the stock TypeScript template did not work.

This is a bug in the Azure Functions Core Tools

Until the bugs are resolved, JavaScript should be preferred over TypeScript.

:::

  1. Start the local server:
npm start
  1. While the server is running, open a new terminal window and make a request:
curl -L http://localhost:7071/api/SheetJSAzure

The terminal should display Hello, world!

Add SheetJS

  1. Install the SheetJS NodeJS module:

{\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz}

  1. Download the sample script:
curl -L -o src/functions/SheetJSAzure.js https://docs.sheetjs.com/azure/index.js

Local Test

  1. Stop and restart the dev server:
npm start
  1. In a separate terminal window, download https://sheetjs.com/pres.numbers and make a POST request to the dev server:
curl -LO https://sheetjs.com/pres.numbers
curl -X POST -F "upload=@pres.numbers" http://localhost:7071/api/SheetJSAzure

If the test succeeded, the terminal will print CSV rows from the test file data.

  1. Open a web browser and access http://localhost:7071/api/SheetJSAzure .

If the test succeeded, the browser will attempt to download SheetJSAzure.xlsx. Open in Excel or another spreadsheet editor to confirm the file is valid.

Create Remote Function

  1. Sign into the Azure Portal

  2. Type "Function App" in the top search box and click "Function App"

  3. Click "+ Create"

  4. Select the following options:

  • Type a memorable "Function Name" ("sheetjsazure" when last tested)

  • "Do you want to deploy code or container image?": select "Code"

  • "Runtime stack": select NodeJS

  • "Hosting options and plans": "Consumption (Serverless)"

  1. Click "Review + create", then click "Create" to create the function.

The page will display a status message

... Deployment is in progress

When the resources are configured, the status will change to

Your deployment is complete

  1. Click "Go to Resource".

  2. Take note of the URL from the table

Deploy to Azure

  1. Sign into Azure:
az login

The login flow resumes in the browser.

  1. Deploy to Azure. Replace FUNCTION_NAME with the name from Step 16:
func azure functionapp publish FUNCTION_NAME

After publishing, the process will print the "Invoke url":

Functions in sheetjsazure:
    SheetJSAzure - [httpTrigger]
// highlight-next-line
        Invoke url: https://sheetjsazure.azurewebsites.net/api/sheetjsazure

Take note of that URL.

Remote Test

  1. In a separate terminal window, download https://sheetjs.com/pres.numbers and make a POST request to the production server. Replace FUNCTION_URL with the Invoke URL from Step 21:
curl -LO https://sheetjs.com/pres.numbers
curl -X POST -F "upload=@pres.numbers" FUNCTION_URL

If the test succeeded, the terminal will print CSV rows from the test file data.

  1. Open a web browser and access the Invoke URL from Step 21.

If the test succeeded, the browser will attempt to download SheetJSAzure.xlsx. Open in Excel or another spreadsheet editor to confirm the file is valid.

Blob Storage

The main module for Azure Blob Storage is @azure/storage-blob. This example was tested using the "Connection String" authentication method. The strings are found in the Azure Portal under "Access Keys" for the storage account.

Downloading Data

The BlobClient#download method returns a Stream. After collecting into a Buffer, the SheetJS read method9 can parse the data into a workbook10.

The following demo uses the sheet_to_csv11 utility function to display the contents of a file in Azure Blob Storage:

import { BlobServiceClient } from "@azure/storage-blob";
import { read, utils } from "xlsx";

/* replace these constants */
// highlight-start
const connStr = "<REPLACE WITH CONNECTION STRING>";
const containerName = "<REPLACE WITH CONTAINER NAME>";
// highlight-end

/* Blob name */
const blobName = "SheetJSBloblobber.xlsx";

/* get a readable stream*/
const blobServiceClient = BlobServiceClient.fromConnectionString(connStr);
const containerClient = blobServiceClient.getContainerClient(containerName);
const blobClient = containerClient.getBlobClient(blobName);
const response = (await blobClient.download()).readableStreamBody;

/* collect data into a Buffer */
const bufs = [];
for await(const buf of response) bufs.push(buf);
const downloaded = Buffer.concat(bufs);

/* parse downloaded buffer */
const wb = read(downloaded);
/* print first worksheet */
console.log(utils.sheet_to_csv(wb.Sheets[wb.SheetNames[0]]));

Uploading Data

The SheetJS write method12 with the option type: "buffer" will generate NodeJS buffers which can be uploaded with BlockBlobClient#upload.

The following example generates a sample worksheet using the aoa_to_sheet13 method, generates a sample workbook using worksheet helper methods14, writes the workbook to XLSX format in a Buffer, and sends the Buffer in the response:

import { BlobServiceClient } from "@azure/storage-blob";
import { write, utils } from "xlsx";

/* replace these constants */
// highlight-start
const connStr = "<REPLACE WITH CONNECTION STRING>";
const containerName = "<REPLACE WITH CONTAINER NAME>";
// highlight-end

/* Blob name */
const blobName = "SheetJSBloblobber.xlsx";

/* Create a simple workbook and write XLSX to buffer */
const ws = utils.aoa_to_sheet(["SheetJS".split(""), [5,4,3,3,7,9,5]]);
const wb = utils.book_new(); utils.book_append_sheet(wb, ws, "Sheet1");
const buf = write(wb, {type: "buffer", bookType: "xlsx"});

/* upload buffer */
const blobServiceClient = BlobServiceClient.fromConnectionString(connStr);
const containerClient = blobServiceClient.getContainerClient(containerName);
const blockBlobClient = containerClient.getBlockBlobClient(blobName);
const uploadBlobResponse = await blockBlobClient.upload(buf, buf.length);

Blob Demo

:::note pass

At the time of writing, new Azure accounts were granted a 12-month trial of Blob Storage. The trial includes 5GB of "Locally-redundant storage" with 20,000 read requests and 2000 write requests per month.

:::

  1. If you do not have an account, create a new Azure free tier account8.

Storage Account Setup

  1. Sign into the Azure Portal

  2. Type "Storage" in the top search box and click "Storage accounts"

  3. Click "+ Create"

  4. Select the following options:

  • Type a memorable "Storage account name" ("sheetjstorage" when last tested)

  • "Redundancy": select LRS (Locally-redundant storage)

  • "Hosting options and plans": "Consumption (Serverless)"

  1. Click "Review", then click "Create" to create the storage.

The page will display a status message

... Deployment is in progress

When the resources are configured, the status will change to

Your deployment is complete

  1. Click "Go to Resource".

Access Keys

  1. Click "Access keys" in the left sidebar (under "Security + networking")

  2. Look for the "Connection string" title under "key1". In the row below the title, click "Show" to reveal the key. Click the copy icon or manually copy the key, storing it in a safe place.

Container Setup

  1. Click "Containers" in the left sidebar.

  2. Click "+ Container"

  3. Select the following options:

  • Type a memorable "Name" ("sheetjs-container" when last tested)
  1. Click "Create" to create the container.

Project Setup

  1. Create a new project folder:
mkdir SheetJSBlob
cd SheetJSBlob
npm init -y
  1. Install dependencies:

{\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz @azure/storage-blob}

  1. Copy the SheetJSReadFromAzure.mjs code block and save to SheetJSReadFromAzure.mjs.

  2. Copy the SheetJSWriteToAzure.mjs code block and save to SheetJSWriteToAzure.mjs.

  3. Edit both SheetJSReadFromAzure.mjs and SheetJSWriteToAzure.mjs:

  • Replace the connStr value with the connection string from Step 8
  • Replace the containerName value with the container name from Step 11

Test

:::note pass

The write demo creates a simple workbook, generates a NodeJS buffer, and uploads the buffer to a file named SheetJSBloblobber.xlsx on Azure Blob Storage.

The read demo fetches SheetJSBloblobber.xlsx and displays the data.

   | A | B | C | D | E | F | G |
---+---|---|---|---|---|---|---|
 1 | S | h | e | e | t | J | S |
 2 | 5 | 4 | 3 | 3 | 7 | 9 | 5 |

:::

  1. Run the write test:
node SheetJSWriteToAzure.mjs

This will write the file SheetJSBloblobber.xlsx to the container.

  1. Run the read test:
node SheetJSReadFromAzure.mjs

It will fetch the file created in the previous step and display CSV rows.

S,h,e,e,t,J,S
5,4,3,3,7,9,5

  1. Sign into the Azure Portal

  2. Type "Storage" in the top search box and click "Storage accounts"

  3. Click on the name of the storage

  4. In the middle column, click "Containers". It will be under "Data storage".

  5. Click on the name of the container in the table

  6. Verify that the table shows SheetJSBloblobber.xlsx:

SheetJSBloblobber.xlsx in the container

  1. Click on the name SheetJSBloblobber.xlsx.

  2. In the right pane, click "Download".

The downloaded file is the raw file stored in Azure Blob Storage. To confirm it is valid, open the file in Excel or another spreadsheet editor.

  1. The platform-specific installers are available at https://learn.microsoft.com/en-us/cli/azure/install-azure-cli ↩︎

  2. See read in "Reading Files" ↩︎

  3. See "Workbook Object" in "SheetJS Data Model" for more details. ↩︎

  4. See sheet_to_csv in "CSV and Text" ↩︎

  5. See write in "Writing Files" ↩︎

  6. See aoa_to_sheet in "Utilities" ↩︎

  7. See "Workbook Helpers" in "Utilities" for details on book_new and book_append_sheet. ↩︎

  8. Registering for a free account on the Azure Free Tier requires a valid phone number and a valid credit card. ↩︎

  9. See read in "Reading Files" ↩︎

  10. See "Workbook Object" in "SheetJS Data Model" for more details. ↩︎

  11. See sheet_to_csv in "CSV and Text" ↩︎

  12. See write in "Writing Files" ↩︎

  13. See aoa_to_sheet in "Utilities" ↩︎

  14. See "Workbook Helpers" in "Utilities" for details on book_new and book_append_sheet. ↩︎