docs.sheetjs.com/docz/docs/03-demos/30-cloud/21-gsheet.md

---
title: Google Sheets Data Interchange
sidebar_label: Google Sheets
pagination_prev: demos/local/index
pagination_next: demos/extensions/index
---

import current from '/version.js';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';

:::note pass

This demo focuses on external data processing.  For Google Apps Script custom
functions, [the "Google Sheets" extension demo](/docs/demos/extensions/gsheet)
covers Apps Script integration.

:::

[Google Sheets](https://google.com/sheets/about/) is a collaborative spreadsheet
service with powerful external APIs for automation.

[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing
data from spreadsheets.

This demo uses SheetJS to properly exchange data with spreadsheet files. We'll
explore how to use NodeJS integration libraries and SheetJS in three data flows:

- "Importing data": Data in a NUMBERS spreadsheet will be parsed using SheetJS
libraries and written to a Google Sheets Document

- "Exporting data": Data in Google Sheets will be pulled into arrays of objects.
A workbook will be assembled and exported to Excel Binary workbooks (XLSB).

- "Exporting files": SheetJS libraries will read XLSX and ODS files exported by
Google Sheets and generate CSV rows from every worksheet.

:::danger pass

It is strongly recommended to create a new Google account for testing.

**One small mistake could result in a block or ban from Google services.**

:::

:::caution pass

Google Sheets deprecates APIs quickly and there is no guarantee that the
referenced APIs will be available in the future.

:::

## Integration Details

This demo uses the following NodeJS modules:

- `google-auth-library`[^1] simplifies authentication with Google APIs
- `node-google-spreadsheet`[^2] interacts with Google Sheets v4 API

:::info Initial Setup

There are a number of steps to enable the Google Sheets API and Google Drive API
for an account. The [Complete Example](#complete-example) covers the process.

:::

### Authentication

It is strongly recommended to use a service account for Google API operations.
The ["Service Account Setup" section](#service-account-setup) covers how to
create a service account and generate a JSON key file.

The generated JSON key file includes `client_email` and `private_key` fields.
These fields can be used in JWT authentication:

```js title="JWT Authentication using a JSON key file"
import { JWT } from 'google-auth-library'
import { GoogleSpreadsheet } from 'google-spreadsheet';

// adjust the path to the actual key file.
// highlight-next-line
import creds from './sheetjs-test-726272627262.json' assert { type: "json" };

const jwt = new JWT({
  email: creds.client_email,
  key: creds.private_key,
  scopes: [
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/drive.file',
  ]
});
```

### Connecting to Documents

To connect to existing documents, the document ID must be specified. This ID can
be found from the edit URL.

The edit URL starts with `https://docs.google.com/spreadsheets/d/` and includes
`/edit`. The ID is the string of characters between the slashes.  For example:

```
https://docs.google.com/spreadsheets/d/a_long_string_of_characters/edit#gid=0
                                      |^^^^^^^^^^^^^^^^^^^^^^^^^^^|--- ID
```

The `GoogleSpreadsheet` constructor accepts a document ID and auth object:

```js title="Connect to a document"
const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt);
await doc.loadInfo();
```

### Creating New Documents

The `createNewSpreadsheetDocument` makes a request to create a new document. It
is strongly recommended to create a blank sheet.

```js title="Create a new document with blank sheet"
const doc = await GoogleSpreadsheet.createNewSpreadsheetDocument(jwt, { title: 'Document Title' });
const newSheet = await doc.addSheet({ title: 'Sheet Title' });
```

### Array of Arrays

["Arrays of Arrays"](/docs/api/utilities/array#array-of-arrays) are the main
data format for interchange with Google Sheets. The outer array object includes
row arrays, and each row array includes data.

SheetJS provides methods for working with Arrays of Arrays:

- `aoa_to_sheet`[^3] creates SheetJS worksheet objects from arrays of arrays
- `sheet_to_json`[^4] can generate arrays of arrays from SheetJS worksheets

## Export Document Data

The goal is to create an XLSB export from a Google Sheet.  Google Sheets does
not natively support the XLSB format.  SheetJS fills the gap.

### Convert a Single Sheet

The idea is to extract the raw data from the Google Sheet headers and combine
with the raw data rows to produce a large array of arrays.

```js
async function wb_append_sheet(sheet, name, wb) {
  /* get the header and data rows */
  await sheet.loadHeaderRow();
  const header = sheet.headerValues;
  const rows = await sheet.getRows();

  /* construct the array of arrays */
  const aoa = [header].concat(rows.map(r => r._rawData));

  /* generate a SheetJS Worksheet */
  const ws = XLSX.utils.aoa_to_sheet(aoa);

  /* add to workbook */
  XLSX.utils.book_append_sheet(wb, ws, name);
}
```

### Convert a Workbook

`doc.sheetsByIndex` is an array of worksheets in the Google Sheet Document.  By
looping across the sheets, the entire workbook can be written:

```js
async function doc_to_wb(doc) {
  /* Create a new workbook object */
  const wb = XLSX.utils.book_new();

  /* Loop across the Document sheets */
  for(let i = 0; i < doc.sheetsByIndex.length; ++i) {
    const sheet = doc.sheetsByIndex[i];
    /* Get the worksheet name */
    const name = sheet.title;

    /* Add sheet to workbook */
    await add_sheet_to_wb(sheet, name, wb);
  }

  return wb;
}
```

## Update Document Data

The goal is to import data from a NUMBERS file to Google Sheets. Google Sheets
does not natively support the NUMBERS format.  SheetJS fills the gap.

### Clear the Document

Google Sheets does not allow users to delete every worksheet.  This function
deletes every worksheet after the first, then clears the first worksheet:

```js
/* clear google sheets doc */
async function doc_clear(doc) {
  /* delete all sheets after the first sheet */
  const old_sheets = doc.sheetsByIndex;
  for(let i = 1; i < old_sheets.length; ++i) await old_sheets[i].delete();

  /* clear first worksheet */
  old_sheets[0].clear();
}
```

### Update First Sheet

There are two steps: "update worksheet name" and "update worksheet data":

#### Update Sheet Name

The worksheet name is assigned by using the `updateProperties` method.  The
desired sheet name is the name of the first worksheet from the file.

```js
async function doc_update_first_sheet_name(doc, wb) {
  /* get first worksheet name */
  const wsname = wb.SheetNames[0];

  /* get first gsheet */
  const sheet = doc.sheetsByIndex[0];

  /* update worksheet name */
  await sheet.updateProperties({title: wsname});
}
```

#### Update Sheet Data

`sheet.addRows` reads an Array of Arrays of values. `XLSX.utils.sheet_to_json`
can generate this exact shape with the option `header: 1`.  Unfortunately
Google Sheets requires at least one "Header Row".  This can be implemented by
converting the entire worksheet to an Array of Arrays and setting the header
row to the first row of the result:

```js
async function doc_update_first_sheet_data(doc, wb) {
  /* get first worksheet */
  const ws = wb.Sheets[wb.SheetNames[0]];
  /* generate array of arrays from the first worksheet */
  const aoa = XLSX.utils.sheet_to_json(ws, {header: 1});

  /* get first gsheet */
  const sheet = doc.sheetsByIndex[0];
  /* set document header row to first row of the AOA */
  await sheet.setHeaderRow(aoa[0]);

  /* add the remaining rows */
  await sheet.addRows(aoa.slice(1));
}
```

### Append Remaining Worksheets

Each name in the SheetJS Workbook `SheetNames` array maps to a worksheet.  The
list of names not including the first sheet is `wb.SheetNames.slice(1)`.

There are two steps for each sheet: "create new sheet" and "load data".

Due to JavaScript `async` idiosyncrasies, a plain `for` loop must be used:

```js
async function doc_append_remaining_sheets(doc, wb) {
  const names = wb.SheetNames.slice(1);

  /* loop across names */
  for(let i = 0; i < names.length; ++i) {
    /* wb.SheetNames[i] is the sheet name */
    const name = wb.SheetNames[i];
    /* wb.Sheets[name] is the worksheet object */
    const ws = wb.Sheets[name];

    /* create new google sheet */
    const sheet = await doc_add_new_sheet(doc, name);
    /* load sheet with data */
    await sheet_load_from_ws(sheet, ws);
  }
}
```

#### Add a New Worksheet

`doc.addSheet` accepts a properties object that includes the worksheet name:

```js
async function doc_add_new_sheet(doc, name) {
  return await doc.addSheet({title: name});
}
```

This creates a new worksheet, sets the tab name, and returns a reference to the
created worksheet.

#### Update Worksheet Data

```js
async function sheet_load_from_ws(sheet, ws) {
  /* generate array of arrays from the first worksheet */
  const aoa = XLSX.utils.sheet_to_json(ws, {header: 1});

  /* set document header row to first row of the AOA */
  await sheet.setHeaderRow(aoa[0]);

  /* add the remaining rows */
  await sheet.addRows(aoa.slice(1));
}
```

## Raw File Exports

In the web interface, Google Sheets can export documents to `XLSX` or `ODS`. The
NodeJS library includes similar methods to perform the download[^5]:

| Format | Google Sheets Description | Method           |
|:-------|:--------------------------|:-----------------|
| XLSX   | Microsoft Excel (.xlsx)   | `downloadAsXLSX` |
| ODS    | OpenDocument (.ods)       | `downloadAsODS`  |

The functions resolve to `Buffer` data. The `Buffer` objects can be parsed using
the SheetJS `read`[^6] method:

```js
/* download XLSX */
const ab = await doc.downloadAsXLSX();

/* parse */
const wb = XLSX.read(buf);
```

At this point `wb` is a SheetJS workbook object[^7].

## Complete Example

:::note Tested Deployments

This demo was last tested on 2023 September 17 using `google-auth-library` for
authentication (`v8.9.0`) and `google-spreadsheet` for API access (`v4.1.0`).

:::

### Account Setup

0) Create a new Google account or log into an existing account.

:::caution pass

A valid phone number (for SMS verification) may be required.

:::

1) Open https://console.cloud.google.com in a web browser. Review the Google
Cloud Platform Terms of Service.

:::danger pass

You must agree to the Google Cloud Platform Terms of Service to use the APIs.

:::

### Project Setup

2) Create a new Project.

If the account does not have an existing project, click "CREATE PROJECT"

If the account has an existing project, click the project selector (`:·` icon)
and click "NEW PROJECT" in the modal.

3) In the New Project screen, enter "SheetJS Test" in the Project name textbox
and select "No organization" in the Location box. Click "CREATE"

### API Setup

:::info pass

The goal of this section is to enable Google Sheets API and Google Drive API.

:::

4) Click the Project Selector (`:·` icon) and select "SheetJS Test"

5) In the left sidebar, click "Enabled APIs and services".

6) Near the top of the page, click "+ ENABLE APIS AND SERVICES".

7) In the search bar near the middle of the page, type "Sheets" and look for
"Google Sheets API". Click the card

8) In the Product Details screen, click the blue "ENABLE" button.

9) Click the left arrow (`<-`) next to API/Service details.

10) In the search bar near the middle of the page, type "Drive" and look for
"Google Drive API". Click the card.

11) In the Product Details screen, click the blue "ENABLE" button.

### Service Account Setup

:::info pass

The goal of this section is to create a service account and generate a JSON key.

:::

#### Create Service Account

12) Go to https://console.cloud.google.com.

13) Click the Project Selector (`:·` icon) and select "SheetJS Test".

14) Click "Dashboard".

15) In the left sidebar, hover over "APIs and Services" and select "Credentials"

16) Click "+ CREATE CREDENTIALS". In the dropdown, select "Service Account"

17) Enter "SheetJService" for Service account name. Click "CREATE AND CONTINUE"

:::note pass

The Service account ID is generated automatically.

:::

18) In Step 2 "Grant this service account access to project", click CONTINUE

19) In Step 3 click "DONE". You will be taken back to the credentials screen

#### Create JSON Key

20) Look for "SheetJService" in the "Service Accounts" table and click the email
address in the row

21) Click the email address of the account in the "Service Accounts" table.

22) Click "KEYS" in the horizontal bar near the top of the page.

23) Click "ADD KEY" and select "Create new key" in the dropdown.

24) In the popup, select the "JSON" radio button and click "CREATE". The page
will download a JSON file.

25) Click "CLOSE"

### Create Document

:::info pass

The goal of this section is to create a document from the service account and
share with the main account.

:::

26) Create a `SheetJSGS` folder and initialize:

```bash
mkdir SheetJSGS
cd SheetJSGS
npm init -y
```

27) Copy the JSON file from step 24 into the project folder.

28) Install dependencies:

<CodeBlock language="bash">{`\
npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz google-spreadsheet google-auth-library`}
</CodeBlock>

29) Save the following script to `init.mjs`:

```js title="init.mjs"
import { JWT } from 'google-auth-library'
import { GoogleSpreadsheet } from 'google-spreadsheet';

// highlight-next-line
import creds from './sheetjs-test-726272627262.json' assert { type: "json" };

const jwt = new JWT({
  email: creds.client_email,
  key: creds.private_key,
  scopes: [
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/drive.file',
  ]
});

const doc = await GoogleSpreadsheet.createNewSpreadsheetDocument(jwt, { title: 'test from NodeJS' });
const newSheet = await doc.addSheet({ title: 'SheetJSTest' });
// highlight-next-line
await doc.share('YOUR_ADDRESS@gmail.com');
```

Edit the highlighted lines as follows:

- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the
JSON file in step 27. The `./` prefix is required!

- `'YOUR_ADDRESS@gmail.com'` should be replaced with the Google Account email
address from step 0.

30) Run the script:

```bash
node init.mjs
```

31) Sign into Google Sheets. A shared document "test from NodeJS" should be
displayed in the table. It will be owned by the service account.

32) Open the shared document from step 31.

33) Copy the URL and extract the document ID.

The URL of the document will look like

```
https://docs.google.com/spreadsheets/d/a_long_string_of_characters/edit#gid=0
---------------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^--- ID
```

The ID is a long string of letters and numbers and underscore characters (`_`)
just before the `/edit` part of the URL.

### Load Data from NUMBERS

:::info pass

The goal of this section is to update the new document with data from a sample
NUMBERS file.

:::

34) Download the [test file `pres.numbers`](https://docs.sheetjs.com/pres.numbers):

```bash
curl -LO https://docs.sheetjs.com/pres.numbers
```

35) Save the following script to `load.mjs`:

```js title="load.mjs"
import { JWT } from 'google-auth-library'
import { GoogleSpreadsheet } from 'google-spreadsheet';

import { set_fs, readFile, utils } from 'xlsx';
import * as fs from 'fs';
set_fs(fs);

// highlight-next-line
import creds from './sheetjs-test-726272627262.json' assert { type: "json" };

const jwt = new JWT({
  email: creds.client_email,
  key: creds.private_key,
  scopes: [
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/drive.file',
  ]
});

// highlight-next-line
const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt);
await doc.loadInfo();

const wb = readFile("pres.numbers");

/* clear workbook */
{
  /* delete all sheets after the first sheet */
  const old_sheets = doc.sheetsByIndex;
  for(let i = 1; i < old_sheets.length; ++i) {
    await old_sheets[i].delete();
  }
  /* clear first worksheet */
  old_sheets[0].clear();
}

/* write worksheets */
{
  const name = wb.SheetNames[0];
  const ws = wb.Sheets[name];
  /* first worksheet already exists */
  const sheet = doc.sheetsByIndex[0];

  /* update worksheet name */
  await sheet.updateProperties({title: name});

  /* generate array of arrays from the first worksheet */
  const aoa = utils.sheet_to_json(ws, {header: 1});

  /* set document header row to first row of the AOA */
  await sheet.setHeaderRow(aoa[0])

  /* add the remaining rows */
  await sheet.addRows(aoa.slice(1));

  /* the other worksheets must be created manually */
  for(let i = 1; i < wb.SheetNames.length; ++i) {
    const name = wb.SheetNames[i];
    const ws = wb.Sheets[name];

    const sheet = await doc.addSheet({title: name});
    const aoa = utils.sheet_to_json(ws, {header: 1});
    await sheet.setHeaderRow(aoa[0])
    await sheet.addRows(aoa.slice(1));
  }
}
```

Edit the highlighted lines as follows:

- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the
JSON file in step 27. The `./` prefix is required!

- `'DOCUMENT_ID'` should be replaced with the Document ID from step 33.

36) Run the script:

```bash
node load.mjs
```

37) Sign into Google Sheets and open the "test from NodeJS" shared document. It
should show a list of Presidents, matching the contents of the test file.

### Export Data to XLSB

:::info pass

The goal of this section is to export the raw data from Google Sheets to XLSB.

:::

38) Save the following script to `dump.mjs`:

```js title="dump.mjs"
import { JWT } from 'google-auth-library'
import { GoogleSpreadsheet } from 'google-spreadsheet';

import { set_fs, writeFile, utils } from 'xlsx';
import * as fs from 'fs';
set_fs(fs);

// highlight-next-line
import creds from './sheetjs-test-726272627262.json' assert { type: "json" };

const jwt = new JWT({
  email: creds.client_email,
  key: creds.private_key,
  scopes: [
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/drive.file',
  ]
});

// highlight-next-line
const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt);
await doc.loadInfo();

const wb = utils.book_new();

for(let i = 0; i < doc.sheetsByIndex.length; ++i) {
  const sheet = doc.sheetsByIndex[i];
  const name = sheet.title;

  /* get the header and data rows */
  await sheet.loadHeaderRow();
  const header = sheet.headerValues;
  const rows = await sheet.getRows();
  const aoa = [header].concat(rows.map(r => r._rawData));

  /* generate a SheetJS Worksheet */
  const ws = utils.aoa_to_sheet(aoa);

  /* add to workbook */
  utils.book_append_sheet(wb, ws, name);
}

/* write to SheetJS.xlsb */
writeFile(wb, "SheetJS.xlsb");
```

Edit the highlighted lines as follows:

- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the
JSON file in step 27. The `./` prefix is required!

- `'DOCUMENT_ID'` should be replaced with the Document ID from step 33.

39) Run the script:

```bash
node dump.mjs
```

The script should create a file `SheetJS.xlsb` in the project folder. This file
can be opened in Excel

40) Sign into Google Sheets and open the "test from NodeJS" shared document. It
should show a list of Presidents, matching the contents of the test file.

### Export Raw Files

:::info pass

The goal of this section is to parse the Google Sheets XLSX export and generate
CSV files for each worksheet.

:::

41) Sign into Google Sheets and open the "test from NodeJS" shared document.

42) Click the Plus (`+`) icon in the lower left corner to create a new worksheet.

43) In the new worksheet, set cell A1 to the formula `=SEQUENCE(3,5)`. This will
assign a grid of values

44) Save the following script to `raw.mjs`:

```js title="raw.mjs"
import { JWT } from 'google-auth-library'
import { GoogleSpreadsheet } from 'google-spreadsheet';

import { read, utils } from 'xlsx';

// highlight-next-line
import creds from './sheetjs-test-726272627262.json' assert { type: "json" };

const jwt = new JWT({
  email: creds.client_email,
  key: creds.private_key,
  scopes: [
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/drive.file',
  ]
});

// highlight-next-line
const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt);
await doc.loadInfo();

const buf = await doc.downloadAsXLSX();

/* Parse with SheetJS */
const wb = read(buf);

/* Loop over the worksheet names */
wb.SheetNames.forEach(name => {
  /* Print the name to the console */
  console.log(name);

  /* Get the corresponding worksheet object */
  const sheet = wb.Sheets[name];

  /* Print a CSV export of the worksheet */
  console.log(utils.sheet_to_csv(sheet));
});
```

Edit the highlighted lines as follows:

- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the
JSON file in step 27. The `./` prefix is required!

- `'DOCUMENT_ID'` should be replaced with the Document ID from step 33.

45) Run the script:

```bash
node raw.mjs
```

The script will display the sheet names and CSV rows from both worksheets.

[^1]: The package name is [`google-auth-library`](https://www.npmjs.com/package/google-auth-library)
[^2]: The project name is [`node-google-spreadsheet`](https://github.com/theoephraim/node-google-spreadsheet) but the module name is `google-spreadsheet`.
[^3]: See [`aoa_to_sheet` in "Utilities"](/docs/api/utilities/array#array-of-arrays-input)
[^4]: See [`sheet_to_json` in "Utilities"](/docs/api/utilities/array#array-output)
[^5]: See ["Exporting Data"](https://theoephraim.github.io/node-google-spreadsheet/#/guides/exports) in the `node-google-spreadsheet` documentation
[^6]: See [`read` in "Reading Files"](/docs/api/parse-options)
[^7]: See ["Workbook Object"](/docs/csf/book) for a description of the workbook object or ["API Reference"](/docs/api) for various methods to work with workbook and sheet objects.