diff --git a/docz/docs/03-demos/02-grid/16-rdg.md b/docz/docs/03-demos/02-grid/16-rdg.md index 39ae362..e07d6fd 100644 --- a/docz/docs/03-demos/02-grid/16-rdg.md +++ b/docz/docs/03-demos/02-grid/16-rdg.md @@ -9,12 +9,11 @@ import CodeBlock from '@theme/CodeBlock'; :::note -This demo was last tested on 2023 April 18 with `react-data-grid 7.0.0-beta.28`, -`create-react-app` 5.0.1 and React 18.2.0. +This demo was last tested on 2023 September 17 with `react-data-grid` version +`7.0.0-beta.39` and React 18.2.0. ::: - The demo creates a site that looks like the screenshot below: ![react-data-grid screenshot](pathname:///rdg/rdg1.png) @@ -117,7 +116,7 @@ cd sheetjs-rdg 2) Install dependencies: {`\ -npm i -S https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz react-data-grid`} +npm i -S https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz react-data-grid@7.0.0-beta.39`} 3) Download [`App.tsx`](pathname:///rdg/App.tsx) and replace `src/App.tsx`. @@ -126,5 +125,15 @@ npm i -S https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz react-data- curl -L -o src/App.tsx https://docs.sheetjs.com/rdg/App.tsx ``` -4) run `npm start`. When you load the page in the browser, it will attempt to - fetch and load the data. +4) Start the development server: + +```bash +npm start +``` + +#### Testing + +5) When the page loads, it will fetch , parse +with SheetJS, and load the data in the data grid. + +6) Click one of the "export" buttons to export the grid data to file. diff --git a/docz/docs/03-demos/05-mobile/03-quasar.md b/docz/docs/03-demos/05-mobile/03-quasar.md index c9d22c6..18c1c2c 100644 --- a/docz/docs/03-demos/05-mobile/03-quasar.md +++ b/docz/docs/03-demos/05-mobile/03-quasar.md @@ -1,5 +1,7 @@ --- -title: Quasar +title: Data in Quasar Apps +sidebar_label: Quasar +description: Build data-intensive mobile apps with Quasar and Cordova. Seamlessly integrate spreadsheets into your app using SheetJS. Let data in your Excel spreadsheets shine. pagination_prev: demos/static/index pagination_next: demos/desktop/index sidebar_position: 3 @@ -10,10 +12,17 @@ sidebar_custom_props: import current from '/version.js'; import CodeBlock from '@theme/CodeBlock'; -The [NodeJS Module](/docs/getting-started/installation/nodejs) can be imported -from the main entrypoint or any script in the project. +[Quasar](https://quasar.dev/) is a VueJS framework for building iOS and Android +apps with the Cordova platform. -The "Complete Example" creates an app that looks like the screenshots below: +[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing +data from spreadsheets. + +This demo uses Quasar and SheetJS to process data and generate spreadsheets. +We'll explore how to load SheetJS in an Quasar app and use Quasar and Cordova +features to extract data from, and write data to, spreadsheets on the device. + +The ["Demo"](#demo) creates an app that looks like the screenshots below: @@ -30,10 +39,14 @@ The "Complete Example" creates an app that looks like the screenshots below: ### Integration Details +The [NodeJS Module](/docs/getting-started/installation/nodejs) can be imported +from the main entrypoint or any script in the project. + This demo will use the Quasar ViteJS starter project with VueJS and Cordova. +The starter places the backing Cordova project in the `src-cordova` folder. The complete solution uses `cordova-plugin-file` for file operations. It can -be installed like any other Cordova plugin: +be installed from the Cordova folder: ```bash cd src-cordova @@ -41,15 +54,21 @@ cordova plugin add cordova-plugin-file cd .. ``` -#### Reading data +### Reading data -The `q-file` component presents an API reminiscent of File Input elements: +The QFile[^1] component presents an API reminiscent of File Input elements: ```html ``` -When binding to the `input` element, the callback receives an `Event` object: +When binding to the `input` element, the callback receives an `Event` object. +Using standard DOM operations, the file data can be pulled into an `ArrayBuffer` +and parsed using the SheetJS `read` method[^2]. `read` returns a workbook[^3] +object that holds data and metadata for each worksheet. + +This snippet reads a workbook, pulls the first worksheet, generates an array of +objects using the SheetJS `sheet_to_json`[^4] method, and updates state: ```ts import { read } from 'xlsx'; @@ -74,14 +93,33 @@ async function updateFile(v) { try { #### Writing data -The API is shaped like the File and Directory Entries API. For clarity, since -the code is a "pyramid of doom", the error handlers are omitted: +Starting from an array of objects, the SheetJS `json_to_sheet` method[^5] +generates a SheetJS worksheet object. The `book_append_sheet` and `book_new` +helper functions[^6] create a SheetJS workbook object that can be exported: -```ts +```js +import { utils } from 'xlsx'; + +// assuming `todos` is a VueJS ref whose value is an array of objects +const ws = utils.json_to_sheet(todos.value); +const wb = utils.book_new(); +utils.book_append_sheet(wb, ws, "SheetJSQuasar"); +``` + +The SheetJS `write` function[^7] with the option `type: "buffer"` will generate +`Uint8Array` objects that can be converted to blobs and exported: + +```js import { write } from 'xlsx'; // on iOS and android, `XLSX.write` with type "buffer" returns a `Uint8Array` const u8: Uint8Array = write(wb, {bookType: "xlsx", type: "buffer"}); +``` + +The `cordova-plugin-file` API writes the data to the filesystem. The code uses +the File and Directory Entries API[^8]: + +```ts // Request filesystem access for persistent storage window.requestFileSystem(window.PERSISTENT, 0, function(fs) { // Request a handle to "SheetJSQuasar.xlsx", making a new file if necessary @@ -107,12 +145,11 @@ window.requestFileSystem(window.PERSISTENT, 0, function(fs) { :::note -This demo was tested on an Intel Mac on 2023 April 08. `create-quasar@1.1.2` -was installed during app creation. The app used Quasar version `2.11.10`. +The Android demo was last tested on 2023 September 18 with Quasar `2.12.7` on an +emulated Pixel 3 + Android 13 ("Tiramisu") API 33. -The iOS simulator runs iOS 16.2 on an iPhone 14 Pro Max. - -The Android simulator runs Android 12.0 (S) API 31 on a Pixel 3. +The iOS demo was last tested on 2023 September 18 with Quasar `2.12.7` on an +emulated iPhone SE (3rd generation) + iOS 16.4. ::: @@ -196,20 +233,38 @@ Return to the project directory: cd .. ``` -4) Start the development server: +11) Enable file sharing and make the documents folder visible in the iOS app. +The following lines must be added to `src-cordova/platforms/ios/SheetJSQuasar/SheetJSQuasar-Info.plist`: + +```xml title="src-cordova/platforms/ios/SheetJSQuasar/SheetJSQuasar-Info.plist (add to file)" + + + + UIFileSharingEnabled + + LSSupportsOpeningDocumentsInPlace + + + CFBundleDevelopmentRegion +``` + +(The root element of the document is `plist` and it contains one `dict` child) + + +5) Start the development server: ```bash quasar dev -m ios ``` -:::caution +:::caution pass If the app is blank or not refreshing, delete the app and close the simulator, then restart the development process. ::: -5) Add the Dialog plugin to `quasar.config.js`: +6) Add the Dialog plugin to `quasar.config.js`: ```js title="quasar.config.js" framework: { @@ -221,7 +276,7 @@ then restart the development process. }, ``` -6) In the template section of `src/pages/IndexPage.vue`, replace the example +7) In the template section of `src/pages/IndexPage.vue`, replace the example with a Table, Save button and Load file picker component: ```html title="src/pages/IndexPage.vue" @@ -266,7 +321,7 @@ then restart the development process. ::: -7) Wire up the `updateFile` function: +8) Wire up the `updateFile` function: ```ts title="src/pages/IndexPage.vue" import { defineComponent, ref } from 'vue'; @@ -315,7 +370,7 @@ To test that reading works: Once selected, the screen should refresh with new contents. -8) Wire up the `saveFile` function: +9) Wire up the `saveFile` function: ```ts title="src/pages/IndexPage.vue" function saveFile() { @@ -401,7 +456,7 @@ id,content **Android** -9) Create the Android project: +10) Create the Android project: ```bash cd src-cordova @@ -409,24 +464,12 @@ cordova platform add android cd .. ``` -10) Start the simulator: +11) Start the simulator: ```bash quasar dev -m android ``` -:::note - -In local testing, the Quasar build process threw an error: - -``` - java.lang.IllegalArgumentException: Unsupported class file major version 63 -``` - -This was resolved by rolling back to Java 1.8 - -::: - To test that reading works: - Click and drag `pres.numbers` from a Finder window into the simulator. @@ -442,3 +485,12 @@ To test that writing works: adb exec-out run-as org.sheetjs.quasar cat files/files/SheetJSQuasar.xlsx > /tmp/SheetJSQuasar.xlsx npx xlsx-cli /tmp/SheetJSQuasar.xlsx ``` + +[^1]: See ["File Picker"](https://quasar.dev/vue-components/file-picker) in the Quasar documentation. +[^2]: See [`read` in "Reading Files"](/docs/api/parse-options) +[^3]: See ["SheetJS Data Model"](/docs/csf/) for more details on workbooks, worksheets, and other concepts. +[^4]: See [`sheet_to_json` in "Utilities"](/docs/api/utilities/array#array-output) +[^5]: See [`json_to_sheet` in "Utilities"](/docs/api/utilities/array#array-of-objects-input) +[^6]: See ["Workbook Helpers" in "Utilities"](/docs/api/utilities/wb) for details on `book_new` and `book_append_sheet`. +[^7]: See [`write` in "Writing Files"](/docs/api/write-options) +[^8]: See [`requestFileSystem`](https://developer.mozilla.org/en-US/docs/Web/API/Window/requestFileSystem) in the MDN Web Docs for more details. \ No newline at end of file diff --git a/docz/docs/03-demos/07-data/index.md b/docz/docs/03-demos/07-data/index.md index 100d0e1..8deacaa 100644 --- a/docz/docs/03-demos/07-data/index.md +++ b/docz/docs/03-demos/07-data/index.md @@ -183,7 +183,9 @@ The following Web APIs are featured in separate demos: {item.label}{item.customProps?.summary && (" - " + item.customProps.summary)} ); })} -
  • IndexedDB API
  • +
  • Local Storage API
  • +
  • IndexedDB API
  • + ### SQL Databases diff --git a/docz/docs/03-demos/07-data/02-storageapi.md b/docz/docs/03-demos/08-local/03-storageapi.md similarity index 93% rename from docz/docs/03-demos/07-data/02-storageapi.md rename to docz/docs/03-demos/08-local/03-storageapi.md index bfadf26..fb5635b 100644 --- a/docz/docs/03-demos/07-data/02-storageapi.md +++ b/docz/docs/03-demos/08-local/03-storageapi.md @@ -1,9 +1,9 @@ --- title: Local Storage API -pagination_prev: demos/desktop/index -pagination_next: demos/local/index +pagination_prev: demos/data/index +pagination_next: demos/cloud/index sidebar_custom_props: - type: web + summary: Reading and writing data in an in-browser Key-Value store --- The Storage API, encompassing `localStorage` and `sessionStorage`, describes @@ -14,6 +14,17 @@ This demo covers two common use patterns: - "Row Objects" shows a simple convention for loading and storing row objects - "Simple Strings" discusses how to persist and recover a raw Storage +:::note + +Each browser demo was tested in the following environments: + +| Browser | Date | +|:------------|:-----------| +| Chrome 116 | 2023-09-17 | +| Safari 16.6 | 2023-09-17 | + +::: + ## Row Objects Consider the following array of objects of data: @@ -70,12 +81,6 @@ function localStorage_to_sheet() { ### Live Demo -:::note - -This demo was last tested on 2023 April 09. - -::: - This example will fetch , fill `localStorage` with rows, then generate a worksheet from the rows and write to a new file. @@ -84,13 +89,15 @@ After saving the exported file, the Local Storage can be inspected in the ![Local Storage view in Developer Tools](pathname:///storageapi/lstorage.png) -:::caution +:::caution pass This example is for illustration purposes. If array of objects is available, it is strongly recommended to convert that array to a worksheet directly. ::: +
    Live Demo (click to show) + ```jsx live function SheetJStorage() { const [url, setUrl] = React.useState("https://sheetjs.com/pres.numbers"); @@ -129,6 +136,8 @@ function SheetJStorage() { } ``` +
    + ## Simple Strings The ["Row Objects" approach](#row-objects) is strongly recommended when trying @@ -155,7 +164,7 @@ The natural representation is an array of arrays: #### Exporting Storage -:::note +:::note pass Web Storage iteration order is not defined. By using indices as keys, the row objects approach has an ordering. That does not apply to the general case. @@ -187,15 +196,11 @@ function ws_to_localStorage(ws) { ### Live Demo -:::note - -This demo was last tested on 2023 April 09. - -::: - This example fills `localStorage` with 10 random keys and 10 random values, generates a worksheet from the data and writes to a new file. +
    Live Demo (click to show) + ```jsx live function SheetJSRandomStorage() { const [out, setOut] = React.useState(""); @@ -231,3 +236,5 @@ function SheetJSRandomStorage() { ); } ``` + +
    \ No newline at end of file diff --git a/docz/docs/03-demos/08-local/09-indexeddb.md b/docz/docs/03-demos/08-local/09-indexeddb.md index db2f38e..74953b6 100644 --- a/docz/docs/03-demos/08-local/09-indexeddb.md +++ b/docz/docs/03-demos/08-local/09-indexeddb.md @@ -3,7 +3,7 @@ title: IndexedDB API pagination_prev: demos/desktop/index pagination_next: demos/local/index sidebar_custom_props: - summary: Reading and writing data in browser storage + summary: Reading and writing data in an in-browser NoSQL database --- diff --git a/docz/docs/03-demos/09-cloud/21-gsheet.md b/docz/docs/03-demos/09-cloud/21-gsheet.md index 6f9d5c6..10bb931 100644 --- a/docz/docs/03-demos/09-cloud/21-gsheet.md +++ b/docz/docs/03-demos/09-cloud/21-gsheet.md @@ -9,16 +9,7 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -This demo uses `node-google-spreadsheet` to interact with Google Sheets v4 API. - -:::caution - -Google Sheets deprecates APIs quickly and there is no guarantee that the -referenced API version will be available in the future. - -::: - -:::note +:::note pass This demo focuses on external data processing. For Google Apps Script custom functions, [the "Google Sheets" extension demo](/docs/demos/extensions/gsheet) @@ -26,127 +17,123 @@ covers Apps Script integration. ::: -## Initial Configuration +[Google Sheets](https://google.com/sheets/about/) is a collaborative spreadsheet +service with powerful external APIs for automation. -Install the dependencies: +[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing +data from spreadsheets. -{`\ -npm i -S https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz google-spreadsheet@3.3.0`} - +This demo uses SheetJS to properly exchange data with spreadsheet files. We'll +explore how to use NodeJS integration libraries and SheetJS in three data flows: -The library README has a [guide](https://theoephraim.github.io/node-google-spreadsheet/#/getting-started/authentication) -for configuring a service worker with write access to the document. Following -the service worker guide, the JSON key should be saved to `key.json`. +- "Importing data": Data in a NUMBERS spreadsheet will be parsed using SheetJS +libraries and written to a Google Sheets Document -The following helper function attempts to authenticate and access the specified -sheet by ID. The code should be copied and saved to `common.js`: +- "Exporting data": Data in Google Sheets will be pulled into arrays of objects. +A workbook will be assembled and exported to Excel Binary workbooks (XLSB). -
    Code (click to show) +- "Exporting files": SheetJS libraries will read XLSX and ODS files exported by +Google Sheets and generate CSV rows from every worksheet. -```js title=common.js -const fs = require("fs"); -const { GoogleSpreadsheet } = require('google-spreadsheet'); +:::warning pass -module.exports = async(ID) => { - /* get credentials */ - const creds = JSON.parse(fs.readFileSync('key.json')); +It is strongly recommended to create a new Google account for testing. - /* initialize sheet and authenticate */ - const doc = new GoogleSpreadsheet(ID); - await doc.useServiceAccountAuth(creds); - await doc.loadInfo(); - return doc; -} +**One small mistake could result in a block or ban from Google services.** + +::: + +:::caution pass + +Google Sheets deprecates APIs quickly and there is no guarantee that the +referenced APIs will be available in the future. + +::: + +## Integration Details + +This demo uses the following NodeJS modules: + +- `google-auth-library`[^1] to authenticate with Google APIs +- `node-google-spreadsheet`[^2] to interact with Google Sheets v4 API + +:::info Initial Setup + +There are a number of steps to enable the Google Sheets API and Google Drive API +for an account. The [Complete Example](#complete-example) covers the process. + +::: + +### Authentication + +It is strongly recommended to use a service account for Google API operations. +The ["Service Account Setup" section](#service-account-setup) covers how to +create a service account and generate a JSON key file. + +```js title="Authenticate using a JSON key file" +import { JWT } from 'google-auth-library' +import { GoogleSpreadsheet } from 'google-spreadsheet'; + +// adjust the path to the actual key file. +// highlight-next-line +import creds from './sheetjs-test-726272627262.json' assert { type: "json" }; + +const jwt = new JWT({ + email: creds.client_email, + key: creds.private_key, + scopes: [ + 'https://www.googleapis.com/auth/spreadsheets', + 'https://www.googleapis.com/auth/drive.file', + ] +}); ``` -
    +### Connecting to Documents + +To connect to existing documents, the document ID must be specified. This ID can +be found from the edit URL. + +The edit URL starts with `https://docs.google.com/spreadsheets/d/` and includes +`/edit`. The ID is the string of characters between the slashes. For example: + +``` +https://docs.google.com/spreadsheets/d/a_long_string_of_characters/edit#gid=0 +---------------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^--- ID +``` + +The `GoogleSpreadsheet` constructor accepts a document ID and auth object: + +```js title="Connect to a document" +const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt); +await doc.loadInfo(); +``` + +### Creating New Documents + +The `createNewSpreadsheetDocument` makes a request to create a new document. It +is strongly recommended to create a blank sheet. + +```js title="Create a new document with blank sheet" +const doc = await GoogleSpreadsheet.createNewSpreadsheetDocument(jwt, { title: 'Document Title' }); +const newSheet = await doc.addSheet({ title: 'Sheet Title' }); +``` + +### Array of Arrays + +["Arrays of Arrays"](/docs/api/utilities/array#array-of-arrays) are the main +data format for interchange with Google Sheets. The outer array object includes +row arrays, and each row array includes data. + +SheetJS provides methods for working with Arrays of Arrays: + +- `aoa_to_sheet`[^3] creates SheetJS worksheet objects from arrays of arrays +- `sheet_to_json`[^4] can generate arrays of arrays from SheetJS worksheets ## Export Document Data The goal is to create an XLSB export from a Google Sheet. Google Sheets does not natively support the XLSB format. SheetJS fills the gap. -
    How to run locally (click to show) - -0) Follow the [Initial Configuration](#initial-configuration). - -1) Create a new Google Sheet and share with the generated service account. It -should be granted the "Editor" role - -2) Install the dependencies: - -{`\ -npm i -S https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz google-spreadsheet@3.3.0`} - - -2) Save the following snippet to `common.js`: - -```js title=common.js -const fs = require("fs"); -const { GoogleSpreadsheet } = require('google-spreadsheet'); - -module.exports = async(ID) => { - /* get credentials */ - const creds = JSON.parse(fs.readFileSync('key.json')); - - /* initialize sheet and authenticate */ - const doc = new GoogleSpreadsheet(ID); - await doc.useServiceAccountAuth(creds); - await doc.loadInfo(); - return doc; -} -``` - -3) Save the following snippet to `pull.js`: - -```js title=pull.js -const XLSX = require("xlsx"); - -/* create a blank workbook */ -const wb = XLSX.utils.book_new(); - -const init = require("./common"); -const ID = ""; - -(async() => { - - const doc = await init(ID); - - for(let i = 0; i < doc.sheetsByIndex.length; ++i) { - const sheet = doc.sheetsByIndex[i]; - const name = sheet.title; - - /* get the header and data rows */ - await sheet.loadHeaderRow(); - const header = sheet.headerValues; - const rows = await sheet.getRows(); - const aoa = [header].concat(rows.map(r => r._rawData)); - - /* generate a SheetJS Worksheet */ - const ws = XLSX.utils.aoa_to_sheet(aoa); - - /* add to workbook */ - XLSX.utils.book_append_sheet(wb, ws, name); - } - - /* write to SheetJS.xlsb */ - XLSX.writeFile(wb, "SheetJS.xlsb"); - -})(); -``` - -4) Replace `` with the ID of the actual document. - -5) Run `node pull.js` once. It will create `SheetJS.xlsb`. - -6) Open `SheetJS.xlsb` and confirm the contents are the same as Google Sheets. - -7) Change some cells in the Google Sheets Document. - -8) Run `node pull.js` again and reopen `SheetJS.xlsb` to confirm value changes. - -
    - ### Convert a Single Sheet The idea is to extract the raw data from the Google Sheet headers and combine @@ -196,115 +183,8 @@ async function doc_to_wb(doc) { ## Update Document Data -The goal is to refresh a Google Sheet based on a local file. - -
    How to run locally (click to show) - -0) Follow the [Initial Configuration](#initial-configuration). - -1) Create a new Google Sheet and share with the generated service account. It -should be granted the "Editor" role - -2) Install the dependencies: - -{`\ -npm i -S https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz google-spreadsheet@3.3.0`} - - -2) Save the following snippet to `common.js`: - -```js title=common.js -const fs = require("fs"); -const { GoogleSpreadsheet } = require('google-spreadsheet'); - -module.exports = async(ID) => { - /* get credentials */ - const creds = JSON.parse(fs.readFileSync('key.json')); - - /* initialize sheet and authenticate */ - const doc = new GoogleSpreadsheet(ID); - await doc.useServiceAccountAuth(creds); - await doc.loadInfo(); - return doc; -} -``` - -3) Save the following snippet to `push.js`: - -```js title=push.js -const XLSX = require("xlsx"); -const fs = require("fs"); -/* create dummy worksheet if `sheetjs.xlsx` does not exist */ -if(!fs.existsSync("sheetjs.xlsx")) { - const wb = XLSX.utils.book_new(); - const ws1 = XLSX.utils.aoa_to_sheet([["a","b","c"],[1,2,3]]); XLSX.utils.book_append_sheet(wb, ws1, "Sheet1"); - const ws2 = XLSX.utils.aoa_to_sheet([["a","b","c"],[4,5,6]]); XLSX.utils.book_append_sheet(wb, ws2, "Sheet2"); - XLSX.writeFile(wb, "sheetjs.xlsx"); -} -/* read and parse sheetjs.xlsx */ -const wb = XLSX.readFile("sheetjs.xlsx"); - -const init = require("./common"); -const ID = ""; - -(async() => { - - const doc = await init(ID); - - /* clear workbook */ - { - /* delete all sheets after the first sheet */ - const old_sheets = doc.sheetsByIndex; - for(let i = 1; i < old_sheets.length; ++i) { - await old_sheets[i].delete(); - } - /* clear first worksheet */ - old_sheets[0].clear(); - } - - /* write worksheets */ - { - const name = wb.SheetNames[0]; - const ws = wb.Sheets[name]; - /* first worksheet already exists */ - const sheet = doc.sheetsByIndex[0]; - - /* update worksheet name */ - await sheet.updateProperties({title: name}); - - /* generate array of arrays from the first worksheet */ - const aoa = XLSX.utils.sheet_to_json(ws, {header: 1}); - - /* set document header row to first row of the AOA */ - await sheet.setHeaderRow(aoa[0]) - - /* add the remaining rows */ - await sheet.addRows(aoa.slice(1)); - - /* the other worksheets must be created manually */ - for(let i = 1; i < wb.SheetNames.length; ++i) { - const name = wb.SheetNames[i]; - const ws = wb.Sheets[name]; - - const sheet = await doc.addSheet({title: name}); - const aoa = XLSX.utils.sheet_to_json(ws, {header: 1}); - await sheet.setHeaderRow(aoa[0]) - await sheet.addRows(aoa.slice(1)); - } - } - -})(); -``` - -4) Replace `` with the ID of the actual document. - -5) Run `node push.js` once. It will create `sheetjs.xlsx` and update the sheet. - -6) Edit `sheetjs.xlsx` with some new data - -7) Run `node push.js` again and watch the Google Sheet update! - -
    +The goal is to import data from a NUMBERS file to Google Sheets. Google Sheets +does not natively support the NUMBERS format. SheetJS fills the gap. ### Clear the Document @@ -428,75 +308,476 @@ async function sheet_load_from_ws(sheet, ws) { ## Raw File Exports -`node-google-spreadsheet` can download the XLSX or ODS export of the document. -The functions return NodeJS `Buffer` data that can be parsed using SheetJS. +In the web interface, Google Sheets can export documents to `XLSX` or `ODS`. The +NodeJS library includes similar methods to perform the download[^5]: -This example prints the worksheet names and CSV exports of each sheet. +| Format | Google Sheets Description | Method | +|:-------|:--------------------------|:-----------------| +| XLSX | Microsoft Excel (.xlsx) | `downloadAsXLSX` | +| ODS | OpenDocument (.ods) | `downloadAsODS` | - - +The functions resolve to `Buffer` data. The `Buffer` objects can be parsed using +the SheetJS `read`[^6] method: ```js -const XLSX = require("xlsx"); +/* download XLSX */ +const ab = await doc.downloadAsXLSX(); -(async() => { - /* Connect to Google Sheet */ - const ID = ""; - const doc = await require("./common")(ID); - - /* Get file export */ - // highlight-next-line - const buf = await doc.downloadAsXLSX(); - - /* Parse with SheetJS */ - const wb = XLSX.read(buf); - - /* Loop over the worksheet names */ - wb.SheetNames.forEach(name => { - /* Print the name to the console */ - console.log(name); - - /* Get the corresponding worksheet object */ - const sheet = wb.Sheets[name]; - - /* Print a CSV export of the worksheet */ - console.log(XLSX.utils.sheet_to_csv(sheet)); - }); -})(); +/* parse */ +const wb = XLSX.read(buf); ``` - +At this point `wb` is a SheetJS workbook object[^7]. - +## Complete Example -```js -const XLSX = require("xlsx"); +:::note -(async() => { - /* Connect to Google Sheet */ - const ID = ""; - const doc = await require("./common")(ID); +This demo was last tested on 2023 September 17 using `google-auth-library` for +authentication (`v8.9.0`) and `google-spreadsheet` for API access (`v4.1.0`). - /* Get file export */ - // highlight-next-line - const buf = await doc.downloadAsODS(); +::: - /* Parse with SheetJS */ - const wb = XLSX.read(buf); +### Account Setup - /* Loop over the worksheet names */ - wb.SheetNames.forEach(name => { - /* Print the name to the console */ - console.log(name); +0) Create a new Google account or log into an existing account. - /* Get the corresponding worksheet object */ - const sheet = wb.Sheets[name]; +:::caution pass - /* Print a CSV export of the worksheet */ - console.log(XLSX.utils.sheet_to_csv(sheet)); - }); -})(); +A valid phone number (for SMS verification) may be required. + +::: + +1) Open in a web browser. Review the Google +Cloud Platform Terms of Service. + +:::warning pass + +You must agree to the Google Cloud Platform Terms of Service to use the APIs. + +::: + +### Project Setup + +2) Create a new Project. + +If the account does not have an existing project, click "CREATE PROJECT" + +If the account has an existing project, click the project selector (`:·` icon) +and click "NEW PROJECT" in the modal. + +3) In the New Project screen, enter "SheetJS Test" in the Project name textbox +and select "No organization" in the Location box. Click "CREATE" + +### API Setup + +:::info pass + +The goal of this section is to enable Google Sheets API and Google Drive API. + +::: + +4) Click the Project Selector (`:·` icon) and select "SheetJS Test" + +5) In the left sidebar, click "Enabled APIs and services". + +6) Near the top of the page, click "+ ENABLE APIS AND SERVICES". + +7) In the search bar near the middle of the page, type "Sheets" and look for +"Google Sheets API". Click the card + +8) In the Product Details screen, click the blue "ENABLE" button. + +9) Click the left arrow (`<-`) next to API/Service details. + +10) In the search bar near the middle of the page, type "Drive" and look for +"Google Drive API". Click the card. + +11) In the Product Details screen, click the blue "ENABLE" button. + +### Service Account Setup + +:::info pass + +The goal of this section is to create a service account and generate a JSON key. + +::: + +#### Create Service Account + +12) Go to . + +13) Click the Project Selector (`:·` icon) and select "SheetJS Test". + +14) Click "Dashboard". + +15) In the left sidebar, hover over "APIs and Services" and select "Credentials" + +16) Click "+ CREATE CREDENTIALS". In the dropdown, select "Service Account" + +17) Enter "SheetJService" for Service account name. Click "CREATE AND CONTINUE" + +:::note pass + +The Service account ID is generated automatically. + +::: + +18) In Step 2 "Grant this service account access to project", click CONTINUE + +19) In Step 3 click "DONE". You will be taken back to the credentials screen + +#### Create JSON Key + +20) Look for "SheetJService" in the "Service Accounts" table and click the email +address in the row + +21) Click the email address of the account in the "Service Accounts" table. + +22) Click "KEYS" in the horizontal bar near the top of the page. + +23) Click "ADD KEY" and select "Create new key" in the dropdown. + +24) In the popup, select the "JSON" radio button and click "CREATE". The page +will download a JSON file. + +25) Click "CLOSE" + +### Create Document + +:::info pass + +The goal of this section is to create a document from the service account and +share with the main account. + +::: + +26) Create a `SheetJSGS` folder and initialize: + +```bash +mkdir SheetJSGS +cd SheetJSGS +npm init -y ``` - - \ No newline at end of file +27) Copy the JSON file from step 24 into the project folder. + +28) Install dependencies: + +{`\ +npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz google-spreadsheet google-auth-library`} + + +29) Save the following script to `init.mjs`: + +```js title="init.mjs" +import { JWT } from 'google-auth-library' +import { GoogleSpreadsheet } from 'google-spreadsheet'; + +// highlight-next-line +import creds from './sheetjs-test-726272627262.json' assert { type: "json" }; + +const jwt = new JWT({ + email: creds.client_email, + key: creds.private_key, + scopes: [ + 'https://www.googleapis.com/auth/spreadsheets', + 'https://www.googleapis.com/auth/drive.file', + ] +}); + +const doc = await GoogleSpreadsheet.createNewSpreadsheetDocument(jwt, { title: 'test from NodeJS' }); +const newSheet = await doc.addSheet({ title: 'SheetJSTest' }); +// highlight-next-line +await doc.share('YOUR_ADDRESS@gmail.com'); +``` + +Edit the highlighted lines as follows: + +- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the +JSON file in step 27. The `./` prefix is required! + +- `'YOUR_ADDRESS@gmail.com'` should be replaced with the Google Account email +address from step 0. + +30) Run the script: + +```bash +node init.mjs +``` + +31) Sign into Google Sheets. A shared document "test from NodeJS" should be +displayed in the table. It will be owned by the service account. + +32) Open the shared document from step 31. + +33) Copy the URL and extract the document ID. + +The URL of the document will look like + +``` +https://docs.google.com/spreadsheets/d/a_long_string_of_characters/edit#gid=0 +---------------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^--- ID +``` + +The ID is a long string of letters and numbers and underscore characters (`_`) +just before the `/edit` part of the URL. + +### Load Data from NUMBERS + +:::info pass + +The goal of this section is to update the new document with data from a sample +NUMBERS file. + +::: + +34) Download the [test file `pres.numbers`](https://sheetjs.com/pres.numbers): + +```bash +curl -LO https://sheetjs.com/pres.numbers +``` + +35) Save the following script to `load.mjs`: + +```js title="load.mjs" +import { JWT } from 'google-auth-library' +import { GoogleSpreadsheet } from 'google-spreadsheet'; + +import { set_fs, readFile, utils } from 'xlsx'; +import * as fs from 'fs'; +set_fs(fs); + +// highlight-next-line +import creds from './sheetjs-test-726272627262.json' assert { type: "json" }; + +const jwt = new JWT({ + email: creds.client_email, + key: creds.private_key, + scopes: [ + 'https://www.googleapis.com/auth/spreadsheets', + 'https://www.googleapis.com/auth/drive.file', + ] +}); + +// highlight-next-line +const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt); +await doc.loadInfo(); + +const wb = readFile("pres.numbers"); + +/* clear workbook */ +{ + /* delete all sheets after the first sheet */ + const old_sheets = doc.sheetsByIndex; + for(let i = 1; i < old_sheets.length; ++i) { + await old_sheets[i].delete(); + } + /* clear first worksheet */ + old_sheets[0].clear(); +} + +/* write worksheets */ +{ + const name = wb.SheetNames[0]; + const ws = wb.Sheets[name]; + /* first worksheet already exists */ + const sheet = doc.sheetsByIndex[0]; + + /* update worksheet name */ + await sheet.updateProperties({title: name}); + + /* generate array of arrays from the first worksheet */ + const aoa = utils.sheet_to_json(ws, {header: 1}); + + /* set document header row to first row of the AOA */ + await sheet.setHeaderRow(aoa[0]) + + /* add the remaining rows */ + await sheet.addRows(aoa.slice(1)); + + /* the other worksheets must be created manually */ + for(let i = 1; i < wb.SheetNames.length; ++i) { + const name = wb.SheetNames[i]; + const ws = wb.Sheets[name]; + + const sheet = await doc.addSheet({title: name}); + const aoa = utils.sheet_to_json(ws, {header: 1}); + await sheet.setHeaderRow(aoa[0]) + await sheet.addRows(aoa.slice(1)); + } +} +``` + +Edit the highlighted lines as follows: + +- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the +JSON file in step 27. The `./` prefix is required! + +- `'DOCUMENT_ID'` should be replaced with the Document ID from step 33. + +36) Run the script: + +```bash +node load.mjs +``` + +37) Sign into Google Sheets and open the "test from NodeJS" shared document. It +should show a list of Presidents, matching the contents of the test file. + +### Export Data to XLSB + +:::info pass + +The goal of this section is to export the raw data from Google Sheets to XLSB. + +::: + +38) Save the following script to `dump.mjs`: + +```js title="dump.mjs" +import { JWT } from 'google-auth-library' +import { GoogleSpreadsheet } from 'google-spreadsheet'; + +import { set_fs, writeFile, utils } from 'xlsx'; +import * as fs from 'fs'; +set_fs(fs); + +// highlight-next-line +import creds from './sheetjs-test-726272627262.json' assert { type: "json" }; + +const jwt = new JWT({ + email: creds.client_email, + key: creds.private_key, + scopes: [ + 'https://www.googleapis.com/auth/spreadsheets', + 'https://www.googleapis.com/auth/drive.file', + ] +}); + +// highlight-next-line +const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt); +await doc.loadInfo(); + +const wb = utils.book_new(); + +for(let i = 0; i < doc.sheetsByIndex.length; ++i) { + const sheet = doc.sheetsByIndex[i]; + const name = sheet.title; + + /* get the header and data rows */ + await sheet.loadHeaderRow(); + const header = sheet.headerValues; + const rows = await sheet.getRows(); + const aoa = [header].concat(rows.map(r => r._rawData)); + + /* generate a SheetJS Worksheet */ + const ws = utils.aoa_to_sheet(aoa); + + /* add to workbook */ + utils.book_append_sheet(wb, ws, name); +} + +/* write to SheetJS.xlsb */ +writeFile(wb, "SheetJS.xlsb"); +``` + +Edit the highlighted lines as follows: + +- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the +JSON file in step 27. The `./` prefix is required! + +- `'DOCUMENT_ID'` should be replaced with the Document ID from step 33. + +39) Run the script: + +```bash +node load.mjs +``` + +The script should create a file `SheetJS.xlsb` in the project folder. This file +can be opened in Excel + +40) Sign into Google Sheets and open the "test from NodeJS" shared document. It +should show a list of Presidents, matching the contents of the test file. + +### Export Raw Files + +:::info pass + +The goal of this section is to parse the Google Sheets XLSX export and generate +CSV files for each worksheet. + +::: + +41) Sign into Google Sheets and open the "test from NodeJS" shared document. + +42) Click the Plus (`+`) icon in the lower left corner to create a new worksheet. + +43) In the new worksheet, set cell A1 to the formula `=SEQUENCE(3,5)`. This will +assign a grid of values + +44) Save the following script to `raw.mjs`: + +```js title="raw.mjs" +import { JWT } from 'google-auth-library' +import { GoogleSpreadsheet } from 'google-spreadsheet'; + +import { read, utils } from 'xlsx'; + +// highlight-next-line +import creds from './sheetjs-test-726272627262.json' assert { type: "json" }; + +const jwt = new JWT({ + email: creds.client_email, + key: creds.private_key, + scopes: [ + 'https://www.googleapis.com/auth/spreadsheets', + 'https://www.googleapis.com/auth/drive.file', + ] +}); + +// highlight-next-line +const doc = new GoogleSpreadsheet('DOCUMENT_ID', jwt); +await doc.loadInfo(); + +const buf = await doc.downloadAsXLSX(); + +/* Parse with SheetJS */ +const wb = read(buf); + +/* Loop over the worksheet names */ +wb.SheetNames.forEach(name => { + /* Print the name to the console */ + console.log(name); + + /* Get the corresponding worksheet object */ + const sheet = wb.Sheets[name]; + + /* Print a CSV export of the worksheet */ + console.log(utils.sheet_to_csv(sheet)); +}); +``` + +Edit the highlighted lines as follows: + +- `'./sheetjs-test-726272627262.json'` should be replaced with the name of the +JSON file in step 27. The `./` prefix is required! + +- `'DOCUMENT_ID'` should be replaced with the Document ID from step 33. + +45) Run the script: + +```bash +node raw.mjs +``` + +The script will display the sheet names and CSV rows from both worksheets. + +[^1]: The package name is [`google-auth-library`](https://www.npmjs.com/package/google-auth-library) +[^2]: The project name is [`node-google-spreadsheet`](https://github.com/theoephraim/node-google-spreadsheet) but the module name is `google-spreadsheet`. +[^3]: See [`aoa_to_sheet` in "Utilities"](/docs/api/utilities/array#array-of-arrays-input) +[^4]: See [`sheet_to_json` in "Utilities"](/docs/api/utilities/array#array-output) +[^5]: See ["Exporting Data"](https://theoephraim.github.io/node-google-spreadsheet/#/guides/exports) in the `node-google-spreadsheet` documentation +[^6]: See [`read` in "Reading Files"](/docs/api/parse-options) +[^7]: See ["Workbook Object"](/docs/csf/book) for a description of the workbook object or ["API Reference"](/docs/api) for various methods to work with workbook and sheet objects. diff --git a/docz/docs/03-demos/32-extensions/04-gsheet.md b/docz/docs/03-demos/32-extensions/04-gsheet.md index f0a65c7..1270bea 100644 --- a/docz/docs/03-demos/32-extensions/04-gsheet.md +++ b/docz/docs/03-demos/32-extensions/04-gsheet.md @@ -7,7 +7,7 @@ pagination_next: demos/bigdata/index import current from '/version.js'; import CodeBlock from '@theme/CodeBlock'; -:::note +:::note pass This demo focuses on Google Apps Script custom functions. For external data processing, [the "Google Sheets" cloud data demo](/docs/demos/cloud/gsheet) @@ -150,7 +150,7 @@ curl -LO https://cdn.sheetjs.com/xlsx-${current}/package/dist/xlsx.full.min.js`} npx @google/clasp push ``` -:::caution +:::caution pass If the Google Apps Script API is not enabled, the command will display an object with `code: 403` and an error message about the Apps Script API: diff --git a/docz/docs/03-demos/37-bigdata/01-stream.md b/docz/docs/03-demos/37-bigdata/01-stream.md index 017a396..5efcaee 100644 --- a/docz/docs/03-demos/37-bigdata/01-stream.md +++ b/docz/docs/03-demos/37-bigdata/01-stream.md @@ -59,7 +59,7 @@ NodeJS push streams were introduced in 2012. The text streaming methods `to_csv` and `to_html` are supported in NodeJS v0.10 and later while the object streaming method `to_json` is supported in NodeJS v0.12 and later. -The first streaming write function, `to_csv`, was introduced in April 2017. It +The first streaming write function, `to_csv`, was introduced in early 2017. It used and still uses the same NodeJS streaming API. Years later, browser vendors are settling on a different stream API. diff --git a/docz/docusaurus.config.js b/docz/docusaurus.config.js index dd54aa6..c283b59 100644 --- a/docz/docusaurus.config.js +++ b/docz/docusaurus.config.js @@ -214,6 +214,7 @@ const config = { { from: '/docs/demos/clipboard', to: '/docs/demos/local/clipboard/' }, { from: '/docs/demos/localfile', to: '/docs/demos/local/file/' }, { from: '/docs/demos/data/indexeddb', to: '/docs/demos/local/indexeddb/' }, + { from: '/docs/demos/data/storageapi', to: '/docs/demos/local/storageapi/' }, /* desktop */ { from: '/docs/demos/cli', to: '/docs/demos/desktop/cli/' }, { from: '/docs/getting-started/demos/cli', to: '/docs/demos/desktop/cli/' }, diff --git a/docz/static/quasar/and.png b/docz/static/quasar/and.png index 043b5aa..38e606c 100644 Binary files a/docz/static/quasar/and.png and b/docz/static/quasar/and.png differ diff --git a/docz/static/quasar/ios.png b/docz/static/quasar/ios.png index b15e75f..790b218 100644 Binary files a/docz/static/quasar/ios.png and b/docz/static/quasar/ios.png differ
    iOS