From 584af134dbe7f0fb1d9d0f3b98c8596555fa9f28 Mon Sep 17 00:00:00 2001 From: SheetJS Date: Wed, 23 Aug 2023 04:05:30 -0400 Subject: [PATCH] standalone --- .../01-installation/01-standalone.mdx | 69 +++++++++-- .../03-demos/06-desktop/06-reactnative.md | 2 +- docz/docs/03-demos/07-data/26-redis.md | 115 +++++++++++++----- docz/docs/09-miscellany/05-contributing.md | 4 +- 4 files changed, 146 insertions(+), 44 deletions(-) diff --git a/docz/docs/02-getting-started/01-installation/01-standalone.mdx b/docz/docs/02-getting-started/01-installation/01-standalone.mdx index 0edbfab..bcf7d49 100644 --- a/docz/docs/02-getting-started/01-installation/01-standalone.mdx +++ b/docz/docs/02-getting-started/01-installation/01-standalone.mdx @@ -28,14 +28,13 @@ new versions are released! ::: -:::warning +:::warning pass A number of services host older versions of the SheetJS libraries. Due to syncing issues, they are generally out of date. -They are known CDN bugs. - - is the authoritative source for SheetJS modules. +**The SheetJS CDN** **is the authoritative source** +**for SheetJS scripts** ::: @@ -67,6 +66,23 @@ scratch, a single script tag should be added at the top of the HTML page: +### Vendoring + +For general stability, "vendoring" scripts is the recommended approach: + +

1) Download the script (xlsx.full.min.js) for + the desired version. The current version is available at https://cdn.sheetjs.com/xlsx-{current}/package/dist/xlsx.full.min.js

+ +2) Move the script to a `public` folder with other scripts. + +3) Reference the local script from HTML pages: + +```html + +``` + +This script assigns to `window.XLSX`. The global can be used in other scripts. + ### Internet Explorer and Older Browsers For broad compatibility with JavaScript engines, the library is written using @@ -104,11 +120,11 @@ importScripts("https://cdn.sheetjs.com/xlsx-${current}/package/dist/xlsx.full.mi ## ECMAScript Module Imports -:::caution +:::caution pass This section refers to imports using `script type="module"`. For imports in modern projects using Webpack or React or Angular or VueJS, the installation is -described [in the next section](/docs/getting-started/installation/frameworks). +described [in "Frameworks and Bundlers"](/docs/getting-started/installation/frameworks). ::: @@ -132,15 +148,19 @@ set_cptable(cptable); `} -Dynamic imports with `import()` can be used in data export scenarios. This -example will download the library only when the export button is pressed: +Web Worker support is noted in [the "Web Workers" demo](/docs/demos/bigdata/worker#installation) + +### Dynamic Imports + +Dynamic imports with `import()` will only download the SheetJS scripts when they +are used. This example will download the library when data is exported: {`\ `} -Web Worker support is noted in [the demo](/docs/demos/bigdata/worker#installation) +:::caution pass + +The callback functions must be marked as `async` and the script block must have +the attribute `type="module"` + +::: + +If Encoding support is required, `cpexcel.full.mjs` must be manually imported: + +{`\ + +`} + ## Bower -:::caution +:::warning pass Bower is deprecated and the maintainers recommend using other tools. diff --git a/docz/docs/03-demos/06-desktop/06-reactnative.md b/docz/docs/03-demos/06-desktop/06-reactnative.md index 5af0591..e343c4d 100644 --- a/docz/docs/03-demos/06-desktop/06-reactnative.md +++ b/docz/docs/03-demos/06-desktop/06-reactnative.md @@ -51,7 +51,7 @@ This demo was tested in the following environments: | Windows 10 | x64 | `v0.71.25` | 2023-07-24 | | Windows 11 | x64 | `v0.71.11` | 2023-05-11 | | MacOS 12.6 | x64 | `v0.71.26` | 2023-07-23 | -| MacOS 13.4 | arm | `v0.71.18` | 2023-07-06 | +| MacOS 13.4 | ARM | `v0.71.18` | 2023-07-06 | ::: diff --git a/docz/docs/03-demos/07-data/26-redis.md b/docz/docs/03-demos/07-data/26-redis.md index d78de7a..d5b71c5 100644 --- a/docz/docs/03-demos/07-data/26-redis.md +++ b/docz/docs/03-demos/07-data/26-redis.md @@ -1,5 +1,7 @@ --- -title: Redis +title: Redis Databases Wrapped in Sheets +sidebar_label: Redis +description: Store complex datasets in Redis. Seamlessly save data to spreadsheets and read data from sheets using SheetJS. Enable Excel spreadsheet experts to update content. pagination_prev: demos/desktop/index pagination_next: demos/local/index sidebar_custom_props: @@ -9,6 +11,18 @@ sidebar_custom_props: import current from '/version.js'; import CodeBlock from '@theme/CodeBlock'; +[Redis](https://redis.io/) is a open source in-memory data store. It is capable +of storing sets, lists and other simple data structures. + +[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing +data from spreadsheets. + +This demo defines a schema for storing Redis databases in spreadsheets. We'll +explore how to use SheetJS and Redis NodeJS connector modules to pull data +from XLSX files to a Redis database and to serialize a database to a workbook. + +#### Overview + Redis has 5 core data types: "String", List", "Set", "Sorted Set", and "Hash". Since the keys and values are limited to simple strings (and numbers), it is possible to store complete databases in a single worksheet. @@ -17,7 +31,7 @@ possible to store complete databases in a single worksheet. ## Integration Details -:::note +:::note pass [`SheetJSRedis.mjs`](pathname:///nosql/SheetJSRedis.mjs) exports the methods: - `redis_to_ws` creates a SheetJS worksheet by querying a redis client @@ -25,11 +39,32 @@ possible to store complete databases in a single worksheet. ::: -The first row holds the data type and the second row holds the property name. +### Array of Arrays -The "Exporting Data" snippets generate arrays of arrays that can be added to a -worksheet using `sheet_add_aoa`. Since the data is column-oriented, the goal is -to add the data starting in the first row of the column after the data: +The shared data representation is an "array of arrays"[^1]. Each array within +the structure corresponds to one row. + +The Redis to SheetJS converter generates an array of arrays of the data by +running queries to fetch data from the database. The SheetJS `aoa_to_sheet` and +`sheet_add_aoa`[^2] methods build up worksheets from arrays of arrays. Once the +worksheet is created, it can be added to a SheetJS workbook object[^3] and +exported using `writeFile`[^4]. + +The SheetJS to Redis converter works in reverse. Workbook files are parsed with +the SheetJS `readFile` method[^5] and the desired worksheet is pulled from the +workbook object. An array of arrays can be created with the `sheet_to_json`[^6] +utility function. The data structure can be scanned to generate Redis queries. + +### Appending Columns + +Since the data is column-oriented, the goal is to add the data starting on the +first row of the column after the data. + +To calculate the starting point for writing data, SheetJS provides `encode_cell` +and `decode_range` utility functions for working with addresses and ranges[^7]. + +The following snippet takes an array of arrays of values and writes the values +to a worksheet starting from the column after the worksheet range: ```js function add_aoa_to_next_column(worksheet, aoa) { @@ -45,14 +80,7 @@ function add_aoa_to_next_column(worksheet, aoa) { } ``` -The "Importing Data" snippets generate redis queries. The `ws_to_redis` function -first generates an array of arrays with `sheet_to_json`: - -```js -const aoa = XLSX.utils.sheet_to_json(worksheet, { header: 1 }); -``` - -#### Strings +### Strings Strings can be stored in a unified String table. The first column holds keys and the second column holds values: @@ -67,7 +95,11 @@ XXX| A | B | ``` The SheetJS array-of-arrays representation of the string table is an array of -key/value pairs: +key/value pairs. + +The pairs can be generated from Redis by querying for all of the keys using the +`KEYS`[^8] method, testing if the corresponding value is a string using the +`TYPE`[^9] method, and fetching string values using the `GET`[^10] method: ```js const aoa = ["Strings"]; aoa.length = 2; // [ "Strings", empty ] @@ -80,7 +112,7 @@ for(let key of keys) { } ``` -#### Lists +### Lists Lists are unidimensional and can be stored in their own columns. @@ -94,15 +126,18 @@ XXX| C | ``` The SheetJS array-of-arrays representation of lists is a column of values. -`LRANGE` returns a simple array of values. `sheet_add_aoa` interprets the result -as one row. The code transposes the result with `values.map(v => [v])`. + +Redis `LRANGE`[^11] returns a simple array of values. `sheet_add_aoa` interprets +the result as one row, so the data should be transposed. + +The code transposes the result with `values.map(v => [v])`. ```js const values = await client.LRANGE(key, 0, -1); const aoa = [ ["List"], [key] ].concat(values.map(v => [v])); ``` -#### Sets +### Sets Sets are unidimensional and can be stored in their own columns. @@ -116,15 +151,18 @@ XXX| D | ``` The SheetJS array-of-arrays representation of sets is a column of values. -`SMEMBERS` returns a simple array of values. `sheet_add_aoa` interprets result -as one row. The code transposes the result with `values.map(v => [v])`. + +Redis `SMEMBERS`[^12] returns an array of values. `sheet_add_aoa` interprets the +result as one row, so the data should be transposed. + +The code transposes the result with `values.map(v => [v])`. ```js const values = await client.SMEMBERS(key); const aoa = [ ["Set"], [key] ].concat(values.map(v => [v])); ``` -#### Sorted Sets +### Sorted Sets Sorted Sets have an associated score which can be stored in the second column. @@ -138,14 +176,15 @@ XXX| E | F | ``` The SheetJS array-of-arrays representation is an array of key/score pairs. -`ZRANGE_WITHSCORES` returns an array of objects which can be reshaped. + +`ZRANGE_WITHSCORES`[^13] returns an array of objects which can be reshaped. ```js const values = await client.ZRANGE_WITHSCORES(key, 0, -1); const aoa = [ ["Sorted"], [key] ].concat(values.map(v => [v.value, v.score])); ``` -#### Hashes +### Hashes Hashes are stored like the string table, with key and value columns in order. @@ -159,7 +198,8 @@ XXX| G | H | ``` The SheetJS array-of-arrays representation is an array of key/value pairs. -`HGETALL` returns a plain object which can be converted using `Object.entries`: + +`HGETALL`[^14] returns an object which can be converted using `Object.entries`: ```js const values = await client.HGETALL(key); @@ -170,8 +210,8 @@ const aoa = [ ["Hash"], [key] ].concat(Object.entries(values)); :::note -This demo was last tested on 2023 May 11 with Redis 7.0.11, Redis connector -module 4.6.6 and NodeJS 20.1.0. +This demo was last tested on 2023 August 22 with Redis 7.2.0, Redis connector +module 4.6.7 and NodeJS 20.5.1. ::: @@ -187,10 +227,10 @@ this demo also requires NodeJS version 18 or later. :::note -This demo was last tested on Intel macOS. Redis was installed with: +This demo was last tested on macOS. Redis was installed with: ```bash -brew install redis@7.0.11 +brew install redis ``` The following command started the server process: @@ -214,7 +254,7 @@ curl -LO https://docs.sheetjs.com/nosql/SheetJSRedisTest.mjs 2) Install dependencies: {`\ -npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz redis@4.6.6`} +npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz redis@4.6.7`} 3) Run the test script: @@ -226,3 +266,18 @@ node SheetJSRedisTest.mjs Inspect the output and compare with the data in `SheetJSRedisTest.mjs`. Open `SheetJSRedis.xlsx` and verify the columns have the correct data + +[^1]: See ["Array of Arrays" in "Utility Functions"](/docs/api/utilities/array#array-of-arrays) +[^2]: See ["Array of Arrays Input" in "Utility Functions"](/docs/api/utilities/array#array-of-arrays-input). +[^3]: See ["Workbook Helpers" in "Utility Functions"](/docs/api/utilities/wb) +[^4]: See [`writeFile` in "Writing Files"](/docs/api/write-options) +[^5]: See [`readFile` in "Reading Files"](/docs/api/parse-options) +[^6]: See [`sheet_to_json` in "Utilities"](/docs/api/utilities/array#array-output) +[^7]: See ["Utilities" in "Addresses and Ranges"](/docs/csf/general#utilities) +[^8]: See [`KEYS`](https://redis.io/commands/keys/) in the Redis documentation. +[^9]: See [`TYPE`](https://redis.io/commands/type/) in the Redis documentation. +[^10]: See [`GET`](https://redis.io/commands/get/) in the Redis documentation. +[^11]: See [`LRANGE`](https://redis.io/commands/lrange/) in the Redis documentation. +[^12]: See [`SMEMBERS`](https://redis.io/commands/smembers/) in the Redis documentation. +[^13]: The official command is [`ZRANGE`](https://redis.io/commands/zrange/). `ZRANGE_WITHSCORES` is a special command supported by the NodeJS wrapper. +[^14]: See [`HGETALL`](https://redis.io/commands/hgetall/) in the Redis documentation. diff --git a/docz/docs/09-miscellany/05-contributing.md b/docz/docs/09-miscellany/05-contributing.md index 5bc64b0..d9c2bff 100644 --- a/docz/docs/09-miscellany/05-contributing.md +++ b/docz/docs/09-miscellany/05-contributing.md @@ -40,9 +40,9 @@ These instructions were tested on the following platforms: | Platform | Test Date | |:------------------------------|:-----------| | Linux (Steam Deck Holo 3.4.8) | 2023-07-12 | -| Linux (Ubuntu 18.04 aarch64) | 2023-04-13 | +| Linux (Ubuntu 18.04 AArch64) | 2023-04-13 | | MacOS 10.13 (x64) | 2023-04-04 | -| MacOS 13.0 (arm64) | 2023-04-13 | +| MacOS 13.0 (ARM64) | 2023-04-13 | | Windows 10 (x64) + WSL Ubuntu | 2023-07-23 | | Windows 11 (x64) + WSL Ubuntu | 2023-04-04 |