diff --git a/docz/docs/02-getting-started/01-installation/02-frameworks.md b/docz/docs/02-getting-started/01-installation/02-frameworks.md index d739e32..e2d33ea 100644 --- a/docz/docs/02-getting-started/01-installation/02-frameworks.md +++ b/docz/docs/02-getting-started/01-installation/02-frameworks.md @@ -65,7 +65,7 @@ Snyk security tooling may report errors involving "Prototype Pollution": Prototype Pollution [Medium Severity][https://security.snyk.io/vuln/SNYK-JS-XLSX-5457926] ``` -As noted in the [Snyk report](https://security.snyk.io/vuln/SNYK-JS-XLSX-5457926): +As noted in the [Snyk report](https://web.archive.org/web/20230920204324/https://security.snyk.io/vuln/SNYK-JS-XLSX-5457926): > The issue is resolved in version 0.19.3 diff --git a/docz/docs/02-getting-started/01-installation/03-nodejs.md b/docz/docs/02-getting-started/01-installation/03-nodejs.md index ab1cc8d..d1dfd8f 100644 --- a/docz/docs/02-getting-started/01-installation/03-nodejs.md +++ b/docz/docs/02-getting-started/01-installation/03-nodejs.md @@ -55,7 +55,7 @@ Snyk security tooling may report errors involving "Prototype Pollution": Prototype Pollution [Medium Severity][https://security.snyk.io/vuln/SNYK-JS-XLSX-5457926] ``` -As noted in the [Snyk report](https://security.snyk.io/vuln/SNYK-JS-XLSX-5457926): +As noted in the [Snyk report](https://web.archive.org/web/20230920204324/https://security.snyk.io/vuln/SNYK-JS-XLSX-5457926): > The issue is resolved in version 0.19.3 diff --git a/docz/docs/02-getting-started/01-installation/07-bun.md b/docz/docs/02-getting-started/01-installation/07-bun.md index a9ced6d..b5dbb87 100644 --- a/docz/docs/02-getting-started/01-installation/07-bun.md +++ b/docz/docs/02-getting-started/01-installation/07-bun.md @@ -50,7 +50,7 @@ Bun supports both "CommonJS" and "ESM" modules. :::info pass -It is strongly recommended to use CommonJS in Bun. +**It is strongly recommended to use CommonJS in Bun.** ::: diff --git a/docz/docs/02-getting-started/02-examples/02-export.mdx b/docz/docs/02-getting-started/02-examples/02-export.mdx index dd801d2..8657155 100644 --- a/docz/docs/02-getting-started/02-examples/02-export.mdx +++ b/docz/docs/02-getting-started/02-examples/02-export.mdx @@ -246,7 +246,7 @@ The first presidential term can be found with the following function: const first_prez_term = prez => prez.terms.find(term => term.type === "prez"); ``` -:::note +:::note pass If no element in the array matches the criterion, `Array#find` does not return a value. In this case, since `prez` was created by filtering for people that @@ -647,10 +647,23 @@ The server process will display a URL (typically `http://127.0.0.1:8080`). Open Install the dependencies: + + + {`\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`} + + + +{`\ +bun install https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`} + + + + + Save the following script to `SheetJSNodeJS.js`: ```js title="SheetJSNodeJS.js" @@ -693,10 +706,23 @@ const XLSX = require("xlsx"); After saving the script, run the script: + + + ```bash node SheetJSNodeJS.js ``` + + + +```bash +bun run SheetJSNodeJS.js +``` + + + + This script will write a new file `Presidents.xlsx` in the same folder. :::caution pass @@ -818,63 +844,6 @@ After saving the script, run the script: deno run -A SheetJSDeno.ts ``` -This script will write a new file `Presidents.xlsx` in the same folder. - - - - -

Download https://cdn.sheetjs.com/xlsx-{current}/package/xlsx.mjs to xlsx.mjs:

- -{`\ -curl -LO https://cdn.sheetjs.com/xlsx-${current}/package/xlsx.mjs`} - - -Save the following script to `SheetJSBun.js`: - -```js title="SheetJSBun.js" -import * as XLSX from './xlsx.mjs'; -import * as fs from 'fs'; -XLSX.set_fs(fs); - -/* fetch JSON data and parse */ -const url = "https://sheetjs.com/data/executive.json"; -const raw_data = await (await fetch(url)).json(); - -/* filter for the Presidents */ -const prez = raw_data.filter((row) => row.terms.some((term) => term.type === "prez")); - -/* sort by first presidential term */ -prez.forEach(row => row.start = row.terms.find(term => term.type === "prez").start); -prez.sort((l,r) => l.start.localeCompare(r.start)); - -/* flatten objects */ -const rows = prez.map((row) => ({ - name: row.name.first + " " + row.name.last, - birthday: row.bio.birthday -})); - -/* generate worksheet and workbook */ -const worksheet = XLSX.utils.json_to_sheet(rows); -const workbook = XLSX.utils.book_new(); -XLSX.utils.book_append_sheet(workbook, worksheet, "Dates"); - -/* fix headers */ -XLSX.utils.sheet_add_aoa(worksheet, [["Name", "Birthday"]], { origin: "A1" }); - -/* calculate column width */ -const max_width = rows.reduce((w, r) => Math.max(w, r.name.length), 10); -worksheet["!cols"] = [ { wch: max_width } ]; - -/* create an XLSX file and try to save to Presidents.xlsx */ -XLSX.writeFile(workbook, "Presidents.xlsx", { compression: true }); -``` - -After saving the script, run the script: - -```bash -bun SheetJSBun.js -``` - This script will write a new file `Presidents.xlsx` in the same folder. @@ -1054,7 +1023,7 @@ export default App; -:::note +:::note pass The Android demo has been tested in Windows 10 and in macOS. @@ -1169,5 +1138,5 @@ see a preview of the data. The Numbers app can open the file. [^4]: See [`book_new` in "Utilities"](/docs/api/utilities/wb) [^5]: See [`book_append_sheet` in "Utilities"](/docs/api/utilities/wb) [^6]: See [`sheet_add_aoa` in "Utilities"](/docs/api/utilities/array#array-of-arrays-input) -[^7]: See ["Row and Column Properties"](/docs/csf/features/#row-and-column-properties) +[^7]: See ["Column Properties"](/docs/csf/features/colprops) [^8]: See [`writeFile` in "Writing Files"](/docs/api/write-options) \ No newline at end of file diff --git a/docz/docs/02-getting-started/02-examples/04-import.md b/docz/docs/02-getting-started/02-examples/04-import.md index ff0f9cd..da6053f 100644 --- a/docz/docs/02-getting-started/02-examples/04-import.md +++ b/docz/docs/02-getting-started/02-examples/04-import.md @@ -775,10 +775,23 @@ The server process will display a URL (typically `http://127.0.0.1:8080`). Open Install the dependencies: + + + {`\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`} + + + +{`\ +bun install https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`} + + + + + Save the following script to `SheetJSNodeJS.js`: ```js title="SheetJSNodeJS.js" @@ -813,10 +826,23 @@ const XLSX = require("xlsx"); After saving the script, run the script: + + + ```bash node SheetJSNodeJS.js ``` + + + +```bash +bun run SheetJSNodeJS.js +``` + + + + This script will print the rows in tab-separated values (TSV) format: ``` @@ -985,7 +1011,7 @@ export default App; -:::note +:::note pass The Android demo has been tested in Windows 10 and in macOS. diff --git a/docz/docs/03-demos/02-grid/01-xs.md b/docz/docs/03-demos/02-grid/01-xs.md index f2198ef..16e14f5 100644 --- a/docz/docs/03-demos/02-grid/01-xs.md +++ b/docz/docs/03-demos/02-grid/01-xs.md @@ -22,7 +22,7 @@ This demo was last verified on 2023 September 03. ## Live Demo -:::note pass +:::caution pass Due to CSS conflicts between the data grid and the documentation generator, features like scrolling may not work as expected. diff --git a/docz/docs/03-demos/02-grid/02-cdg.md b/docz/docs/03-demos/02-grid/02-cdg.md index 5e72635..0a4fd4e 100644 --- a/docz/docs/03-demos/02-grid/02-cdg.md +++ b/docz/docs/03-demos/02-grid/02-cdg.md @@ -21,7 +21,7 @@ This demo was last verified on 2023 September 03. ## Live Demo -:::note +:::caution pass Due to CSS conflicts between the data grid and the documentation generator, features like scrolling may not work as expected. diff --git a/docz/docs/03-demos/02-grid/16-rdg.md b/docz/docs/03-demos/02-grid/16-rdg.md index e07d6fd..1c11149 100644 --- a/docz/docs/03-demos/02-grid/16-rdg.md +++ b/docz/docs/03-demos/02-grid/16-rdg.md @@ -75,7 +75,7 @@ function rdg_to_ws(rows: Row[]): WorkSheet { } ``` -:::caution +:::caution pass When the demo was last refreshed, row array objects were preserved. This was not the case in a later release. The row arrays must be re-created. diff --git a/docz/docs/03-demos/02-grid/18-mui.md b/docz/docs/03-demos/02-grid/18-mui.md index e7933d9..2c30bdf 100644 --- a/docz/docs/03-demos/02-grid/18-mui.md +++ b/docz/docs/03-demos/02-grid/18-mui.md @@ -131,7 +131,7 @@ function ws_to_muidg(ws: WorkSheet): RowCol { In the other direction, a worksheet can be generated with `aoa_to_sheet`: -:::caution +:::caution pass `x-data-grid` does not properly preserve row array objects, so the row arrays must be re-created. The snippet defines a `arrayify` function. diff --git a/docz/docs/03-demos/02-grid/index.md b/docz/docs/03-demos/02-grid/index.md index 48e093f..7e33d43 100644 --- a/docz/docs/03-demos/02-grid/index.md +++ b/docz/docs/03-demos/02-grid/index.md @@ -11,7 +11,7 @@ Various JavaScript UI components provide a more interactive editing experience. Most are able to interchange with arrays of arrays or arrays of data objects. This demo focuses on a few open source data grids. -:::note +:::tip pass [SheetJS Pro](https://sheetjs.com/pro) offers additional features like styling and images. The UI tools typically support many of these advanced features. diff --git a/docz/docs/03-demos/03-net/01-network.mdx b/docz/docs/03-demos/03-net/01-network.mdx index 1795806..b6f28c7 100644 --- a/docz/docs/03-demos/03-net/01-network.mdx +++ b/docz/docs/03-demos/03-net/01-network.mdx @@ -362,7 +362,7 @@ async function workbook_dl_axios(url) { This demo uses `axios` to download and show the data in an HTML table. -:::caution +:::caution pass If the live demo shows a message @@ -471,7 +471,7 @@ superagent This demo uses `superagent` to download and show the data in an HTML table. -:::caution +:::caution pass If the live demo shows a message diff --git a/docz/docs/03-demos/03-net/03-email.md b/docz/docs/03-demos/03-net/03-email.md index e64bee1..c0750ba 100644 --- a/docz/docs/03-demos/03-net/03-email.md +++ b/docz/docs/03-demos/03-net/03-email.md @@ -54,7 +54,7 @@ It is strongly recommended to first test with an independent service provider. This demo will start with a free 30-day trial of Fastmail. At the time the demo was last tested, no payment details were required. -:::caution +:::caution pass A valid phone number (for SMS verification) was required. @@ -105,7 +105,7 @@ const msg = { from: "*", to: "*", subject: "*", text: "*", } ``` -:::caution +:::caution pass The file name must have the expected extension for the `bookType`! @@ -195,7 +195,7 @@ including `accepted` and `response`. The recipient inbox should receive an email shortly. The email will include an attachment `SheetJSMailExport.xlsb` which can be opened in Excel. -:::caution +:::caution pass The app password must be entered in step 3. If the account password was used, the mailer will fail with a message that includes: diff --git a/docz/docs/03-demos/04-static/02-gatsbyjs.md b/docz/docs/03-demos/04-static/02-gatsbyjs.md index d9f030e..27dff66 100644 --- a/docz/docs/03-demos/04-static/02-gatsbyjs.md +++ b/docz/docs/03-demos/04-static/02-gatsbyjs.md @@ -17,7 +17,7 @@ is a transformer that generates GraphQL nodes for each row of each worksheet. The plugin is officially supported by the Gatsby team. The plugin documentation includes examples and more detailed usage instructions. -:::note +:::note pass `gatsby-transformer-excel` is maintained by the Gatsby core team and all bugs should be directed to the main Gatsby project. If it is determined to be a bug @@ -25,7 +25,7 @@ in the parsing logic, issues should then be raised with the SheetJS project. ::: -:::caution +:::caution pass `gatsby-transformer-excel` uses an older version of the library. It can be overridden through a `package.json` override in the latest versions of NodeJS: diff --git a/docz/docs/03-demos/04-static/06-webpack.md b/docz/docs/03-demos/04-static/06-webpack.md index 03de037..2a63add 100644 --- a/docz/docs/03-demos/04-static/06-webpack.md +++ b/docz/docs/03-demos/04-static/06-webpack.md @@ -9,10 +9,11 @@ sidebar_custom_props: import current from '/version.js'; import CodeBlock from '@theme/CodeBlock'; -:::note +:::info pass This demo covers static asset imports. For processing files in the browser, the -["Bundlers" demo](/docs/demos/frontend/bundler#webpack) includes an example. +["Bundlers" demo](/docs/demos/frontend/bundler#webpack) includes an example of +importing the SheetJS library in a browser script. ::: diff --git a/docz/docs/03-demos/05-mobile/03-quasar.md b/docz/docs/03-demos/05-mobile/03-quasar.md index 58d426f..fe5c98a 100644 --- a/docz/docs/03-demos/05-mobile/03-quasar.md +++ b/docz/docs/03-demos/05-mobile/03-quasar.md @@ -314,7 +314,7 @@ The app should now show two buttons at the bottom: ![Quasar Step 6](pathname:///mobile/quasar6.png) -:::caution +:::caution pass If the app is blank or not refreshing, delete the app and close the simulator, then restart the development process. diff --git a/docz/docs/03-demos/06-desktop/01-electron.md b/docz/docs/03-demos/06-desktop/01-electron.md index 709c0df..e2e0b18 100644 --- a/docz/docs/03-demos/06-desktop/01-electron.md +++ b/docz/docs/03-demos/06-desktop/01-electron.md @@ -121,7 +121,7 @@ async function importFile() { } ``` -:::note +:::note pass In older versions of Electron, `showOpenDialog` returned the path directly: @@ -161,7 +161,7 @@ async function exportFile(workbook) { } ``` -:::note +:::note pass In older versions of Electron, `showSaveDialog` returned the path directly: @@ -208,7 +208,7 @@ The demo project is wired for `electron-forge` to build the standalone binary. - [`index.html`](pathname:///electron/index.html) : window page - [`index.js`](pathname:///electron/index.js) : script loaded in render context -:::caution +:::caution pass Right-click each link and select "Save Link As...". Left-clicking a link will try to load the page in your browser. The goal is to save the file contents. @@ -302,7 +302,7 @@ the required changes for Electron 23.0.0. There are no Electron-specific workarounds in the library, but Electron broke backwards compatibility multiple times. A summary of changes is noted below. -:::caution +:::caution pass Electron 6.x changed the `dialog` API. Methods like `showSaveDialog` originally returned an array of strings, but now returns a `Promise`. This change was not diff --git a/docz/docs/03-demos/06-desktop/02-nwjs.md b/docz/docs/03-demos/06-desktop/02-nwjs.md index 2bdfa8f..5145200 100644 --- a/docz/docs/03-demos/06-desktop/02-nwjs.md +++ b/docz/docs/03-demos/06-desktop/02-nwjs.md @@ -130,7 +130,7 @@ This demo was tested against NW.js 0.78.0 on 2023 July 27. 2) Download [`index.html`](pathname:///nwjs/index.html) into the same folder. -:::caution +:::caution pass Right-click the link and select "Save Link As...". Left-clicking the link will try to load the page in your browser. The goal is to save the file contents. diff --git a/docz/docs/03-demos/06-desktop/06-reactnative.md b/docz/docs/03-demos/06-desktop/06-reactnative.md index c447da2..c4ab3e9 100644 --- a/docz/docs/03-demos/06-desktop/06-reactnative.md +++ b/docz/docs/03-demos/06-desktop/06-reactnative.md @@ -228,7 +228,7 @@ export default App; ## Native Modules -:::caution +:::caution pass As with the mobile versions of React Native, file operations are not provided by the base SDK. The examples include native code for both Windows and macOS. @@ -387,7 +387,7 @@ There is no simple standalone executable file at the end of the process. ::: -:::note +:::note pass React Native Windows supports writing native code in C++ or C#. This demo has been tested against both application types. @@ -396,7 +396,7 @@ been tested against both application types. 0) Follow the ["Getting Started" guide](https://microsoft.github.io/react-native-windows/docs/getting-started) -:::caution +:::caution pass At the time of testing, NodeJS `v16` was required. A tool like [`nvm-windows`](https://github.com/coreybutler/nvm-windows/releases) should be @@ -447,7 +447,7 @@ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz`} npx react-native run-windows --no-telemetry ``` -:::caution +:::caution pass When the demo was tested in Windows 11, the run step failed with the message: diff --git a/docz/docs/03-demos/06-desktop/09-cli.md b/docz/docs/03-demos/06-desktop/09-cli.md index 61da845..29cc9e8 100644 --- a/docz/docs/03-demos/06-desktop/09-cli.md +++ b/docz/docs/03-demos/06-desktop/09-cli.md @@ -112,7 +112,7 @@ npx nexe -t 14.15.3 xlsx-cli.js This generates `xlsx-cli` or `xlsx-cli.exe` depending on platform. -:::caution +:::caution pass When the demo was tested on `darwin-arm`, the `mac-arm64` pre-built package was missing. The package must be built from source: diff --git a/docz/docs/03-demos/07-data/01-websql.md b/docz/docs/03-demos/07-data/01-websql.md index 1150807..ace4501 100644 --- a/docz/docs/03-demos/07-data/01-websql.md +++ b/docz/docs/03-demos/07-data/01-websql.md @@ -16,7 +16,7 @@ work as-is in WebSQL. The public demo generates a database from workbook. -:::caution +:::caution pass WebSQL is only supported in Chromium-based browsers including Chrome. diff --git a/docz/docs/03-demos/07-data/14-knex.md b/docz/docs/03-demos/07-data/14-knex.md index 9e5af6d..4731e5a 100644 --- a/docz/docs/03-demos/07-data/14-knex.md +++ b/docz/docs/03-demos/07-data/14-knex.md @@ -9,33 +9,169 @@ sidebar_custom_props: import current from '/version.js'; import CodeBlock from '@theme/CodeBlock'; +[KnexJS](https://knexjs.org/) is a SQL query builder with support for a number +of SQL dialects. + +[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing +data from spreadsheets. + +This demo uses KnexJS and SheetJS to exchange data between spreadsheets and SQL +servers. We'll explore how to use save tables from a database to spreadsheets +and how to add data from spreadsheets into a database. + :::note -This demo was last tested on 2023 April 19 with Knex 2.4.2 and `better-sqlite`. +This demo was last tested on 2023 September 23 with Knex 2.5.1. The demo uses +the SQLite backend with the `better-sqlite3` connector. ::: ## Integration Details -#### Importing Data +The [SheetJS NodeJS module](/docs/getting-started/installation/nodejs) can be +loaded in NodeJS scripts that use KnexJS. -`sheet_to_json` generates an array of objects. An `INSERT` statement can be -generated from each row object using `knex.insert`: +### Exporting Data + +The KnexJS `select` method[^1] creates a `SELECT` query. The return value is a +Promise that resolves to an array of objects. + +The SheetJS `json_to_sheet` method[^2] can generate a worksheet object[^3] from +the array of objects: ```js -const aoo = XLSX.utils.sheet_to_json(ws); -for(let i = 0; i < aoo.length; ++i) await knex.insert(aoo[i]).into(table_name); -``` +const table_name = "Tabeller1"; // name of table -#### Exporting Data - -The result of a `SELECT` statement is an array of objects: - -```js +/* fetch all data from specified table */ const aoo = await knex.select("*").from(table_name); + +/* generate a SheetJS worksheet object from the data */ const worksheet = XLSX.utils.json_to_sheet(aoo); ``` +A workbook object can be built from the worksheet using utility functions[^4]. +The workbook can be exported using the SheetJS `writeFile` method[^5]: + +```js +/* create a new workbook and add the worksheet */ +const wb = XLSX.utils.book_new(); +XLSX.utils.book_append_sheet(wb, worksheet, "Sheet1"); + +/* export workbook to XLSX */ +XLSX.writeFile(wb, "SheetJSKnexJSExport.xlsx"); +``` + +### Importing Data + +The SheetJS `sheet_to_json` function[^6] takes a worksheet object and generates +an array of objects. + +The KnexJS `insert` method[^7] creates `INSERT` queries. The return value is a +Promise that resolves when the query is executed: + +```js +const table_name = "Blatte1"; // name of table + +/* generate an array of arrays from the worksheet */ +const aoo = XLSX.utils.sheet_to_json(ws); + +/* insert every row into the specified database table */ +await knex.insert(aoo).into(table_name); +``` + +### Creating a Table + +The KnexJS Schema Builder supports creating tables with `createTable`[^8] and +dropping tables with `dropTableIfExists`[^9]. + +The array of objects can be scanned to determine column names and types. + +
Implementation Details (click to show) + +The `aoo_to_knex_table` function: + +- scans each row object to determine column names and types +- drops and creates a new table with the determined column names and types +- loads the entire dataset into the new table + +```js +/* create table and load data given an array of objects and a Knex connection */ +async function aoo_to_knex_table(knex, aoo, table_name) { + /* define types that can be converted (e.g. boolean can be stored in float) */ + const T_FLOAT = ["float", "boolean"]; + const T_BOOL = ["boolean"]; + + /* types is a map from column headers to Knex schema column type */ + const types = {}; + + /* names is an ordered list of the column header names */ + const names = []; + + /* loop across each row object */ + aoo.forEach(row => + /* Object.entries returns a row of [key, value] pairs */ + Object.entries(row).forEach(([k,v]) => { + + /* If this is first occurrence, mark unknown and append header to names */ + if(!types[k]) { types[k] = ""; names.push(k); } + + /* skip null and undefined values */ + if(v == null) return; + + /* check and resolve type */ + switch(typeof v) { + /* change type if it is empty or can be stored in a float */ + case "number": if(!types[k] || T_FLOAT.includes(types[k])) types[k] = "float"; break; + /* change type if it is empty or can be stored in a boolean */ + case "boolean": if(!types[k] || T_BOOL.includes(types[k])) types[k] = "boolean"; break; + /* no other type can hold strings */ + case "string": types[k] = "text"; break; + default: types[k] = "text"; break; + } + }) + ); + + /* Delete table if it exists in the DB */ + await knex.schema.dropTableIfExists(table_name); + + /* use column type info to create table */ + await knex.schema.createTable(table_name, (table) => { + names.forEach(h => { + /* call schema function e.g. table.text("Name"); table.float("Index"); */ + table[types[h] || "text"](h); + }); + }); + + /* insert each row */ + await knex.insert(aoo).into(table_name); + return knex; +} +``` + +
+ +:::note pass + +The `Knex` constructor may display a warning when connecting to SQLite: + +``` +sqlite does not support inserting default values. Set the `useNullAsDefault` flag to hide this warning. (see docs https://knexjs.org/guide/query-builder.html#insert). +``` + +That flag should be added to the options argument: + +```js +const Knex = require('knex'); +let knex = Knex({ + client: 'better-sqlite3', + connection: { filename: "SheetJSKnex.db" }, + // highlight-next-line + useNullAsDefault: true +}); +``` + +::: + ## Complete Example 1) Install dependencies: @@ -50,109 +186,22 @@ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz knex be curl -LO https://sheetjs.com/pres.numbers ``` -3) Save the following utility script to `sheetjsknex.js`: +3) Download [`SheetJSKnexTest.js`](pathname:///knex/SheetJSKnexTest.js): -```js title="sheetjsknex.js" -const XLSX = require("xlsx"); - -// define mapping between determined types and Knex types -const PG = { "n": "float", "s": "text", "b": "boolean" }; - -async function ws_to_knex(knex, ws, table_name) { - - // generate an array of objects from the data - const aoo = XLSX.utils.sheet_to_json(ws); - - // types will map column headers to types, while hdr holds headers in order - const types = {}, hdr = []; - - // loop across each row object - aoo.forEach(row => - // Object.entries returns a row of [key, value] pairs. Loop across those - Object.entries(row).forEach(([k,v]) => { - - // If this is first time seeing key, mark unknown and append header array - if(!types[k]) { types[k] = "?"; hdr.push(k); } - - // skip null and undefined - if(v == null) return; - - // check and resolve type - switch(typeof v) { - case "string": // strings are the broadest type - types[k] = "s"; break; - case "number": // if column is not string, number is the broadest type - if(types[k] != "s") types[k] = "n"; break; - case "boolean": // only mark boolean if column is unknown or boolean - if("?b".includes(types[k])) types[k] = "b"; break; - default: types[k] = "s"; break; // default to string type - } - }) - ); - - await knex.schema.dropTableIfExists(table_name); - - // use column type info to create table - await knex.schema.createTable(table_name, (table) => { hdr.forEach(h => { table[PG[types[h]] || "text"](h); }); }); - - // insert each non-empty row object - for(let i = 0; i < aoo.length; ++i) { - if(!aoo[i] || !Object.keys(aoo[i]).length) continue; - try { await knex.insert(aoo[i]).into(table_name); } catch(e) {} - } - return knex; -} - -async function knex_to_ws(knex, table_name) { - const aoo = await knex.select("*").from(table_name); - return XLSX.utils.json_to_sheet(aoo); -} - -module.exports = { ws_to_knex, knex_to_ws }; +```bash +curl -LO https://docs.sheetjs.com/knex/SheetJSKnexTest.js ``` -4) Save the following to `SheetJSKnexTest.js`: -```js title="SheetJSKnexTest.js" -const { ws_to_knex, knex_to_ws } = require("./sheetjsknex"); -const Knex = require('knex'); +This script will: +- read and parse the test file `pres.numbers` +- create a connection to a SQLite database stored at `SheetJSKnex.db` +- load data from the first worksheet into a table with name `Test_Table` +- disconnect and reconnect to the database +- dump data from the table `Test_Table` +- export the dataset to `SheetJSKnex.xlsx` -/* read file and get first worksheet */ -const XLSX = require("xlsx"); -const oldwb = XLSX.readFile("pres.numbers"); -const wsname = oldwb.SheetNames[0]; -const oldws = oldwb.Sheets[wsname]; - -(async() => { - /* open connection to SheetJSKnex.db */ - let knex = Knex({ client: 'better-sqlite3', connection: { filename: "SheetJSKnex.db" } }); - try { - /* load data into database and close connection */ - await ws_to_knex(knex, oldws, "Test_Table"); - } finally { knex.destroy(); } - - /* reconnect to SheetJSKnex.db */ - knex = Knex({ client: 'better-sqlite3', connection: { filename: "SheetJSKnex.db" } }); - try { - /* get data from db */ - const aoo = await knex.select("*").from("Test_Table"); - - /* export to file */ - const newws = await knex_to_ws(knex, "Test_Table"); - const newwb = XLSX.utils.book_new(); - XLSX.utils.book_append_sheet(newwb, newws, "Export"); - XLSX.writeFile(newwb, "SheetJSKnex.xlsx"); - } finally { knex.destroy(); } -})(); -``` - -This script will read `pres.numbers` and load data into a table `Test_Table`. -The SQLite database will be saved to `SheetJSKnex.db`. - -After closing the connection, the script will make a new connection and export -data to `SheetJSKnex.xlsx`. - -5) Run the script: +4) Run the script: ```bash node SheetJSKnexTest.js @@ -171,3 +220,13 @@ npx xlsx-cli SheetJSKnex.xlsx ```bash sqlite3 SheetJSKnex.db 'select * from Test_Table' ``` + +[^1]: See [`select`](https://knexjs.org/guide/query-builder.html#select) in the KnexJS query builder documentation. +[^2]: See [`json_to_sheet` in "Utilities"](/docs/api/utilities/array#array-of-objects-input) +[^3]: See ["Sheet Objects"](/docs/csf/sheet) in "SheetJS Data Model" for more details. +[^4]: See ["Workbook Helpers" in "Utilities"](/docs/api/utilities/wb) for details on `book_new` and `book_append_sheet`. +[^5]: See [`writeFile` in "Writing Files"](/docs/api/write-options) +[^6]: See [`sheet_to_json` in "Utilities"](/docs/api/utilities/array#array-output) +[^7]: See [`insert`](https://knexjs.org/guide/query-builder.html#insert) in the KnexJS query builder documentation. +[^8]: See [`createTable`](https://knexjs.org/guide/schema-builder.html#createtable) in the KnexJS Schema Builder documentation. +[^9]: See [`dropTableIfExists`](https://knexjs.org/guide/schema-builder.html#droptableifexists) in the KnexJS Schema Builder documentation. diff --git a/docz/docs/03-demos/07-data/19-alasql.md b/docz/docs/03-demos/07-data/19-alasql.md index 0bd8f6b..046185e 100644 --- a/docz/docs/03-demos/07-data/19-alasql.md +++ b/docz/docs/03-demos/07-data/19-alasql.md @@ -59,7 +59,7 @@ INTO XLSX( -- export data to file -:::caution +:::caution pass If the live demo shows a message @@ -196,7 +196,7 @@ The `XLSX` "into" target calls `XLSX.writeFile` under the hood: ## NodeJS -:::caution +:::caution pass `alasql` uses an older version of the library. It can be overridden through a `package.json` override in the latest versions of NodeJS: diff --git a/docz/docs/03-demos/07-data/index.md b/docz/docs/03-demos/07-data/index.md index 8deacaa..68bbea1 100644 --- a/docz/docs/03-demos/07-data/index.md +++ b/docz/docs/03-demos/07-data/index.md @@ -116,7 +116,7 @@ const ws = XLSX.utils.json_to_sheet(aoo); When a strict schema is needed, the `sheet_to_json` helper function generates arrays of JS objects that can be scanned to determine the column "types". -:::note +:::note pass Document databases like MongoDB tend not to require schemas. Arrays of objects can be used directly without setting up a schema: diff --git a/docz/docs/03-demos/08-local/01-file.md b/docz/docs/03-demos/08-local/01-file.md index b2e4250..2011e37 100644 --- a/docz/docs/03-demos/08-local/01-file.md +++ b/docz/docs/03-demos/08-local/01-file.md @@ -390,7 +390,7 @@ the feature in version 86. Safari did not support File System Access API. ::: -:::caution +:::caution pass When this demo was last tested, Google Chrome did not add an entry to the "Downloads" list. Nevertheless the actual file was written correctly. diff --git a/docz/docs/03-demos/08-local/09-indexeddb.md b/docz/docs/03-demos/08-local/09-indexeddb.md index 6920989..2bc0b4b 100644 --- a/docz/docs/03-demos/08-local/09-indexeddb.md +++ b/docz/docs/03-demos/08-local/09-indexeddb.md @@ -28,7 +28,7 @@ a popular choice for offline storage. A number of popular wrapper libraries seek to simplify IndexedDB operations. -:::note +:::note pass The wrapper libraries in this section have been used by SheetJS users in production sites. diff --git a/docz/docs/03-demos/09-cloud/01-salesforce.md b/docz/docs/03-demos/09-cloud/01-salesforce.md index 3577e39..43d34fe 100644 --- a/docz/docs/03-demos/09-cloud/01-salesforce.md +++ b/docz/docs/03-demos/09-cloud/01-salesforce.md @@ -12,7 +12,7 @@ Salesforce apps can use third-party libraries in "Lightning Web Components". This demo assumes familiarity with Lightning Web Components. Salesforce has a [detailed introduction.](https://developer.salesforce.com/docs/component-library/documentation/en/lwc/lwc.get_started_introduction) -:::caution +:::caution pass Salesforce may change the platform in backwards-incompatible ways, so the demo may require some adjustments. The official documentation should be consulted. @@ -152,7 +152,7 @@ The easiest approach is to right-click the link and select "Save Link As..." sfdx force:source:deploy -p force-app -u SALESFORCE@USER.NAME # replace with actual username ``` -:::note +:::note pass The official documentation recommends adding a static resource with a ZIP file. That approach is not explored in this demo. @@ -207,7 +207,7 @@ export default class SheetComponent extends LightningElement { ## Exporting Data from SF Lists -:::note +:::note pass There are many different data types and APIs. This demo uses the deprecated `getListUi` function to pull account data. @@ -381,7 +381,7 @@ The simple export has all of the data: ![Excel Export](pathname:///files/sfxlexport.png) -:::note +:::tip pass [SheetJS Pro](https://sheetjs.com/pro) offers additional styling options like cell styling, automatic column width calculations, and frozen rows. diff --git a/docz/docs/03-demos/09-cloud/11-aws.md b/docz/docs/03-demos/09-cloud/11-aws.md index faecfb9..c100c10 100644 --- a/docz/docs/03-demos/09-cloud/11-aws.md +++ b/docz/docs/03-demos/09-cloud/11-aws.md @@ -10,7 +10,7 @@ import CodeBlock from '@theme/CodeBlock'; AWS is a Cloud Services platform which includes traditional virtual machine support, "Serverless Functions", cloud storage and much more. -:::caution +:::caution pass AWS iterates quickly and there is no guarantee that the referenced services will be available in the future. diff --git a/docz/docs/03-demos/09-cloud/12-azure.md b/docz/docs/03-demos/09-cloud/12-azure.md index 609fe8f..2ac2a24 100644 --- a/docz/docs/03-demos/09-cloud/12-azure.md +++ b/docz/docs/03-demos/09-cloud/12-azure.md @@ -10,7 +10,7 @@ import CodeBlock from '@theme/CodeBlock'; Azure is a Cloud Services platform which includes traditional virtual machine support, "Serverless Functions", cloud storage and much more. -:::caution +:::caution pass Azure iterates quickly and there is no guarantee that the referenced services will be available in the future. diff --git a/docz/docs/03-demos/09-cloud/18-github.md b/docz/docs/03-demos/09-cloud/18-github.md index 2e06955..d77a6b0 100644 --- a/docz/docs/03-demos/09-cloud/18-github.md +++ b/docz/docs/03-demos/09-cloud/18-github.md @@ -48,7 +48,7 @@ As a project from the company, the entire lifecycle uses GitHub offerings: - `flat-postprocessing` Post-processing helper functions and examples - "Flat Viewer": Web viewer for structured CSV and JSON data on GitHub -:::caution +:::caution pass A GitHub account is required. When the demo was tested, free GitHub accounts had no Actions usage limits for public repositories. diff --git a/docz/docs/03-demos/09-cloud/19-deno.md b/docz/docs/03-demos/09-cloud/19-deno.md index e17f280..5fa24f4 100644 --- a/docz/docs/03-demos/09-cloud/19-deno.md +++ b/docz/docs/03-demos/09-cloud/19-deno.md @@ -20,7 +20,7 @@ This breaks web frameworks that use the filesystem in body parsing. ::: -:::caution +:::caution pass When the demo was last tested, Deno Deploy required a GitHub account. diff --git a/docz/docs/03-demos/32-extensions/02-chromium.md b/docz/docs/03-demos/32-extensions/02-chromium.md index 793c2c4..6487a38 100644 --- a/docz/docs/03-demos/32-extensions/02-chromium.md +++ b/docz/docs/03-demos/32-extensions/02-chromium.md @@ -4,7 +4,7 @@ pagination_prev: demos/cloud/index pagination_next: demos/bigdata/index --- -:::note +:::note pass This demo showcases Manifest V2 and Manifest V3 extensions. Chrome Web Store will not accept new V2 extensions, but these can be sideloaded using the diff --git a/docz/docs/03-demos/42-engines/01_duktape.md b/docz/docs/03-demos/42-engines/01_duktape.md index 0daeaa0..112756e 100644 --- a/docz/docs/03-demos/42-engines/01_duktape.md +++ b/docz/docs/03-demos/42-engines/01_duktape.md @@ -119,7 +119,7 @@ This demo was tested in the following deployments: |:-------------|:--------|:-----------| | `darwin-x64` | `2.7.0` | 2023-07-24 | | `darwin-arm` | `2.7.0` | 2023-06-05 | -| `linux-x64` | `2.7.0` | 2023-06-02 | +| `linux-x64` | `2.7.0` | 2023-09-22 | | `linux-arm` | `2.7.0` | 2023-08-30 | | `win10-x64` | `2.7.0` | 2023-07-24 | diff --git a/docz/docs/03-demos/42-engines/03_rhino.md b/docz/docs/03-demos/42-engines/03_rhino.md index 3c4130a..649a7f4 100644 --- a/docz/docs/03-demos/42-engines/03_rhino.md +++ b/docz/docs/03-demos/42-engines/03_rhino.md @@ -15,7 +15,7 @@ can be parsed and evaluated in a Rhino context. This demo wraps workbooks and sheets into separate Java classes. The final result is a JAR. -:::caution +:::caution pass Rhino does not support Uint8Array, so certain formats like NUMBERS cannot be parsed or written from Rhino JS code! @@ -24,7 +24,7 @@ parsed or written from Rhino JS code! ## Integration Details -:::note +:::caution pass Due to code generation errors, optimization must be turned off: diff --git a/docz/docs/03-demos/42-engines/05-jint.md b/docz/docs/03-demos/42-engines/05-jint.md index 994fc15..cfffdaa 100644 --- a/docz/docs/03-demos/42-engines/05-jint.md +++ b/docz/docs/03-demos/42-engines/05-jint.md @@ -165,6 +165,7 @@ This demo was tested in the following deployments: |:-------------|:------------------|:-----------| | `darwin-x64` | `3.0.0-beta-2051` | 2023-09-16 | | `win10-x64` | `3.0.0-beta-2051` | 2023-09-16 | +| `linux-x64` | `3.0.0-beta-2051` | 2023-09-22 | ::: @@ -337,6 +338,7 @@ tested platforms are listed below: |:-----------------|:------------| | Intel Mac | `osx-x64` | | Windows 10 (x64) | `win10-x64` | +| Linux (x64) | `linux-x64` | 9) Build the standalone application. Replace `$RID` with the real value in: @@ -353,6 +355,15 @@ For Intel Mac, the RID is `osx-x64` and the command is ```bash dotnet publish -c Release -r osx-x64 --self-contained true -p:PublishSingleFile=true -p:PublishTrimmed=true +``` + + + + +For x64 Linux, the RID is `linux-x64` and the command is + +```bash +dotnet publish -c Release -r linux-x64 --self-contained true -p:PublishSingleFile=true -p:PublishTrimmed=true ``` @@ -383,6 +394,15 @@ For Intel Mac, the RID is `osx-x64` and the command is: ```bash cp bin/Release/net6.0/osx-x64/publish/SheetJSJint . +``` + + + + +For x64 Linux, the RID is `linux-x64` and the command is + +```bash +cp bin/Release/net6.0/linux-x64/publish/SheetJSJint . ``` @@ -422,5 +442,5 @@ copy .\bin\Release\net6.0\win10-x64\publish\SheetJSJint.exe . [^2]: See [`read` in "Reading Files"](/docs/api/parse-options) [^3]: See [`write` in "Writing Files"](/docs/api/write-options) [^4]: See ["Supported Output Formats" in "Writing Files"](/docs/api/write-options#supported-output-formats) for details on `bookType` -[^5]: At the time of writing, is the official endpoint. +[^5]: At the time of writing, is the official endpoint. For Steam Deck Holo and other Arch Linux distributions, the `dotnet-sdk` and `dotnet-runtime` packages should be installed using `pacman`. [^6]: See [".NET RID Catalog"](https://learn.microsoft.com/en-us/dotnet/core/rid-catalog) in the .NET documentation \ No newline at end of file diff --git a/docz/docs/03-demos/42-engines/09_hermes.md b/docz/docs/03-demos/42-engines/09_hermes.md index d6ef595..884fbd5 100644 --- a/docz/docs/03-demos/42-engines/09_hermes.md +++ b/docz/docs/03-demos/42-engines/09_hermes.md @@ -26,7 +26,7 @@ command-line tool for reading data from files. :::info pass Many Hermes functions are not documented. The explanation was verified against -commit `70af78b`. +commit `49e1930`. ::: @@ -207,7 +207,7 @@ Standard SheetJS operations can pick the first worksheet and generate CSV string data from the worksheet. Hermes provides methods to convert the JS strings back to `std::string` objects for further processing in C++. -:::note +:::note pass It is strongly recommended to create a stub function to perform the entire workflow in JS code and pass the final result back to C++. @@ -340,7 +340,7 @@ This demo was tested in the following deployments: |:-------------|:-----------|:-----------| | `darwin-x64` | `70af78b` | 2023-08-27 | | `darwin-arm` | `869312f` | 2023-06-05 | -| `linux-x64` | `70af78b` | 2023-08-27 | +| `linux-x64` | `49e1930` | 2023-09-22 | | `linux-arm` | `70af78b` | 2023-08-27 | ::: diff --git a/docz/docs/03-demos/42-engines/15_rb.md b/docz/docs/03-demos/42-engines/15_rb.md index 665f549..ef89a28 100644 --- a/docz/docs/03-demos/42-engines/15_rb.md +++ b/docz/docs/03-demos/42-engines/15_rb.md @@ -81,7 +81,7 @@ This demo was tested in the following deployments: gem install execjs ``` -:::note +:::note pass The command may need to be run as an administrator or root user: diff --git a/docz/docs/03-demos/42-engines/20_chakra.md b/docz/docs/03-demos/42-engines/20_chakra.md index d71fbba..7ed7750 100644 --- a/docz/docs/03-demos/42-engines/20_chakra.md +++ b/docz/docs/03-demos/42-engines/20_chakra.md @@ -40,7 +40,7 @@ JsSetCurrentContext(JS_INVALID_REFERENCE); JsDisposeRuntime(runtime); ``` -:::note +:::note pass Cleanup and validation code is omitted from the discussion. The integration example shows structured validation and controlled memory usage. @@ -181,7 +181,7 @@ cd ChakraCore cd .. ``` -:::caution +:::caution pass When this demo was last tested, the build failed with the message: @@ -216,7 +216,7 @@ cd .. ``` -:::caution +:::caution pass When this demo was last tested, the build failed with the message: @@ -238,7 +238,7 @@ cd .. -:::caution +:::caution pass When the demo was last tested, ChakraCore JIT was not supported. @@ -297,7 +297,7 @@ curl -L -O https://docs.sheetjs.com/chakra/Makefile make ``` -:::caution +:::caution pass When this demo was last tested on macOS, the build failed with the message: diff --git a/docz/docs/03-demos/42-engines/index.md b/docz/docs/03-demos/42-engines/index.md index f48032b..00148aa 100644 --- a/docz/docs/03-demos/42-engines/index.md +++ b/docz/docs/03-demos/42-engines/index.md @@ -125,14 +125,23 @@ generates a C library and a standalone CLI tool. The simplest way to interact with the engine is to pass Base64 strings. -:::note +:::note pass + +This demo was tested in the following deployments: + +| Architecture | Commit | Date | +|:-------------|:----------|:-----------| +| `darwin-x64` | `a588e49` | 2023-09-22 | +| `linux-x64` | `a588e49` | 2023-09-22 | + +:::note pass While applications should link against the official libraries, the standalone tool is useful for verifying functionality. ::: -:::caution +:::caution pass This demo requires a much larger heap size than is normally used in JerryScript deployments! In local testing, the following sizes were needed: diff --git a/docz/docs/06-solutions/01-input.md b/docz/docs/06-solutions/01-input.md index 8f13f1f..e643db2 100644 --- a/docz/docs/06-solutions/01-input.md +++ b/docz/docs/06-solutions/01-input.md @@ -143,7 +143,7 @@ import * as XLSX from 'https://cdn.sheetjs.com/xlsx-${current}/package/xlsx.mjs' \n\ const workbook = XLSX.readFile("test.xlsx");`} -:::note +:::note pass Applications reading files must be invoked with the `--allow-read` flag. @@ -369,7 +369,7 @@ const server = new Drash.Server({ hostname: "", port: 7262, protocol: "http", \n\ server.run();`} -:::note +:::note pass Deno must be run with the `--allow-net` flag to enable network requests: @@ -518,7 +518,7 @@ const data = await (await fetch(url)).arrayBuffer(); /* data is an ArrayBuffer */ const workbook = XLSX.read(data);`} -:::note +:::note pass Deno must be run with the `--allow-net` flag to enable network requests: @@ -557,7 +557,7 @@ req.end(); ### Example: Readable Streams -:::caution +:::caution pass The recommended approach is to buffer streams in memory and process once all of the data has been collected. A proper streaming parse is technically impossible. diff --git a/docz/docs/06-solutions/03-processing.md b/docz/docs/06-solutions/03-processing.md index b74b886..02cffcd 100644 --- a/docz/docs/06-solutions/03-processing.md +++ b/docz/docs/06-solutions/03-processing.md @@ -15,7 +15,7 @@ read from data sources and write to data sources. ### Worksheets -:::note +:::note pass Worksheet names are case-sensitive. @@ -191,7 +191,7 @@ worksheet["!merges"].push({ }); ``` -:::caution +:::caution pass This approach does not verify if two merged ranges intersect. diff --git a/docz/docs/06-solutions/05-output.md b/docz/docs/06-solutions/05-output.md index 8b93ead..10ba726 100644 --- a/docz/docs/06-solutions/05-output.md +++ b/docz/docs/06-solutions/05-output.md @@ -54,7 +54,7 @@ referencing the other export functions. The second `opts` argument is optional. ["Writing Options"](/docs/api/write-options) covers the supported properties and behaviors. -:::note +:::note pass The `writeFile` and `writeFileXLSX` methods uses platform-specific APIs to save files. The APIs do not generally provide feedback on whether files were created. @@ -218,7 +218,7 @@ import * as XLSX from 'https://cdn.sheetjs.com/xlsx-${current}/package/xlsx.mjs' \n\ XLSX.writeFile(workbook, "test.xlsx");`} -:::note +:::note pass Applications writing files must be invoked with the `--allow-write` flag. @@ -303,7 +303,7 @@ This example focuses on responses to network requests in a server-side platform like NodeJS. While files can be generated in the web browser, server-side file generation allows for exact audit trails and has better mobile user support. -:::caution +:::caution pass Production deployments should use a server framework like ExpressJS. These snippets use low-level APIs for illustration purposes. @@ -489,7 +489,7 @@ like `XMLHttpRequest` and `fetch` as well as third-party libraries. -:::caution +:::caution pass Some platforms like Azure and AWS will attempt to parse POST request bodies as UTF-8 strings before user code can see the data. This will result in corrupt diff --git a/docz/docs/07-csf/07-features/01-dates.md b/docz/docs/07-csf/07-features/01-dates.md index be03847..72bc2eb 100644 --- a/docz/docs/07-csf/07-features/01-dates.md +++ b/docz/docs/07-csf/07-features/01-dates.md @@ -151,7 +151,7 @@ The following table covers some common formats: | `A/P` | Meridiem ("A" or "P") | | `AM/PM` | Meridiem ("AM" or "PM") | -:::note +:::note pass `m` and `mm` are context-dependent. It is interpreted as "minutes" when the previous or next date token represents a time (hours or seconds): @@ -278,7 +278,7 @@ function LocalInfo() { )} ``` -:::caution +:::caution pass The timezone information is provided by the JavaScript engine and local settings. There are outstanding Google Chrome and V8 bugs related to rounded offsets for diff --git a/docz/docs/07-csf/07-features/04-comments.md b/docz/docs/07-csf/07-features/04-comments.md index 208b6de..2a8acda 100644 --- a/docz/docs/07-csf/07-features/04-comments.md +++ b/docz/docs/07-csf/07-features/04-comments.md @@ -34,7 +34,7 @@ support Excel styled comments or Excel legacy notes. The letter R (R) marks features parsed but not written in the format. -:::note +:::note pass [SheetJS Pro](https://sheetjs.com/pro) supports comment rich text and styling. @@ -78,6 +78,10 @@ characters may cause issues with other formats. ::: +## Demos + +#### Export +
Live Export Example (click to hide) This example creates a small worksheet with a comment in cell A1: @@ -98,7 +102,9 @@ function SheetJSComments1() {
-
Live Import Example (click to hide) +#### Import + +
Live Import Example (click to show) This example displays every comment in the workbook: @@ -149,7 +155,10 @@ cell.c.hidden = true; cell.c.push({a:"SheetJS", t:"This comment will be hidden"}); ``` -
Live Example (click to hide) +
Live Example (click to show) + +The following demo creates a worksheet with two comments. The comment in cell A1 +will be visibile and the comment in cell A2 will be hidden. ```jsx live function SheetJSComments2() { @@ -203,7 +212,7 @@ There is no Active Directory or Office 365 metadata associated with authors.
Live Example (click to hide) ```jsx live -function SheetJSComments2() { +function SheetJSThreadedComments() { return (