LankyMoose/docs.sheetjs.com
1
Fork 0
forked from sheetjs/docs.sheetjs.com
Code Pull Requests Activity
92e3c5aa72
docs.sheetjs.com/docz/docs/03-demos/23-data/11-sqlite.md
SheetJS e3ca16d997 node-sea
2024-03-19 05:00:03 -04:00

9.6 KiB
Raw Blame History

title sidebar_label pagination_prev pagination_next sidebar_custom_props
Sheets with SQLite SQLite demos/cli/index demos/local/index
sql
true
<head> </head>

import current from '/version.js'; import CodeBlock from '@theme/CodeBlock';

SQLite is a lightweight embeddable SQL database engine. There are connector libraries for many popular JavaScript server-side platforms.

SheetJS is a JavaScript library for reading and writing data from spreadsheets.

This demo uses SQLite and SheetJS to exchange data between spreadsheets and SQL servers. We'll explore how to use save tables from a database to spreadsheets and how to add data from spreadsheets into a database.

:::info pass

This demo covers SQLite .db file processing.

The WebSQL demo covers the Web SQL Database API, a SQLite-compatible database built into Chromium and Google Chrome.

:::

:::note Tested Deployments

This demo was tested in the following environments:

Platform Connector Library Date
Chromium 119 sql.js (1.8.0) 2023-12-04
NodeJS 20.10.0 better-sqlite3 (9.2.0) 2023-12-04
BunJS 1.0.15 (built-in) 2023-12-04
Deno 1.38.4 sqlite (3.8) 2023-12-04

:::

Demo

The following examples show how to query for each table in an SQLite database, query for the data for each table, add each non-empty table to a workbook, and export as XLSX.

Sample Database

The Chinook database is a MIT-licensed sample database. The original source code repository http://chinookdatabase.codeplex.com is no longer available, so the raw SQL queries are mirrored here.

Exporting Data

Connector libraries typically provide a way to generate an array of objects from the result of a SELECT query. For example, using better-sqlite3 in NodeJS:

import Database from "better-sqlite3";

/* open database */
var db = Database("chinook.db");

/* get data from the `Invoice` table */
var aoo = db.prepare("SELECT * FROM 'Invoice' LIMIT 100000").all();

The SheetJS json_to_sheet method1 can take the result and generate a worksheet object2. The book_new and book_append_sheet methods3 help build a workbook object4. The writeFile method5 generates a file:

import * as XLSX from "xlsx";

/* Create Worksheet from the row objects */
var ws = XLSX.utils.json_to_sheet(aoo, {dense: true});

/* Add to Workbook */
XLSX.utils.book_append_sheet(wb, ws, "Sheet1");

/* Write File */
XLSX.writeFile(wb, "SheetJSQLiteNode.xlsx");

Importing Data

The "Generating Tables" section includes a code snippet for generating SQLite-compatible SQL queries from a SheetJS worksheet object. Each query can be run sequentially.

Browser

sql.js6 is a compiled version of SQLite into WebAssembly, making it usable in web browsers.

SQLite database files can be fetched and loaded:

/* Load sql.js library */
const SQL = await initSqlJs(config);
/* fetch sqlite database */
const ab = await (await fetch("/sqlite/chinook.db")).arrayBuffer();
/* connect to DB */
const db = new SQL.Database(new Uint8Array(ab));

The sql.js connector library uses an iterator-like interface. After preparing a statement, Statement#step loops over the result and Statement#getAsObject pulls each row as a row object:

/* perform query and get iterator */
const sql = db.prepare("SELECT * FROM 'Invoice' LIMIT 100000").all();

/* create worksheet from the row objects */
let ws;

while(sql.step()) {
  const row = sql.getAsObject();

  if(!ws) ws = XLSX.utils.json_to_sheet([row], {dense: true, header});
  else XLSX.utils.sheet_add_json(ws, [row], { header, origin: -1, skipHeader: true});
}

Demo

This demo fetches chinook.db, loads into the SQLite engine and performs a series of queries to extract the data. Worksheets are created from the data. A workbook is created from the worksheets and exported to a XLSX file.

function SheetJSQLJS() { return (<button onClick={async() => {
  /* Load sql.js library */
  const config = {
    locateFile: filename => `https://cdnjs.cloudflare.com/ajax/libs/sql.js/1.8.0/${filename}`
  }
  const SQL = await initSqlJs(config);

  /* Initialize database */
  const ab = await (await fetch("/sqlite/chinook.db")).arrayBuffer();
  const db = new SQL.Database(new Uint8Array(ab));

  /* Create new workbook */
  const wb = XLSX.utils.book_new();

  /* Get all table names */
  const sql = db.prepare("SELECT name FROM sqlite_master WHERE type='table'");
  while(sql.step()) {
    const row = sql.getAsObject();

    /* Get first 100K rows */
    const stmt = db.prepare("SELECT * FROM '" + row.name + "' LIMIT 100000");
    let header = [];
    let ws;
    while(stmt.step()) {
      /* create worksheet from headers */
      if(!ws) ws = XLSX.utils.aoa_to_sheet([header = stmt.getColumnNames()])

      const rowobj = stmt.getAsObject();
      /* add to sheet */
      XLSX.utils.sheet_add_json(ws, [rowobj], { header, origin: -1, skipHeader: true });
    }
    if(ws) XLSX.utils.book_append_sheet(wb, ws, row.name);
  }
  XLSX.writeFile(wb, "SheetJSQLJS.xlsx");
}}><b>Click here to start</b></button>) }

Server-Side Platforms

NodeJS

The better-sqlite37 native module embeds the SQLite C library. Statement#all runs a prepared statement and returns an array of objects:

import Database from "better-sqlite3";
import * as XLSX from "xlsx";

/* open database */
var db = Database("chinook.db");

/* get data from the `Invoice` table */
var aoo = db.prepare("SELECT * FROM 'Invoice' LIMIT 100000").all();

/* create worksheet from the row objects */
var ws = XLSX.utils.json_to_sheet(aoo, {dense: true});

NodeJS Demo

  1. Build chinook.db from the SQL statements:
curl -LO https://docs.sheetjs.com/sqlite/chinook.sql
sqlite3 chinook.db ".read chinook.sql"
  1. Install the dependencies:

{\ npm init -y npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz better-sqlite3@9.2.0}

  1. Download SheetJSQLiteNode.mjs:
curl -LO https://docs.sheetjs.com/sqlite/SheetJSQLiteNode.mjs
  1. Run the script:
node SheetJSQLiteNode.mjs

Open SheetJSQLiteNode.xlsx with a spreadsheet editor.

Bun

Bun ships with a built-in high-performance module bun:sqlite8:

import { Database } from "bun:sqlite";
import * as XLSX from "xlsx";

/* open database */
var db = Database.open("chinook.db");

/* get data from the `Invoice` table */
var aoo = db.prepare("SELECT * FROM 'Invoice' LIMIT 100000").all();

/* create worksheet from the row objects */
var ws = XLSX.utils.json_to_sheet(aoo, {dense: true});

BunJS Demo

  1. Build chinook.db from the SQL statements:
curl -LO https://docs.sheetjs.com/sqlite/chinook.sql
sqlite3 chinook.db ".read chinook.sql"
  1. Install the dependencies:

{\ bun install https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz}

  1. Download SheetJSQLiteBun.mjs:
curl -LO https://docs.sheetjs.com/sqlite/SheetJSQLiteBun.mjs
  1. Run the script:
bun run SheetJSQLiteBun.mjs

Open SheetJSQLiteBun.xlsx with a spreadsheet editor.

Deno

Deno sqlite library9 returns raw arrays of arrays:

{\ import { DB } from "https://deno.land/x/sqlite/mod.ts"; // @deno-types="https://cdn.sheetjs.com/xlsx-${current}/package/types/index.d.ts" import * as XLSX from "https://cdn.sheetjs.com/xlsx-${current}/package/xlsx.mjs"; \n\ /* open database */ var db = new DB("chinook.db"); \n\ /* get data from the \Invoice` table / var aoa = db.prepareQuery("SELECT * FROM 'Invoice' LIMIT 100000").all(); \n
/ create worksheet from the row objects */ var data = [query.columns().map(x => x.name)].concat(aoa); var ws = XLSX.utils.aoa_to_sheet(data, {dense: true});`}

Deno Demo

  1. Build chinook.db from the SQL statements:
curl -LO https://docs.sheetjs.com/sqlite/chinook.sql
sqlite3 chinook.db ".read chinook.sql"
  1. Download SheetJSQLiteDeno.ts:
curl -LO https://docs.sheetjs.com/sqlite/SheetJSQLiteDeno.ts
  1. Run the script:
deno run --allow-read --allow-write SheetJSQLiteDeno.ts

Open SheetJSQLiteDeno.xlsx with a spreadsheet editor.

  1. See json_to_sheet in "Utilities" ↩︎

  2. See "Sheet Objects" in "SheetJS Data Model" for more details. ↩︎

  3. See "Workbook Helpers" in "Utilities" for details on book_new and book_append_sheet. ↩︎

  4. See "Workbook Objects" in "SheetJS Data Model" for more details. ↩︎

  5. See writeFile in "Writing Files" ↩︎

  6. See the sql.js documentation ↩︎

  7. The documentation can be found in the project repository. ↩︎

  8. See "SQLite" in the BunJS documentation. ↩︎

  9. See the sqlite module on the Deno module registry. ↩︎