docs.sheetjs.com/08-nextjs.md at 67294eeeae8d4ba8a354761cfb45aeacea30c293

sheetjs/docs.sheetjs.com

SheetJS b571592055 workarounds for docusaurus breaking changes

2024-04-14 03:40:38 -04:00

23 KiB

Raw Blame History

title	sidebar_label	description	pagination_prev	pagination_next
Sheets in ReactJS Sites with NextJS	NextJS	Make static websites from spreadsheets using NextJS. Seamlessly integrate data into the data layer using SheetJS. Create content without leaving the comfort of Excel.	demos/net/index	demos/mobile/index

import current from '/version.js'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock';

NextJS is a server-side framework for building static and dynamic sites using the ReactJS framework.

SheetJS is a JavaScript library for reading and writing data from spreadsheets.

This discussion covers three key SheetJS + NextJS operations:

Loading Data: NextJS can read files in lifecycle methods OR custom Webpack loaders can create asset modules.
Lifecycle Methods: NextJS includes strategies for static pages (getStaticProps) as well as dynamic pages (getServerSideProps).
Data Presentation: Pages use React and JSX.

The "Demo" uses NextJS and SheetJS to pull data from a spreadsheet. We'll explore how to create asset modules that process spreadsheet data at build time and how to read files on the server in NextJS lifecycle methods.

:::danger Telemetry

NextJS collects telemetry by default. The telemetry subcommand can disable it:

npx next@13.5.6 telemetry disable

The setting can be verified by running

npx next@13.5.6 telemetry status

:::

:::info pass

This demo focuses on static sites where data files are processed at build time.

The ReactJS demo shows NextJS "Client Components".

:::

:::caution Next 13+ and SWC

Next 13 switched to the SWC minifier. There are known issues with the minifier. Until those issues are resolved, SWC should be disabled in next.config.js:

module.exports = {
// highlight-next-line
  swcMinify: false
};

:::

:::note Tested Deployments

This demo was tested in the following environments:

NextJS	NodeJS	Date
`9.5.5`	`16.20.2`	2023-12-04
`10.2.3`	`16.20.2`	2023-12-04
`11.1.4`	`16.20.2`	2023-12-04
`12.3.4`	`20.10.0`	2023-12-04
`13.5.6`	`20.10.0`	2023-12-04
`14.0.3`	`20.10.0`	2023-12-04

:::

Loading Data

At a high level, there are two ways to pull spreadsheet data into NextJS apps: loading an asset module or performing the file read operations from the NextJS lifecycle methods.

Asset modules are appropriate for static sites when the file names are known in advance. Performing file read operations in lifecycle methods is more flexible but does not support live reloading.

Asset Module

:::caution pass

When the demo was last tested, Turbopack did not support true raw loaders. For development use, the normal npx next dev should be used.

:::

The SheetJS NodeJS module can be imported in Webpack asset modules¹.

The following diagram depicts the workbook waltz:

flowchart LR
  file[(workbook\nfile)]
  subgraph SheetJS operations
    base64(base64\nstring)
    aoo(array of\nobjects)
  end
  html{{HTML\nTABLE}}
  file --> |base64-loader.js\ncustom plugin| base64
  base64 --> |page\nlifecycle method| aoo
  aoo --> |page\nIndex method| html

In this flow, it is strongly recommended to make a loader return a Base64 string:

function loader(content) {
  /* since `loader.raw` is true, `content` is a Buffer */
  return `export default '${content.toString("base64")}'`;
}
/* ensure the function receives a Buffer */
loader.raw = true;
module.exports = loader;

The webpack configuration is controlled in next.config.js:

module.exports = {
  webpack: (config) => {
    // highlight-start
    /* add to the webpack config module.rules array */
    config.module.rules.push({
      /* `test` matches file extensions */
      test: /\.(numbers|xls|xlsx|xlsb)/,
      /* use the loader script */
      use: [ { loader: './base64-loader' } ]
    });
    // highlight-end
    return config;
  }
};

Module alias directories can be defined in jsconfig.json or tsconfig.json:

{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      // highlight-next-line
      "@/*": ["*"]
    }
  }
}

Pages can import the files directly. It is strongly recommended to store files in a data folder.

In this example, the import statement pulls the sheetjs.xlsx file as a Base64 string. The SheetJS read method² parses the string and returns a workbook object³. The sheet_to_json⁴ utility function generates an array of objects based on the data. As long as the base64 variable is only used in getStaticProps, the library and file will be processed at build time.

import { read, utils } from 'xlsx';
// highlight-next-line
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticProps() {
  /* parse base64 data */
  // highlight-next-line
  const wb = read(base64, { type: "base64" });
  return { props: {
      /* generate array of objects from the first sheet */
      data: utils.sheet_to_json(wb.Sheets[wb.SheetNames[0]])
  } };
}

Raw Operations

The SheetJS NodeJS module can be imported from page scripts.

:::danger pass

The SheetJS ESM build does not load NodeJS native modules directly. The Installation section includes a note on dynamic import of fs within lifecycle methods.

:::

Files can be read using the SheetJS readFile⁵ method in lifecycle methods. The cwd method in the process module will point to the root of the project.

The following diagram depicts the workbook waltz:

flowchart LR
  file[(workbook\nfile)]
  subgraph SheetJS operations
    buffer(NodeJS\nBuffer)
    aoo(array of\nobjects)
  end
  html{{HTML\nTABLE}}
  file --> |page\nlifecycle method| buffer
  buffer --> |page\nlifecycle method| aoo
  aoo --> |page\nIndex method| html

This example reads the file sheetjs.xlsx in the data folder in the project and uses sheet_to_json⁶ to generate data rows.

import { readFile, utils, set_fs } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getServerSideProps() {
// highlight-start
  set_fs(await import("fs")); // dynamically import 'fs' when needed
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(filename);
  // highlight-end

  /* generate and return the html from the first worksheet */
  const data = utils.sheet_to_json(wb.Sheets[wb.SheetNames[0]]);
  return { props: { data } };
}

:::danger Reading and writing files during the build process

As the NextJS workaround is non-traditional, it bears repeating:

fs cannot be statically imported from the top level in NextJS pages. The dynamic import must happen within a lifecycle function.

:::

NextJS Strategies

NextJS currently provides 3 strategies:

"Static Site Generation" using getStaticProps⁷
"SSG with Dynamic Routes" using getStaticPaths⁸
"Server-Side Rendering" using getServerSideProps⁹

Static Site Generation

When using getStaticProps, the file will be read once during build time. This example reads sheetjs.xlsx from the data folder:

import { read, utils } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticProps() {
  const wb = read(base64, { type: "base64" });

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

import { readFile, set_fs, utils } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getStaticProps() {
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(filename);

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

Dynamic Routes

Typically a static site with dynamic routes has an endpoint /sheets/[id] that implements both getStaticPaths and getStaticProps.

getStaticPaths should return an array of worksheet indices:

import { read } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticPaths() {
  /* read file */
  const wb = read(base64, { type: "base64" });

  /* generate an array of objects that will be used for generating pages */
  const paths = wb.SheetNames.map((name, idx) => ({ params: { id: idx.toString() } }));
  return { paths, fallback: false };
};

import { readFile, set_fs } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getStaticPaths() {
  /* read file */
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(path);

  /* generate an array of objects that will be used for generating pages */
  const paths = wb.SheetNames.map((name, idx) => ({ params: { id: idx.toString() } }));
  return { paths, fallback: false };
};

:::note pass

For a pure static site, fallback must be set to false¹⁰

:::

getStaticProps will generate the actual HTML for each page:

import { read, utils } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticProps(ctx) {
  /* read file */
  const wb = read(base64, { type: "base64" });

  /* get the corresponding worksheet and generate HTML */
  const ws = wb.Sheets[wb.SheetNames[ctx.params.id]]; // id from getStaticPaths
  const html = utils.sheet_to_html(ws);
  return { props: { html } };
};

import { readFile, set_fs, utils } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getStaticProps(ctx) {
  /* read file */
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(path);

  /* get the corresponding worksheet and generate HTML */
  const ws = wb.Sheets[wb.SheetNames[ctx.params.id]]; // id from getStaticPaths
  const html = utils.sheet_to_html(ws);
  return { props: { html } };
};

Server-Side Rendering

:::caution Do not use on a static site

These routes require a NodeJS dynamic server. Static page generation will fail!

getStaticProps and getStaticPaths support static site generation (SSG).

getServerSideProps is suited for NodeJS hosted deployments where the workbook changes frequently and a static site is undesirable.

:::

When using getServerSideProps, the file will be read on each request.

:::caution Consider using a static strategy

When using asset modules, the file names and file paths are processed during the build step. The content is fixed. In this situation, a static approach such as getStaticProps is strongly recommended.

:::

import { read } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getServerSideProps() {
  /* read file */
  const wb = read(base64, { type: "base64" });

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

import { readFile, set_fs, utils } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getServerSideProps() {
  /* read file */
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(path);

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

Data Presentation

The ReactJS demo compares common approaches.

HTML

HTML output can be generated using the SheetJS sheet_to_html¹¹ method and inserted into the document using the dangerouslySetInnerHTML¹² attribute:

flowchart LR
  subgraph SheetJS operations
    data(File\nData)
    code{{HTML\nTABLE}}
  end
  html{{Rendered\nPage}}
  data --> |lifecycle\nsheet_to_html| code
  code --> |Index\ninnerHTML| html

export default function Index({html, type}) { return (
  <div dangerouslySetInnerHTML={{ __html: html }} />
); }

Arrays of Objects

Arrays of objects can be generated using the SheetJS sheet_to_json¹³ method and inserted into the document using standard JSX¹⁴:

flowchart LR
  subgraph SheetJS operations
    data(File\nData)
    aoo(array of\nobjects)
  end
  html{{Rendered\nPage}}
  data --> |lifecycle\nsheet_to_json| aoo
  aoo --> |Index\nReact + JSX| html

export default function Index({aoo, type}) { return (
  <table><thead><tr key={0}><th>Name</th><th>Index</th></tr></thead><tbody>
// highlight-start
    {aoo.map(row => ( <tr>
      <td>{row.Name}</td>
      <td>{row.Index}</td>
    </tr>))}
// highlight-end
  </tbody></table>
); }

Demo

:::note pass

This demo showcases the following SheetJS + NextJS flows:

Page	Loading Data	Lifecycle Method	SheetJS API
`/getStaticProps`	asset module	`getStaticProps`	`sheet_to_json`
`/sheets/[id]`	asset module	`getStaticPaths`	`sheet_to_html`
`/getServerSideProps`	lifecycle	`getServerSideProps`	`sheet_to_html`

The commands in this demo use next@13.5.6. Other versions were tested by replacing the version number in the relevant commands.

:::

:::caution pass

Older versions of NextJS will refuse to run in newer versions of NodeJS. The error message points to an issue with OpenSSL:

Error: error:0308010C:digital envelope routines::unsupported

When upgrading NextJS is not an option, NodeJS should be downgraded to v16.

:::

Initial Setup

Disable NextJS telemetry:

npx next@13.5.6 telemetry disable

Confirm it is disabled by running

npx next@13.5.6 telemetry status

Set up folder structure. At the end, a pages folder with a sheets subfolder must be created. On Linux or MacOS or WSL:

mkdir sheetjs-next
cd sheetjs-next
mkdir -p pages/sheets/

Download the test file and place in the project root. On Linux or MacOS or WSL:

curl -LO https://docs.sheetjs.com/next/sheetjs.xlsx

Install dependencies:

:::note pass

The next@13.5.6 dependency can be adjusted to pick a different version. For example, NextJS 12.3.4 is installed with

{\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz next@12.3.4}

:::

{\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz next@13.5.6}

Download NextJS config scripts and place in the root folder:

On Linux or MacOS or WSL:

curl -LO https://docs.sheetjs.com/next/base64-loader.js
curl -LO https://docs.sheetjs.com/next/jsconfig.json
curl -LO https://docs.sheetjs.com/next/next.config.js
curl -LO https://docs.sheetjs.com/next/styles.css

Download test scripts:

Download and place the following scripts in the pages subfolder:

Download [id].js and place in the pages/sheets subfolder.

:::caution Percent-Encoding in the script name

The [id].js script must have the literal square brackets in the name. If your browser saved the file to %5Bid%5D.js. rename the file.

:::

On Linux or MacOS or WSL:

cd pages
curl -LO https://docs.sheetjs.com/next/_app.js
curl -LO https://docs.sheetjs.com/next/index.js
curl -LO https://docs.sheetjs.com/next/getServerSideProps.js
curl -LO https://docs.sheetjs.com/next/getStaticPaths.js
curl -LO https://docs.sheetjs.com/next/getStaticProps.js
cd sheets
curl -LOg 'https://docs.sheetjs.com/next/[id].js'
cd ../..

Testing

Test the deployment:

npx next

Open a web browser and access:

http://localhost:3000 landing page
http://localhost:3000/getStaticProps shows data from the first sheet
http://localhost:3000/getServerSideProps shows data from the first sheet
http://localhost:3000/getStaticPaths shows a list (2 sheets)

The individual worksheets are available at

http://localhost:3000/sheets/0
http://localhost:3000/sheets/1

While the development server is running, open the /getStaticProps page and open sheetjs.xlsx with a spreadsheet editor. In the editor, add a row to the bottom of the "Indices" worksheet (set A7 to "SheetJS Dev" and B7 to 47)

After saving the file, the website should refresh with the new row.

Production Build

Stop the server and run a production build:

npx next build

The final output will show a list of the routes and types:

Route (pages)                              Size     First Load JS
┌ ○ /                                      563 B          75.3 kB
├   /_app                                  0 B            74.8 kB
├ ○ /404                                   182 B            75 kB
├ λ /getServerSideProps                    522 B          75.3 kB
├ ● /getStaticPaths                        2.91 kB        77.7 kB
├ ● /getStaticProps                        586 B          75.4 kB
└ ● /sheets/[id] (303 ms)                  522 B          75.3 kB
    ├ /sheets/0
    └ /sheets/1

As explained in the summary, the /getStaticPaths and /getStaticProps routes are completely static. 2 /sheets/# pages were generated, corresponding to 2 worksheets in the file. /getServerSideProps is server-rendered.

Try to build a static site:

npx next export

:::danger NextJS breaking changes

NextJS 14 removed the export subcommand!

:::

Edit next.config.js and add the highlighted line:

module.exports = {
  // highlight-next-line
  output: "export",
  webpack: (config) => {

After adding the line, run the build command:

npx next build

This build will fail. A static page cannot be generated at this point because /getServerSideProps is server-rendered.

Static Site

Delete pages/getServerSideProps.js and rebuild:

Edit next.config.js and comment the highlighted line:

module.exports = {
  // highlight-next-line
  // output: "export",
  webpack: (config) => {

After editing next.config.js:

rm -f pages/getServerSideProps.js
npx next build

Inspecting the output, there should be no lines with the λ symbol:

Route (pages)                              Size     First Load JS
┌ ○ /                                      563 B          75.3 kB
├   /_app                                  0 B            74.8 kB
├ ○ /404                                   182 B            75 kB
├ ● /getStaticPaths                        2.91 kB        77.7 kB
├ ● /getStaticProps                        586 B          75.4 kB
└ ● /sheets/[id]                           522 B          75.3 kB
    ├ /sheets/0
    └ /sheets/1

Generate the static site:

npx next export

Edit next.config.js and restore the highlighted line:

module.exports = {
  // highlight-next-line
  output: "export",
  webpack: (config) => {

After adding the line, run the build command:

npx next build

The static site will be written to the out subfolder

Serve the static site:

npx http-server out

The command will start a local HTTP server at http://localhost:8080/ for testing the generated site. Note that /getServerSideProps will 404 since the page was removed.

See the "Webpack" asset module demo for more details. ↩︎
See read in "Reading Files" ↩︎
See "SheetJS Data Model" for more details. ↩︎
See sheet_to_json in "Utilities" ↩︎
See readFile in "Reading Files" ↩︎
See sheet_to_json in "Utilities" ↩︎
See getStaticProps in the NextJS documentation. ↩︎
See getStaticPaths in the NextJS documentation. ↩︎
See getServerSideProps in the NextJS documentation. ↩︎
See fallback in getStaticPaths in the NextJS documentation. ↩︎
See sheet_to_html in "Utilities" ↩︎
dangerouslySetInnerHTML is a ReactJS prop supported for all built-in components. ↩︎
See sheet_to_json in "Utilities" ↩︎
See "Array of Objects" in the ReactJS demo ↩︎

23 KiB Raw Blame History

Loading Data

Asset Module

Raw Operations

NextJS Strategies

Static Site Generation

Dynamic Routes

Server-Side Rendering

Data Presentation

HTML

Arrays of Objects

Demo

Initial Setup

Testing

Production Build

Static Site

23 KiB

Raw Blame History