13 KiB

Raw Blame History

sidebar_position
5

Contributing

Due to the precarious nature of the Open Specifications Promise, it is very important to ensure code is cleanroom. Contribution Notes

File organization (click to show)

Folders:

folder	contents
`bin`	server-side bin scripts (`xlsx.njs`)
`bits`	raw source files that make up the final script
`dist`	dist files for web browsers and nonstandard JS environments
`misc`	miscellaneous supporting scripts
`modules`	TypeScript source files that generate some of the bits
`packages`	Support libraries and tools
`test_files`	test files (pulled from the test files repository)
`tests`	browser tests (run `make ctest` to rebuild)
`types`	TypeScript definitions and tests

After cloning the repo, running make help will display a list of commands.

Setup

These instructions will cover system configuration, cloning the source repo, building, reproducing official releases, and running NodeJS and browser tests.

:::note

These instructions were tested on the following platforms:

Platform	Test Date
Linux (Steam Deck Holo x64)	2023-09-22
Linux (Ubuntu 18 AArch64)	2023-09-07
MacOS 10.13.6 (x64)	2023-09-30
MacOS 13.6 (ARM64)	2023-09-30
Windows 10 (x64) + WSL Ubuntu	2023-07-23
Windows 11 (x64) + WSL Ubuntu	2023-08-31
Windows 11 (ARM) + WSL Ubuntu	2023-09-18

With some additional dependencies, the unminified scripts are reproducible and tests will pass in Windows XP with NodeJS 5.10.0.

:::

Install dependencies

OS-Specific Setup

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

A) Ensure WSL ("WSL 2" in Windows 10) and the Ubuntu distribution are installed.

B) Install mercurial and subversion:

sudo apt-get update
sudo apt-get install mercurial subversion

:::note pass

In some Windows 10 runs, mercurial and subversion were not available in the default Ubuntu distro. A separate repository is available:

# Install support programs for the build and test commands
sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial subversion
sudo add-apt-repository --remove ppa:mercurial-ppa/releases

If the first command displays an error involving a missing release file, remove the repo before proceeding:

sudo add-apt-repository --remove ppa:mercurial-ppa/releases

:::

C) Install NodeJS

:::info pass

In the most recent test, the script showed a deprecation notice.

The script worked as expected.

The official workaround does not currently work with WSL. When the issues are resolved, the instructions will be updated.

:::

# Install bootstrap NodeJS and NPM within the WSL
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs

Exit the WSL window and open a new one before proceeding:

# Switch to `n`-managed NodeJS
sudo npm i -g n
sudo n 16

:::note pass

If npm is missing, it can be installed with

sudo apt-get install -y npm

:::

D) Test clone the js-crc32 repo

git clone https://git.sheetjs.com/sheetjs/js-crc32

If this clone fails with an error message that mentions SSL or secure connection or certificates, build and install a version of Git with proper SSL support:

# Git does not support OpenSSL out of the box, must do this
curl -LO https://github.com/niko-dunixi/git-openssl-shellscript/raw/main/compile-git-with-openssl.sh
chmod +x compile-git-with-openssl.sh
./compile-git-with-openssl.sh

E) Set git config core.autocrlf setting to false. The following commands should be run twice, once within PowerShell (if Git for Windows is installed) and once within WSL bash:

git config --global --add core.autocrlf false
git config --global --unset core.autocrlf true

F) Run unzip. If the program is missing, install manually:

sudo apt-get install -y unzip

A) Open a terminal window and run git.

If Xcode or the command-line tools are not installed, you will be asked to install. Click "Install" and run through the steps.

B) Open a terminal window and install the Homebrew package manager:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

C) Close the window, open a new terminal window, and disable analytics:

brew analytics off

To confirm analytics are disabled, run

brew analytics state

The message should state that analytics are disabled or destroyed.

D) Install Mercurial and Subversion:

brew install mercurial subversion

E) Install NodeJS.

:::note pass

The official NodeJS site provides installers for "LTS" and "Current" releases. The "LTS" version should be installed.

Older versions of macOS are not compatible with newer versions of NodeJS.

In local testing, macOS 10.13 required NodeJS version 12.22.12:

curl -LO https://nodejs.org/download/release/v12.22.12/node-v12.22.12.pkg 
open node-v12.22.12.pkg

:::

A) Install curl, mercurial, git, and subversion using the system package manager. On Debian and Ubuntu systems, apt-get installs packages:

sudo apt update
sudo apt-get install curl git mercurial subversion

Other Linux distributions may use other package managers.

Steam Deck (click to show)

Desktop Mode on the Steam Deck uses pacman. It also requires a few steps.

Switch to Desktop mode and open Konsole
Set a password for the user by running passwd and following instructions.
Disable read-only mode:

sudo steamos-readonly disable

(When prompted, enter the password assigned in step 1)

Configure keyring:

sudo sh -c 'echo "keyserver hkps://keyserver.ubuntu.com" >> /etc/pacman.d/gnupg/gpg.conf'
sudo pacman-key --init
sudo pacman-key --populate
sudo pacman-key --refresh-keys

Install dependencies:

sudo pacman -S base-devel mercurial subversion

:::note pass

In local testing on the Steam Deck, some of the C / C++ demos failed to build. This issue was resolved by manually installing glibc and linux-api-headers:

sudo pacman -S glibc linux-api-headers

This is not required for building or testing the library.

:::

After installing mercurial and subversion, install NodeJS.

:::note pass

The official NodeJS site provides installers for "LTS" and "Current" releases. The "LTS" version should be installed.

After installing, if running node in the terminal fails with a glibc error, an older version of NodeJS should be installed. For example, Ubuntu 18.04 does not support Node 18 support Node 16.20.0.

:::

Build from source tree

Clone the project:

git clone https://git.sheetjs.com/sheetjs/sheetjs
cd sheetjs

:::note pass

On older platforms, the clone may fail due to SSL certificate problems:

fatal: unable to access 'https://git.sheetjs.com/sheetjs/sheetjs/': SSL certificate problem: certificate has expired

The simplest workaround is to disable SSL verification:

git config --global http.sslVerify false

It is strongly recommended to re-enable SSL verification after cloning:

git config --global http.sslVerify true

:::

Install NodeJS modules for building the scripts:

npm i
sudo npm i -g mocha@2.5.3 voc @sheetjs/uglify-js

:::note Older Versions of Dependencies

Some of the dependencies are wildly out of date. While SheetJS aims to run in older versions of NodeJS and browsers, some libraries have chosen to break backwards compatibility. The specific versions are used because they are known to work and known to produce consistent results.

:::

Initialize the test files:

make init

This step may take a while as it will be downloading a number of test files.

Run the esbuild tool once:

npx -y esbuild@0.14.14

Run a build and verify with a short test:

# Full Build
cd modules; make clean; make; cd ..
make
make dist

# Short test
make test_misc

# Reset repo
git checkout -- .

:::info pass

In some tests on older releases of macOS, the build failed with an error:

ReferenceError: n is not defined

The first error in the call stack points to dist/xlsx.zahl.js.

Older versions of macOS sed are known to misinterpret newline characters. The workaround is to upgrade to a newer version of sed. On macOS:

brew install gnu-sed
echo 'export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"' >> ~/.profile
. ~/.profile

:::

Reproduce official builds

Run git log and search for the commit that matches a particular release version. For example, version 0.20.0 can be found with:

git log | grep -B4 "version bump 0.20.0"

The output should look like:

$ git log | grep -B4 "version bump 0.20.0"
# highlight-next-line
commit 955543147dac0274d20307057c5a9f3e3e5d5307 <-- this is the commit hash
Author: SheetJS <dev@sheetjs.com>
Date:   Fri Jun 23 05:48:47 2023 -0400

    version bump 0.20.0

Switch to that commit:

git checkout 955543147dac0274d20307057c5a9f3e3e5d5307

Run the full build sequence

make clean; make
cd modules; make clean; make; cd ..
make
make dist

To verify that the files are intact, use md5sum (md5 on MacOS).

The local checksum for the browser script can be computed with:

md5sum dist/xlsx.full.min.js

md5 dist/xlsx.full.min.js

md5sum dist/xlsx.full.min.js

The checksum for the CDN version can be computed with:

curl -L https://cdn.sheetjs.com/xlsx-0.20.0/package/dist/xlsx.full.min.js | md5sum -

curl -k -L https://cdn.sheetjs.com/xlsx-0.20.0/package/dist/xlsx.full.min.js | md5

curl -L https://cdn.sheetjs.com/xlsx-0.20.0/package/dist/xlsx.full.min.js | md5sum -

When the demo was last tested on macOS, against version 0.20.0:

$ md5 dist/xlsx.full.min.js
# highlight-next-line
MD5 (dist/xlsx.full.min.js) = 0b2f539797f92d35c6394274818f2c22
$ curl -k -L https://cdn.sheetjs.com/xlsx-0.20.0/package/dist/xlsx.full.min.js | md5
# highlight-next-line
0b2f539797f92d35c6394274818f2c22

The two hashes should match.

Return to the HEAD commit:

git checkout master

Test in web browsers

Start local server:

make ctestserv

The terminal will display a port number. For example:

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/)

Open a browser window and access http://localhost:8000, replacing 8000 with the port number from the terminal window.

Development

The xlsx.js and xlsx.mjs files are constructed from the files in the bits subfolder. The build script (run make) will concatenate the individual bits to produce the scripts.

When changing the .js scripts in bits, the following sequence rebuilds the xlsx.js and xlsx.mjs scripts:

make

When changing the .ts scripts in modules, the following sequence rebuilds the xlsx.js and xlsx.mjs scripts:

cd modules; make clean; make; cd ..

To produce the dist files, run make dist.

:::info pass

The various xlsx.* scripts in the base folder and the files in the dist folder are updated on each version release.

They should not be committed between versions!

:::

Tests

The test_misc target runs the targeted feature tests. It should take 5-10 seconds to perform feature tests without testing against the full test battery. New features should be accompanied with tests for the relevant file formats.

For tests involving the read side, an appropriate feature test would involve reading an existing file and checking the resulting workbook object. If a parameter is involved, files should be read with different values to verify that the feature is working as expected.

For tests involving a new write feature which can already be parsed, appropriate feature tests would involve writing a workbook with the feature and then opening and verifying that the feature is preserved.

For tests involving a new write feature without an existing read ability, please add a feature test to the kitchen sink tests/write.js.

13 KiB Raw Blame History

Contributing

Setup

Install dependencies

OS-Specific Setup

Build from source tree

Reproduce official builds

Test in web browsers

Development

Tests

13 KiB

Raw Blame History