docs.sheetjs.com/docz/docs/09-miscellany/05-contributing.md

429 lines
10 KiB
Markdown
Raw Normal View History

2022-05-16 03:26:04 +00:00
---
2022-06-21 12:26:53 +00:00
sidebar_position: 5
2022-05-16 03:26:04 +00:00
---
# Contributing
Due to the precarious nature of the Open Specifications Promise, it is very
2022-10-19 10:05:59 +00:00
important to ensure code is cleanroom. [Contribution Notes](https://git.sheetjs.com/sheetjs/sheetjs/src/branch/master/CONTRIBUTING.md)
2022-05-16 03:26:04 +00:00
<details>
<summary><b>File organization</b> (click to show)</summary>
Folders:
| folder | contents |
|:-------------|:--------------------------------------------------------------|
| `bin` | server-side bin scripts (`xlsx.njs`) |
2022-09-05 10:00:35 +00:00
| `bits` | raw source files that make up the final script |
2022-05-16 03:26:04 +00:00
| `dist` | dist files for web browsers and nonstandard JS environments |
| `misc` | miscellaneous supporting scripts |
2022-09-05 10:00:35 +00:00
| `modules` | TypeScript source files that generate some of the bits |
| `packages` | Support libraries and tools |
2022-05-16 03:26:04 +00:00
| `test_files` | test files (pulled from the test files repository) |
2022-09-05 10:00:35 +00:00
| `tests` | browser tests (run `make ctest` to rebuild) |
| `types` | TypeScript definitions and tests |
2022-05-16 03:26:04 +00:00
</details>
After cloning the repo, running `make help` will display a list of commands.
2023-01-14 03:13:35 +00:00
## Setup
These instructions will cover system configuration, cloning the source repo,
building, reproducing official releases, and running NodeJS and browser tests.
2023-04-05 03:17:05 +00:00
:::note
These instructions were tested on the following platforms:
| Platform | Test Date |
|:------------------------------|:-----------|
2023-07-19 03:27:47 +00:00
| Linux (Steam Deck Holo 3.4.8) | 2023-07-12 |
2023-08-30 03:44:38 +00:00
| Linux (Debian 11 AArch64) | 2023-08-29 |
2023-04-05 03:17:05 +00:00
| MacOS 10.13 (x64) | 2023-04-04 |
2023-08-23 08:05:30 +00:00
| MacOS 13.0 (ARM64) | 2023-04-13 |
2023-07-25 02:31:22 +00:00
| Windows 10 (x64) + WSL Ubuntu | 2023-07-23 |
2023-08-31 22:09:08 +00:00
| Windows 11 (x64) + WSL Ubuntu | 2023-08-31 |
2023-04-05 03:17:05 +00:00
2023-07-25 02:31:22 +00:00
With some additional dependencies, the unminified build is reproducible and
tests will pass in Windows XP with NodeJS 5.10.0.
2023-04-05 03:17:05 +00:00
:::
2023-01-14 03:13:35 +00:00
### Install dependencies
#### OS-Specific Setup
2022-05-16 03:26:04 +00:00
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs>
2022-05-17 03:19:39 +00:00
<TabItem value="wsl" label="Windows WSL">
2023-04-05 03:17:05 +00:00
A) Ensure WSL ("WSL 2" in Windows 10) and the Ubuntu distribution are installed.
B) Install mercurial and subversion.
2022-05-16 03:26:04 +00:00
```bash
2022-05-17 03:19:39 +00:00
# Install support programs for the build and test commands
sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial subversion
sudo add-apt-repository --remove ppa:mercurial-ppa/releases
2022-05-16 03:26:04 +00:00
```
2023-04-05 03:17:05 +00:00
C) Install NodeJS
2022-05-16 03:26:04 +00:00
2023-08-31 22:09:08 +00:00
:::info pass
When this was last tested, the script showed a deprecation notice.
**The script worked as expected.**
The official workaround does not currently work with WSL. When the issues are
resolved, the instructions will be updated.
:::
2022-05-17 03:19:39 +00:00
```bash
# Install bootstrap NodeJS and NPM within the WSL
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs
2023-01-14 03:13:35 +00:00
```
2022-05-16 03:26:04 +00:00
2023-01-14 03:13:35 +00:00
Exit the WSL window and open a new one before proceeding:
2022-05-16 03:26:04 +00:00
2023-01-14 03:13:35 +00:00
```bash
2022-05-17 03:19:39 +00:00
# Switch to `n`-managed NodeJS
sudo npm i -g n
sudo n 16
2022-05-16 03:26:04 +00:00
```
2023-07-25 02:31:22 +00:00
:::note pass
If `npm` is missing, it can be installed with
```bash
sudo apt-get install -y npm
```
:::
2023-04-05 03:17:05 +00:00
D) Build and install a version of Git with proper SSL support:
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
```bash
# Git does not support OpenSSL out of the box, must do this
curl -LO https://github.com/paul-nelson-baker/git-openssl-shellscript/raw/main/compile-git-with-openssl.sh
chmod +x compile-git-with-openssl.sh
./compile-git-with-openssl.sh
2022-05-16 03:26:04 +00:00
```
2023-04-05 03:17:05 +00:00
E) Set `git` config `core.autocrlf` setting to `false`. The following commands
2023-01-14 03:13:35 +00:00
should be run twice, once within PowerShell and once within WSL bash:
2023-05-11 21:28:29 +00:00
```bash
2023-01-14 03:13:35 +00:00
git config --global --add core.autocrlf false
git config --global --unset core.autocrlf true
2023-07-25 02:31:22 +00:00
```
F) Run `unzip`. If the program is missing, install manually:
```bash
sudo apt-get install -y unzip
2023-01-14 03:13:35 +00:00
```
2022-05-17 03:19:39 +00:00
</TabItem>
2023-01-14 03:13:35 +00:00
<TabItem value="osx" label="MacOS">
A) Run `git`. If Xcode or the command-line tools are not installed, you will be
asked to install. Click "Install" and run through the steps.
B) Open a terminal window and install Homebrew:
```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
C) Close the window, open a new terminal window, and disable analytics:
```bash
brew analytics off
```
To confirm analytics are disabled, run
```bash
brew analytics state
```
It should print `Analytics are disabled.`
2022-05-17 03:19:39 +00:00
2023-01-14 03:13:35 +00:00
D) Install Mercurial and Subversion:
```bash
brew install mercurial subversion
```
2022-05-17 03:19:39 +00:00
2023-01-14 03:13:35 +00:00
E) Install NodeJS
2022-05-17 03:19:39 +00:00
2022-11-12 09:16:51 +00:00
:::note
[The official NodeJS site](https://nodejs.org/en/download/) provides installers
for "LTS" and "Current" releases. The "LTS" version should be installed.
2023-04-05 03:17:05 +00:00
Older versions of macOS are not compatible with newer versions of NodeJS. In
local testing, macOS 10.13 required NodeJS version `12.22.12`
2022-11-12 09:16:51 +00:00
:::
2023-01-14 03:13:35 +00:00
</TabItem>
2022-11-12 09:16:51 +00:00
<TabItem value="l" label="Linux">
2023-04-14 08:13:40 +00:00
A) Install `curl`, `mercurial`, `git`, and `subversion` using the system package
2023-01-14 03:13:35 +00:00
manager. On Debian and Ubuntu systems, `apt-get` installs packages:
2022-05-17 03:19:39 +00:00
```bash
2023-04-14 08:13:40 +00:00
sudo apt update
sudo apt-get install curl git mercurial subversion
2022-05-16 03:26:04 +00:00
```
2022-11-12 09:16:51 +00:00
Other Linux distributions may use other package managers.
2023-01-14 03:13:35 +00:00
<details open><summary><b>Steam Deck</b> (click to show)</summary>
2022-11-12 09:16:51 +00:00
Desktop Mode on the Steam Deck uses `pacman`. It also requires a few steps.
0) Switch to Desktop mode and open `Konsole`
1) Set a password for the user by running `passwd` and following instructions.
2) Disable read-only mode:
```bash
sudo steamos-readonly disable
```
(When prompted, enter the password assigned in step 1)
3) Configure keyring:
```bash
2023-03-26 00:55:35 +00:00
sudo sh -c 'echo "keyserver hkps://keyserver.ubuntu.com" >> /etc/pacman.d/gnupg/gpg.conf'
2022-11-12 09:16:51 +00:00
sudo pacman-key --init
sudo pacman-key --populate
sudo pacman-key --refresh-keys
```
4) Install dependencies:
```bash
2023-03-26 00:55:35 +00:00
sudo pacman -S base-devel mercurial subversion
2022-11-12 09:16:51 +00:00
```
2023-06-03 09:10:50 +00:00
:::note
In local testing on the Steam Deck, some of the C / C++ demos failed to build.
This issue was resolved by manually installing `glibc` and `linux-api-headers`:
```bash
sudo pacman -S glibc linux-api-headers
```
This is *not required* for building or testing the library.
:::
2022-11-12 09:16:51 +00:00
</details>
2023-01-14 03:13:35 +00:00
After installing mercurial and subversion, install NodeJS.
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
:::note
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
[The official NodeJS site](https://nodejs.org/en/download/) provides installers
for "LTS" and "Current" releases. The "LTS" version should be installed.
2022-11-12 09:16:51 +00:00
2023-04-14 08:13:40 +00:00
After installing, if running `node` in the terminal fails with a `glibc` error,
an older version of NodeJS should be installed. For example, Ubuntu 18.04 does
not support Node 18 support Node 16.20.0.
2023-01-14 03:13:35 +00:00
:::
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
</TabItem>
</Tabs>
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
### Build from source tree
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
0) Clone the project:
2022-05-16 03:26:04 +00:00
```bash
2023-01-14 03:13:35 +00:00
git clone https://git.sheetjs.com/sheetjs/sheetjs
cd sheetjs
2022-05-17 03:19:39 +00:00
```
2023-08-31 22:09:08 +00:00
1) Install NodeJS modules for building the scripts:
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
```bash
2023-05-07 13:58:36 +00:00
npm i
2022-07-25 23:18:00 +00:00
sudo npm i -g mocha@2.5.3 voc @sheetjs/uglify-js
2022-05-17 03:19:39 +00:00
```
2022-11-12 09:16:51 +00:00
:::note Older Versions of Dependencies
Some of the dependencies are wildly out of date. While SheetJS aims to run in
older versions of NodeJS and browsers, some libraries have chosen to break
backwards compatibility. The specific versions are used because they are known
to work and known to produce consistent results.
:::
2022-05-17 03:19:39 +00:00
2) Initialize the test files:
```bash
make init
```
This step may take a while as it will be downloading a number of test files.
2023-08-30 03:44:38 +00:00
3) Run a build and verify with a short test:
2022-05-17 03:19:39 +00:00
```bash
# Full Build
2023-03-26 00:55:35 +00:00
cd modules; make clean; make; cd ..
2023-08-30 03:44:38 +00:00
make
2022-05-17 03:19:39 +00:00
make dist
2023-04-05 03:17:05 +00:00
2023-08-30 03:44:38 +00:00
# Short test
make test_misc
2023-04-05 03:17:05 +00:00
# Reset repo
git checkout -- .
2022-05-20 03:25:45 +00:00
```
2023-01-14 03:13:35 +00:00
### Reproduce official builds
4) Run `git log` and search for the commit that matches a particular release
2023-06-25 09:36:58 +00:00
version. For example, version `0.20.0` can be found with:
2022-05-20 03:25:45 +00:00
```bash
2023-06-25 09:36:58 +00:00
git log | grep -B4 "version bump 0.20.0"
2022-07-23 09:06:31 +00:00
```
2023-01-14 03:13:35 +00:00
The output should look like:
2022-07-23 09:06:31 +00:00
```bash
2023-06-25 09:36:58 +00:00
$ git log | grep -B4 "version bump 0.20.0"
2023-01-14 03:13:35 +00:00
# highlight-next-line
2023-06-25 09:36:58 +00:00
commit 955543147dac0274d20307057c5a9f3e3e5d5307 <-- this is the commit hash
2023-01-14 03:13:35 +00:00
Author: SheetJS <dev@sheetjs.com>
2023-06-25 09:36:58 +00:00
Date: Fri Jun 23 05:48:47 2023 -0400
2023-01-14 03:13:35 +00:00
2023-06-25 09:36:58 +00:00
version bump 0.20.0
2022-05-16 03:26:04 +00:00
```
2023-01-14 03:13:35 +00:00
5) Switch to that commit:
```bash
2023-06-25 09:36:58 +00:00
git checkout 955543147dac0274d20307057c5a9f3e3e5d5307
2023-01-14 03:13:35 +00:00
```
6) Run the full build sequence
```bash
make clean
2023-03-26 00:55:35 +00:00
cd modules; make clean; make; cd ..
2023-01-14 03:13:35 +00:00
make
make dist
```
7) To verify that the files are intact, use `md5sum` (`md5` on MacOS).
The local checksum for the browser script can be computed with:
```bash
$ md5sum dist/xlsx.full.min.js
2023-06-25 09:36:58 +00:00
0b2f539797f92d35c6394274818f2c22 dist/xlsx.full.min.js
2023-01-14 03:13:35 +00:00
```
The checksum for the CDN version can be computed with:
```bash
2023-06-25 09:36:58 +00:00
$ curl -L https://cdn.sheetjs.com/xlsx-0.20.0/package/dist/xlsx.full.min.js | md5sum -
0b2f539797f92d35c6394274818f2c22 -
2023-01-14 03:13:35 +00:00
```
The two hashes should match.
8) To return to the HEAD commit, run
```bash
2023-03-26 00:55:35 +00:00
git checkout master
2023-01-14 03:13:35 +00:00
```
### Test in web browsers
9) Start local server:
```bash
make ctestserv
```
The terminal will display a port number. For example:
```
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/)
```
2022-05-16 03:26:04 +00:00
2023-01-14 03:13:35 +00:00
10) Open a browser window and access `http://localhost:8000`, replacing `8000`
with the port number from the terminal window.
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
## Development
The `xlsx.js` and `xlsx.mjs` files are constructed from the files in the `bits`
2022-08-24 00:51:18 +00:00
subfolder. The build script (run `make`) will concatenate the individual bits
to produce the scripts.
2022-05-17 03:19:39 +00:00
2023-08-30 03:44:38 +00:00
When changing the `.js` scripts in `bits`, the following sequence rebuilds the
`xlsx.js` and `xlsx.mjs` scripts:
```bash
make
```
When changing the `.ts` scripts in `modules`, the following sequence rebuilds
the `xlsx.js` and `xlsx.mjs` scripts:
```bash
cd modules; make clean; make; cd ..
```
To produce the dist files, run `make dist`.
:::info pass
The various `xlsx.*` scripts in the base folder and the files in the `dist`
folder are updated on each version release.
**They should not be committed between versions!**
:::
2022-05-17 03:19:39 +00:00
## Tests
The `test_misc` target runs the targeted feature tests. It should take 5-10
seconds to perform feature tests without testing against the full test battery.
New features should be accompanied with tests for the relevant file formats.
2022-05-16 03:26:04 +00:00
For tests involving the read side, an appropriate feature test would involve
reading an existing file and checking the resulting workbook object. If a
parameter is involved, files should be read with different values to verify that
the feature is working as expected.
For tests involving a new write feature which can already be parsed, appropriate
feature tests would involve writing a workbook with the feature and then opening
and verifying that the feature is preserved.
For tests involving a new write feature without an existing read ability, please
add a feature test to the kitchen sink `tests/write.js`.