docs.sheetjs.com/docz/docs/09-miscellany/05-contributing.md

338 lines
8.4 KiB
Markdown
Raw Normal View History

2022-05-16 03:26:04 +00:00
---
2022-06-21 12:26:53 +00:00
sidebar_position: 5
2022-05-16 03:26:04 +00:00
---
# Contributing
Due to the precarious nature of the Open Specifications Promise, it is very
2022-10-19 10:05:59 +00:00
important to ensure code is cleanroom. [Contribution Notes](https://git.sheetjs.com/sheetjs/sheetjs/src/branch/master/CONTRIBUTING.md)
2022-05-16 03:26:04 +00:00
<details>
<summary><b>File organization</b> (click to show)</summary>
Folders:
| folder | contents |
|:-------------|:--------------------------------------------------------------|
| `bin` | server-side bin scripts (`xlsx.njs`) |
2022-09-05 10:00:35 +00:00
| `bits` | raw source files that make up the final script |
2022-05-16 03:26:04 +00:00
| `dist` | dist files for web browsers and nonstandard JS environments |
| `misc` | miscellaneous supporting scripts |
2022-09-05 10:00:35 +00:00
| `modules` | TypeScript source files that generate some of the bits |
| `packages` | Support libraries and tools |
2022-05-16 03:26:04 +00:00
| `test_files` | test files (pulled from the test files repository) |
2022-09-05 10:00:35 +00:00
| `tests` | browser tests (run `make ctest` to rebuild) |
| `types` | TypeScript definitions and tests |
2022-05-16 03:26:04 +00:00
</details>
After cloning the repo, running `make help` will display a list of commands.
2023-01-14 03:13:35 +00:00
## Setup
These instructions will cover system configuration, cloning the source repo,
building, reproducing official releases, and running NodeJS and browser tests.
### Install dependencies
#### OS-Specific Setup
2022-05-16 03:26:04 +00:00
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs>
2022-05-17 03:19:39 +00:00
<TabItem value="wsl" label="Windows WSL">
2023-01-14 03:13:35 +00:00
A) Install mercurial and subversion.
2022-05-16 03:26:04 +00:00
```bash
2022-05-17 03:19:39 +00:00
# Install support programs for the build and test commands
sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial subversion
sudo add-apt-repository --remove ppa:mercurial-ppa/releases
2022-05-16 03:26:04 +00:00
```
2023-01-14 03:13:35 +00:00
B) Install NodeJS
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
```bash
# Install bootstrap NodeJS and NPM within the WSL
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs
2023-01-14 03:13:35 +00:00
```
2022-05-16 03:26:04 +00:00
2023-01-14 03:13:35 +00:00
Exit the WSL window and open a new one before proceeding:
2022-05-16 03:26:04 +00:00
2023-01-14 03:13:35 +00:00
```bash
2022-05-17 03:19:39 +00:00
# Switch to `n`-managed NodeJS
sudo npm i -g n
sudo n 16
2022-05-16 03:26:04 +00:00
```
2023-01-14 03:13:35 +00:00
C) Build and install a version of Git with proper SSL support:
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
```bash
# Git does not support OpenSSL out of the box, must do this
curl -LO https://github.com/paul-nelson-baker/git-openssl-shellscript/raw/main/compile-git-with-openssl.sh
chmod +x compile-git-with-openssl.sh
./compile-git-with-openssl.sh
2022-05-16 03:26:04 +00:00
```
2023-01-14 03:13:35 +00:00
D) Set `git` config `core.autocrlf` setting to `false`. The following commands
should be run twice, once within PowerShell and once within WSL bash:
```powershell
git config --global --add core.autocrlf false
git config --global --unset core.autocrlf true
```
2022-05-17 03:19:39 +00:00
</TabItem>
2023-01-14 03:13:35 +00:00
<TabItem value="osx" label="MacOS">
A) Run `git`. If Xcode or the command-line tools are not installed, you will be
asked to install. Click "Install" and run through the steps.
B) Open a terminal window and install Homebrew:
```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
C) Close the window, open a new terminal window, and disable analytics:
```bash
brew analytics off
```
To confirm analytics are disabled, run
```bash
brew analytics state
```
It should print `Analytics are disabled.`
2022-05-17 03:19:39 +00:00
2023-01-14 03:13:35 +00:00
D) Install Mercurial and Subversion:
```bash
brew install mercurial subversion
```
2022-05-17 03:19:39 +00:00
2023-01-14 03:13:35 +00:00
E) Install NodeJS
2022-05-17 03:19:39 +00:00
2022-11-12 09:16:51 +00:00
:::note
[The official NodeJS site](https://nodejs.org/en/download/) provides installers
for "LTS" and "Current" releases. The "LTS" version should be installed.
:::
2023-01-14 03:13:35 +00:00
</TabItem>
2022-11-12 09:16:51 +00:00
<TabItem value="l" label="Linux">
2023-01-14 03:13:35 +00:00
A) Install `mercurial`, `git`, and `subversion` using the system package
manager. On Debian and Ubuntu systems, `apt-get` installs packages:
2022-05-17 03:19:39 +00:00
```bash
2023-01-14 03:13:35 +00:00
sudo apt-get install git mercurial subversion
2022-05-16 03:26:04 +00:00
```
2022-11-12 09:16:51 +00:00
Other Linux distributions may use other package managers.
2023-01-14 03:13:35 +00:00
<details open><summary><b>Steam Deck</b> (click to show)</summary>
2022-11-12 09:16:51 +00:00
Desktop Mode on the Steam Deck uses `pacman`. It also requires a few steps.
0) Switch to Desktop mode and open `Konsole`
1) Set a password for the user by running `passwd` and following instructions.
2) Disable read-only mode:
```bash
sudo steamos-readonly disable
```
(When prompted, enter the password assigned in step 1)
3) Configure keyring:
```bash
2023-03-26 00:55:35 +00:00
sudo sh -c 'echo "keyserver hkps://keyserver.ubuntu.com" >> /etc/pacman.d/gnupg/gpg.conf'
2022-11-12 09:16:51 +00:00
sudo pacman-key --init
sudo pacman-key --populate
sudo pacman-key --refresh-keys
```
4) Install dependencies:
```bash
2023-03-26 00:55:35 +00:00
sudo pacman -S base-devel mercurial subversion
2022-11-12 09:16:51 +00:00
```
</details>
2023-01-14 03:13:35 +00:00
After installing mercurial and subversion, install NodeJS.
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
:::note
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
[The official NodeJS site](https://nodejs.org/en/download/) provides installers
for "LTS" and "Current" releases. The "LTS" version should be installed.
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
:::
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
</TabItem>
</Tabs>
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
### Build from source tree
2022-11-12 09:16:51 +00:00
2023-01-14 03:13:35 +00:00
0) Clone the project:
2022-05-16 03:26:04 +00:00
```bash
2023-01-14 03:13:35 +00:00
git clone https://git.sheetjs.com/sheetjs/sheetjs
cd sheetjs
2022-05-17 03:19:39 +00:00
```
1) Install NodeJS modules for building the scripts
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
```bash
2022-05-16 03:26:04 +00:00
# Install dev dependencies
2022-05-17 03:19:39 +00:00
npm install
# Install global dependencies
2022-07-25 23:18:00 +00:00
sudo npm i -g mocha@2.5.3 voc @sheetjs/uglify-js
2022-05-17 03:19:39 +00:00
```
2022-11-12 09:16:51 +00:00
:::note Older Versions of Dependencies
Some of the dependencies are wildly out of date. While SheetJS aims to run in
older versions of NodeJS and browsers, some libraries have chosen to break
backwards compatibility. The specific versions are used because they are known
to work and known to produce consistent results.
:::
2022-05-17 03:19:39 +00:00
2) Initialize the test files:
```bash
make init
```
This step may take a while as it will be downloading a number of test files.
3) Run a short test, then run a build
```bash
# Short test
make test_misc
# Full Build
2023-03-26 00:55:35 +00:00
cd modules; make clean; make; cd ..
2022-05-17 03:19:39 +00:00
make dist
2022-05-20 03:25:45 +00:00
```
2023-01-14 03:13:35 +00:00
### Reproduce official builds
4) Run `git log` and search for the commit that matches a particular release
version. For example, version `0.19.1` can be found with:
2022-05-20 03:25:45 +00:00
```bash
2023-01-14 03:13:35 +00:00
git log | grep -B4 "version bump 0.19.1"
2022-07-23 09:06:31 +00:00
```
2023-01-14 03:13:35 +00:00
The output should look like:
2022-07-23 09:06:31 +00:00
```bash
2023-01-14 03:13:35 +00:00
$ git log | grep -B4 "version bump 0.19.1"
# highlight-next-line
commit 51a861900092bfc01b2fa9960d4c932dcbeb6925 <-- this is the commit hash
Author: SheetJS <dev@sheetjs.com>
Date: Thu Nov 17 04:35:34 2022 -0500
version bump 0.19.1
2022-05-16 03:26:04 +00:00
```
2023-01-14 03:13:35 +00:00
5) Switch to that commit:
```bash
git checkout 51a861900092bfc01b2fa9960d4c932dcbeb6925
```
6) Run the full build sequence
```bash
make clean
2023-03-26 00:55:35 +00:00
cd modules; make clean; make; cd ..
2023-01-14 03:13:35 +00:00
make
make dist
```
7) To verify that the files are intact, use `md5sum` (`md5` on MacOS).
The local checksum for the browser script can be computed with:
```bash
$ md5sum dist/xlsx.full.min.js
00b46a9f2d30ddc69780ab8049d6809e dist/xlsx.full.min.js
```
The checksum for the CDN version can be computed with:
```bash
curl -L https://cdn.sheetjs.com/xlsx-0.19.1/package/dist/xlsx.full.min.js | md5sum -
00b46a9f2d30ddc69780ab8049d6809e -
```
The two hashes should match.
8) To return to the HEAD commit, run
```bash
2023-03-26 00:55:35 +00:00
git checkout master
2023-01-14 03:13:35 +00:00
```
### Test in web browsers
9) Start local server:
```bash
make ctestserv
```
The terminal will display a port number. For example:
```
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/)
```
2022-05-16 03:26:04 +00:00
2023-01-14 03:13:35 +00:00
10) Open a browser window and access `http://localhost:8000`, replacing `8000`
with the port number from the terminal window.
2022-05-16 03:26:04 +00:00
2022-05-17 03:19:39 +00:00
## Development
The `xlsx.js` and `xlsx.mjs` files are constructed from the files in the `bits`
2022-08-24 00:51:18 +00:00
subfolder. The build script (run `make`) will concatenate the individual bits
to produce the scripts.
2022-05-17 03:19:39 +00:00
To produce the dist files, run `make dist`. The dist files are updated in each
version release and *should not be committed between versions*.
## Tests
The `test_misc` target runs the targeted feature tests. It should take 5-10
seconds to perform feature tests without testing against the full test battery.
New features should be accompanied with tests for the relevant file formats.
2022-05-16 03:26:04 +00:00
For tests involving the read side, an appropriate feature test would involve
reading an existing file and checking the resulting workbook object. If a
parameter is involved, files should be read with different values to verify that
the feature is working as expected.
For tests involving a new write feature which can already be parsed, appropriate
feature tests would involve writing a workbook with the feature and then opening
and verifying that the feature is preserved.
For tests involving a new write feature without an existing read ability, please
add a feature test to the kitchen sink `tests/write.js`.