useeffect-async

This commit is contained in:
SheetJS 2024-01-31 03:14:07 -05:00
parent 952244b917
commit 17d9d3d7cf
6 changed files with 204 additions and 49 deletions

@ -1,7 +1,7 @@
---
title: Spreadsheet Data in Pandas
sidebar_label: Python + Pandas
description: Process structured data in Python with Pandas. Seamlessly integrate spreadsheets into your workflow with SheetJS. Analyze complex Excel spreadsheets with confidence.
title: Spreadsheet Data in Python
sidebar_label: Python DataFrames
description: Process structured data in Python DataFrames. Seamlessly integrate spreadsheets into your workflow with SheetJS. Analyze complex Excel spreadsheets with confidence.
pagination_prev: demos/index
pagination_next: demos/frontend/index
---
@ -11,7 +11,7 @@ import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';
Pandas[^1] is a Python software library for data analysis.
[Pandas](https://pandas.pydata.org/) is a Python library for data analysis.
[SheetJS](https://sheetjs.com) is a JavaScript library for reading and writing
data from spreadsheets.
@ -28,7 +28,7 @@ simplifies importing and exporting spreadsheets.
Pandas includes limited support for reading spreadsheets (`pandas.from_excel`)
and writing XLSX spreadsheets (`pandas.DataFrame.to_excel`).
**SheetJS supports common spreadsheet formats that Pandas cannot process.**
**SheetJS supports many common spreadsheet formats that Pandas cannot process.**
SheetJS operations also offer more flexibility in processing complex worksheets.
@ -55,8 +55,8 @@ included in the ["Complete Example"](#complete-example) section.
JS code cannot be directly evaluated in Python implementations.
To run JS code from Python, JavaScript engines[^2] can be embedded in Python
modules or dynamically loaded using the `ctypes` foreign function library[^3].
To run JS code from Python, JavaScript engines[^1] can be embedded in Python
modules or dynamically loaded using the `ctypes` foreign function library[^2].
This demo uses `ctypes` with the [Duktape engine](/docs/demos/engines/duktape).
### Wrapper
@ -138,12 +138,12 @@ flowchart LR
2) SheetJS libraries parse the string and generate a clean CSV.
- The `read` method[^4] parses file bytes into a SheetJS workbook object[^5]
- After selecting a worksheet, `sheet_to_csv`[^6] generates a CSV string
- The `read` method[^3] parses file bytes into a SheetJS workbook object[^4]
- After selecting a worksheet, `sheet_to_csv`[^5] generates a CSV string
3) Python operations convert the CSV string to a stream object.[^7]
3) Python operations convert the CSV string to a stream object.[^6]
4) The Pandas `read_csv` method[^8] ingests the stream and generate a DataFrame.
4) The Pandas `read_csv` method[^7] ingests the stream and generate a DataFrame.
### Writing Files
@ -179,15 +179,15 @@ flowchart LR
u8a --> |`open`/`write`\nPython ops| file
```
1) The Pandas DataFrame `to_json` method[^9] generates a JSON string.
1) The Pandas DataFrame `to_json` method[^8] generates a JSON string.
2) JS engine operations translate the JSON string to an array of objects.
3) SheetJS libraries process the data array and generate file bytes.
- The `json_to_sheet` method[^10] creates a SheetJS sheet object from the data.
- The `book_new` method[^11] creates a SheetJS workbook that includes the sheet.
- The `write` method[^12] generates the spreadsheet file bytes.
- The `json_to_sheet` method[^9] creates a SheetJS sheet object from the data.
- The `book_new` method[^10] creates a SheetJS workbook that includes the sheet.
- The `write` method[^11] generates the spreadsheet file bytes.
4) Pure Python operations write the bytes to file.
@ -317,15 +317,166 @@ dtypes: int64(1), object(1)
It will also export the DataFrame to `SheetJSPandas.xlsb`. The file can be
inspected with a spreadsheet editor that supports XLSB files.
[^1]: The official documentation site is <https://pandas.pydata.org/> and the official distribution point is <https://pypi.org/project/pandas/>
[^2]: See ["Other Languages"](/docs/demos/engines/) for more examples.
[^3]: See [`ctypes`](https://docs.python.org/3/library/ctypes.html) in the Python documentation.
[^4]: See [`read` in "Reading Files"](/docs/api/parse-options)
[^5]: See ["Workbook Object"](/docs/csf/book)
[^6]: See [`sheet_to_csv` in "Utilities"](/docs/api/utilities/csv#delimiter-separated-output)
[^7]: See [the examples in "IO tools"](https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html) in the Pandas documentation.
[^8]: See [`pandas.read_csv`](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html) in the Pandas documentation.
[^9]: See [`pandas.DataFrame.to_json`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html) in the Pandas documentation.
[^10]: See [`json_to_sheet` in "Utilities"](/docs/api/utilities/array#array-of-objects-input)
[^11]: See [`book_new` in "Utilities"](/docs/api/utilities/wb)
[^12]: See [`write` in "Writing Files"](/docs/api/write-options)
## Other Libraries
Other Python DataFrame libraries mirror the Pandas DataFrame API.
### Polars
[Polars](https://pola.rs/) is a similar DataFrame library that offers many
features from Pandas DataFrames.
:::info pass
Polars includes limited support for reading and writing spreadsheets by wrapping
third-party libraries. In practice, Polars communicates with the third-party
libraries using intermediate CSV files.[^12]
**SheetJS supports many common spreadsheet formats that Polars cannot process.**
SheetJS operations also offer more flexibility in processing complex worksheets.
:::
The Pandas example requires a few slight changes to work with Polars:
- Polars DataFrames expose `write_json` instead of `to_json`:
```diff
- json = df.to_json(orient="records")
+ json = df.write_json(row_oriented=True)
```
- Polars DataFrames do not expose `info`
#### Polars Demo
:::note Tested Environments
This demo was tested in the following deployments:
| Architecture | JS Engine | Polars | Python | Date |
|:-------------|:----------------|:-------|:-------|:-----------|
| `darwin-x64` | Duktape `2.7.0` | 0.20.6 | 3.11.7 | 2024-01-30 |
| `linux-x64` | Duktape `2.7.0` | 0.20.6 | 3.11.3 | 2024-01-30 |
:::
0) Follow the [Pandas "Complete Example"](#complete-example) through the end.
1) Edit `sheetjs.py`.
Near the top of the script, change the import from `pandas` to `polars`:
```diff title="sheetjs.py (apply changes)"
-from pandas import read_csv
+from polars import read_csv
```
:::note pass
The red lines starting with `-` should be removed from the file and the green
lines starting with `+` should be added to the file. Black lines show the source
context and should not be changed.
:::
Within the `export_df_to_wb` function, change the `df.to_json` line:
```diff title="sheetjs.py (apply changes)"
def export_df_to_wb(ctx, df, path, sheet_name="Sheet1", book_type=None):
- json = df.to_json(orient="records")
+ json = df.write_json(row_oriented=True)
```
2) Edit `SheetJSPandas.py`.
In the script, change `df.info()` to `df`:
```diff title="SheetJSPandas.py (apply changes)"
def export_df_to_wb(ctx, df, path, sheet_name="Sheet1", book_type=None):
- print(df.info())
+ print(df)
```
Change the export filename from `SheetJSPandas.xlsb` to `SheetJSPolars.xlsb`:
```diff
# Export DataFrame to XLSB
- sheetjs.write_df(df, "SheetJSPandas.xlsb", sheet_name="DataFrame")
+ sheetjs.write_df(df, "SheetJSPolars.xlsb", sheet_name="DataFrame")
```
3) Install Polars:
```bash
sudo python3 -m pip install polars
```
:::caution pass
On Arch Linux-based platforms including the Steam Deck, the install may fail:
```
error: externally-managed-environment
```
It is recommended to use a virtual environment for Polars:
```bash
mkdir sheetjs-polars
cd sheetjs-polars
python -m venv .
./bin/pip install polars
```
:::
4) Run the script:
```bash
python3 SheetJSPandas.py pres.numbers
```
:::note pass
If the virtual environment was configured in the previous step, run:
```bash
./bin/python3 SheetJSPandas.py pres.numbers
```
:::
If successful, the script will display DataFrame data:
```
shape: (5, 2)
┌──────────────┬───────┐
│ Name ┆ Index │
│ --- ┆ --- │
│ str ┆ i64 │
╞══════════════╪═══════╡
│ Bill Clinton ┆ 42 │
│ GeorgeW Bush ┆ 43 │
│ Barack Obama ┆ 44 │
│ Donald Trump ┆ 45 │
│ Joseph Biden ┆ 46 │
└──────────────┴───────┘
```
It will also export the DataFrame to `SheetJSPolars.xlsb`. The file can be
inspected with a spreadsheet editor that supports XLSB files.
[^1]: See ["Other Languages"](/docs/demos/engines/) for more examples.
[^2]: See [`ctypes`](https://docs.python.org/3/library/ctypes.html) in the Python documentation.
[^3]: See [`read` in "Reading Files"](/docs/api/parse-options)
[^4]: See ["Workbook Object"](/docs/csf/book)
[^5]: See [`sheet_to_csv` in "Utilities"](/docs/api/utilities/csv#delimiter-separated-output)
[^6]: See [the examples in "IO tools"](https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html) in the Pandas documentation.
[^7]: See [`pandas.read_csv`](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html) in the Pandas documentation.
[^8]: See [`pandas.DataFrame.to_json`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html) in the Pandas documentation.
[^9]: See [`json_to_sheet` in "Utilities"](/docs/api/utilities/array#array-of-objects-input)
[^10]: See [`book_new` in "Utilities"](/docs/api/utilities/wb)
[^11]: See [`write` in "Writing Files"](/docs/api/write-options)
[^12]: As explained [in the Polars documentation](https://docs.pola.rs/py-polars/html/reference/api/polars.read_excel.html), "... the target Excel sheet is first converted to CSV ... and then parsed with Polars `read_csv()` function."

@ -82,7 +82,7 @@ Each browser demo was tested in the following environments:
| Browser | Date |
|:------------|:-----------|
| Chrome 120 | 2024-01-15 |
| Chrome 120 | 2024-01-30 |
| Safari 17.2 | 2024-01-15 |
:::
@ -118,7 +118,7 @@ function SheetJSXHRDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
React.useEffect(() => { (async() => {
/* Fetch file */
const req = new XMLHttpRequest();
req.open("GET", "https://sheetjs.com/pres.numbers", true);
@ -132,7 +132,7 @@ function SheetJSXHRDL() {
setHTML(XLSX.utils.sheet_to_html(ws));
};
req.send();
}, []);
})(); }, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
@ -170,7 +170,7 @@ function SheetJSFetchDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
React.useEffect(() => { (async() => {
/* Fetch file */
const res = await fetch("https://sheetjs.com/pres.numbers");
const ab = await res.arrayBuffer();
@ -181,7 +181,7 @@ function SheetJSFetchDL() {
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
}, []);
})(); }, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
@ -276,7 +276,7 @@ function SheetJSAxiosDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
React.useEffect(() => { (async() => {
if(typeof axios != "function") return setHTML("ReferenceError: axios is not defined");
/* Fetch file */
const res = await axios("https://sheetjs.com/pres.numbers", {responseType: "arraybuffer"});
@ -287,7 +287,7 @@ function SheetJSAxiosDL() {
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
}, []);
})(); }, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}
@ -337,7 +337,7 @@ function SheetJSSuperAgentDL() {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
React.useEffect(() => { (async() => {
if(typeof superagent == "undefined" || typeof superagent.get != "function")
return setHTML("ReferenceError: superagent is not defined");
/* Fetch file */
@ -352,7 +352,7 @@ function SheetJSSuperAgentDL() {
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
});
}, []);
})(); }, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}

@ -231,7 +231,7 @@ please refresh the page. This is a known bug in the documentation generator.
function SheetJSEnregistrez() {
const [msg, setMsg] = React.useState("Press the button to write XLS file");
const btn = useRef(), tbl = useRef();
React.useEffect(async() => {
React.useEffect(() => { (async() => {
if(typeof Dropbox == "undefined") return setMsg("Dropbox is not defined");
/* fetch data and write table (sample data) */
const f = await(await fetch("https://sheetjs.com/pres.xlsx")).arrayBuffer();
@ -255,7 +255,7 @@ function SheetJSEnregistrez() {
});
/* add button to page */
btn.current.appendChild(button);
}, []);
})(); }, []);
return ( <><b>{msg}</b><br/><div ref={btn}/><div ref={tbl}/></> );
}
```

@ -155,9 +155,9 @@ function ConcatFormula(props) {
};
/* Fetch sample file */
React.useEffect(async() => {
React.useEffect(() => {(async() => {
process_ab(await (await fetch("/files/concat.xlsx")).arrayBuffer());
}, []);
})(); }, []);
const process_file = async(e) => {
process_ab(await e.target.files[0].arrayBuffer());
};
@ -415,14 +415,14 @@ function Translator(props) {
const [names, setNames] = React.useState([]);
const [name, setName] = React.useState("Enter a function name");
/* Fetch and display formula */
React.useEffect(async() => {
React.useEffect(() => { (async() => {
/* Fetch data */
const json = await (await fetch("https://oss.sheetjs.com/notes/fmla/table.json")).json();
setLocales(Object.keys(json));
setData(json);
setNames(json.en);
setName(json.es[0])
}, []);
})(); }, []);
const update_name = React.useCallback(() => {
const nameelt = document.getElementById("fmla");

@ -110,14 +110,14 @@ function Visibility(props) {
const [sheets, setSheets] = React.useState([]);
const vis = [ "Visible", "Hidden", "Very Hidden" ];
React.useEffect(async() => {
React.useEffect(() => { (async() => {
const f = await fetch("/files/sheet_visibility.xlsx");
const ab = await f.arrayBuffer();
const wb = XLSX.read(ab);
setWB(wb);
/* State will be set to the `Sheets` property array */
setSheets(wb.Workbook.Sheets);
}, []);
})(); }, []);
return (<table>
<thead><tr><th>Name</th><th>Value</th><th>Hidden</th></tr></thead>

@ -185,22 +185,26 @@ using the `puppeteer` and `playwright` browser automation frameworks.
<details open><summary><b>Live Example</b> (click to hide)</summary>
This example uses a ReactJS `ref` to reference the HTML TABLE element. ReactJS
details are covered in the [ReactJS demo](/docs/demos/frontend/react#html)
```jsx live
/* The live editor requires this function wrapper */
function Table2XLSX(props) {
/* reference to the table element */
const tbl = React.useRef();
/* Callback invoked when the button is clicked */
const xport = React.useCallback(() => {
/* Create worksheet from HTML DOM TABLE */
const table = document.getElementById("Table2XLSX");
const wb = XLSX.utils.table_to_book(table);
const wb = XLSX.utils.table_to_book(tbl.current);
/* Export to file (start a download) */
XLSX.writeFile(wb, "SheetJSTable.xlsx");
});
return ( <>
<table id="Table2XLSX"><tbody>
<table ref={tbl}><tbody>
<tr><td colSpan="3">SheetJS Table Export</td></tr>
<tr><td>Author</td><td>ID</td><td>你好!</td></tr>
<tr><td>SheetJS</td><td>7262</td><td>வணக்கம்!</td></tr>
@ -267,7 +271,7 @@ function Numbers2HTML(props) {
const [__html, setHTML] = React.useState("");
/* Fetch and update HTML */
React.useEffect(async() => {
React.useEffect(() => { (async() => {
/* Fetch file */
const f = await fetch("https://sheetjs.com/pres.numbers");
const ab = await f.arrayBuffer();
@ -278,7 +282,7 @@ function Numbers2HTML(props) {
/* Generate HTML */
setHTML(XLSX.utils.sheet_to_html(ws));
}, []);
})(); }, []);
return ( <div dangerouslySetInnerHTML={{ __html }}/> );
}