iwa
This commit is contained in:
parent
02e5725832
commit
fbb4b03953
@ -8,6 +8,7 @@ Various spreadsheet file format notes.
|
||||
- [Number Formats](/ssf/README.md)
|
||||
- [SpreadsheetML2003](/xlml/README.md)
|
||||
- [LibreOffice Bugs](/lobugs/README.md)
|
||||
- [Numbers 3+ (iWork 2013+)](/iwa/README.md)
|
||||
|
||||
Project sponsored by [SheetJS](https://sheetjs.com)
|
||||
|
||||
|
||||
195
iwa/README.md
Normal file
195
iwa/README.md
Normal file
@ -0,0 +1,195 @@
|
||||
# iWork 2013+
|
||||
|
||||
This covers the standalone ZIP-based format. iCloud uses a different format to
|
||||
support differential sync between devices.
|
||||
|
||||
The ZIP container holds a number of Mac binary "property list" files (`.plist`)
|
||||
which can be safely ignored or blanked. It also can hold preview images that
|
||||
can be safely ignored.
|
||||
|
||||
## File Structure
|
||||
|
||||
The `.numbers` file is a ZIP file containing a number of `.iwa` entries. The
|
||||
primary entrypoint is `/Index/Document.iwa`.
|
||||
|
||||
`TSPersistence.framework` handles the byte-level operations for the files.
|
||||
|
||||
`.iwa` files are sequential blocks of compressed data. Each "block" starts with
|
||||
a 4-byte header consisting of a `0` byte followed by the compressed length
|
||||
(stored as a 3-byte little-endian integer)
|
||||
|
||||
Each block follows the Snappy compressed format as described in
|
||||
<https://github.com/google/snappy/blob/main/format_description.txt> . iWork
|
||||
apps do not expect a particular compression level, and it is possible to create
|
||||
the equivalent of a "STORED" block.
|
||||
|
||||
# Data Storage
|
||||
|
||||
The decompressed data is a series of chunks.
|
||||
|
||||
Each chunk starts with a `length` stored in a Base 128 `varint`, followed by a
|
||||
protobuf packet of type `.TSP.ArchiveInfo`.
|
||||
|
||||
The `.TSP.ArchiveInfo` message contains a number of `.TSP.MessageInfo` messages
|
||||
(tag 2). Each `MessageInfo` has a `length` field (tag 3, type `uint32`) for the
|
||||
actual message body. The data for the message bodies are stored immediately
|
||||
after the `ArchiveInfo`, in the same order as the `MessageInfo` parts.
|
||||
|
||||
The message type from the `MessageInfo` (tag 1, type `uint32`) corresponds to a
|
||||
dynamic registry spread across the embedded frameworks. The actual message data
|
||||
is a protobuf packet.
|
||||
|
||||
## Dynamic Registry and Message Types
|
||||
|
||||
The `.TSP.Reference` type acts as a pointer, referencing another message. The
|
||||
references do not include message type info, so readers and writers must be
|
||||
aware of the message types and their interpretations.
|
||||
|
||||
Each framework is responsible for registering message types with the master
|
||||
registry by sending a message to the `TSPRegistry`. The actual types can be
|
||||
discovered from the frameworks. Some common message types are listed below:
|
||||
|
||||
| type | message |
|
||||
|-----:|:-------------------------|
|
||||
| 1 | `.TN.DocumentArchive` |
|
||||
| 2 | `.TN.SheetArchive` |
|
||||
| 6000 | `.TST.TableInfoArchive` |
|
||||
| 6001 | `.TST.TableModelArchive` |
|
||||
| 6002 | `.TST.Tile` |
|
||||
|
||||
All referenced types must be registered, but ancillary types do not need to be
|
||||
registered. For example:
|
||||
|
||||
```proto
|
||||
message .TST.TableInfoArchive {
|
||||
// ...
|
||||
required .TSP.Reference tableModel = 2;
|
||||
// ...
|
||||
}
|
||||
|
||||
message .TST.TableModelArchive {
|
||||
// ...
|
||||
required .TST.DataStore base_data_store = 4;
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
The reference in field 2 from `.TST.TableInfoArchive` is expected to be of type
|
||||
`.TST.TableModelArchive` so the latter must be registered.
|
||||
|
||||
`.TST.DataStore` is the type of field 4 from `.TST.TableModelArchive`. Since it
|
||||
is not referenced indirectly, the message type does not have to be registered.
|
||||
|
||||
## iWork Proto Definitions
|
||||
|
||||
The iWork apps (Keynote, Numbers, Pages) include embedded Protocol Buffers
|
||||
definitions as part of the file format processors.
|
||||
|
||||
The [`otorp` package on `npm`](https://npm.im/otorp) ships with a command-line
|
||||
tool for extracting definitions from a Mach-O binary.
|
||||
|
||||
### App-Specific Definitions
|
||||
|
||||
The listed definitions only appear in one app:
|
||||
|
||||
**Keynote**
|
||||
- `KNArchives.proto`
|
||||
- `KNArchives.sos.proto`
|
||||
- `KNCommandArchives.proto`
|
||||
- `KNCommandArchives.sos.proto`
|
||||
|
||||
**Numbers**
|
||||
- `TNArchives.proto`
|
||||
- `TNArchives.sos.proto`
|
||||
- `TNCommandArchives.proto`
|
||||
- `TNCommandArchives.sos.proto`
|
||||
|
||||
**Pages**
|
||||
- `TPArchives.proto`
|
||||
- `TPCommandArchives.proto`
|
||||
- `TPCommandArchives.sos.proto`
|
||||
|
||||
The other files are common across the apps.
|
||||
|
||||
## Determining File Type
|
||||
|
||||
All three file types use the same message tag (1) for the root `DocumentArchive`
|
||||
message. However, the required fields vary between formats.
|
||||
|
||||
In the 11.2 apps, the required fields are:
|
||||
|
||||
```proto
|
||||
// Keynote optional fields 4
|
||||
message .KN.DocumentArchive {
|
||||
required .TSA.DocumentArchive super = 3;
|
||||
required .TSP.Reference show = 2;
|
||||
}
|
||||
|
||||
// Numbers optional fields 1, 3, 7, 9, 10, 11, 12
|
||||
message .TN.DocumentArchive {
|
||||
required .TSA.DocumentArchive super = 8;
|
||||
required .TSP.Reference stylesheet = 4;
|
||||
required .TSP.Reference sidebar_order = 5;
|
||||
required .TSP.Reference theme = 6;
|
||||
}
|
||||
|
||||
// Pages optional fields 2 - 7, 11 - 14, 16, 17, 20, 21, 30 - 49
|
||||
message .TP.DocumentArchive {
|
||||
required .TSA.DocumentArchive super = 15;
|
||||
}
|
||||
```
|
||||
|
||||
Pages is the only format to use and require field 15. Keynote requires field 2,
|
||||
a field that does not appear in Numbers.
|
||||
|
||||
|
||||
## Misc
|
||||
|
||||
### MD5 Checksums
|
||||
|
||||
The checksums are based on definitions from version 11.2 (7032.0.145)
|
||||
|
||||
```proto
|
||||
MD5 (KNArchives.proto) = 4d15ddb1dacdf0e2a321d09234130abd
|
||||
MD5 (KNArchives.sos.proto) = 5b8e5101e946eaddca341b78b5f6e660
|
||||
MD5 (KNCommandArchives.proto) = 3aa3d1796a8cb1f1cdaccca2d7a67704
|
||||
MD5 (KNCommandArchives.sos.proto) = af09afc07db32005ff1eaf183fe99c02
|
||||
MD5 (TNArchives.proto) = c6af8ddec038ccfb8d683f2c92bb8b2c
|
||||
MD5 (TNArchives.sos.proto) = 7a8ce65fa23d554fba3549f6ec5d1aa5
|
||||
MD5 (TNCommandArchives.proto) = 4b7e7ebe4583d0a0166118292d338272
|
||||
MD5 (TNCommandArchives.sos.proto) = 279eafd525689d2e8805e951f9a0b9a6
|
||||
MD5 (TPArchives.proto) = 5ae0d81aded078509f8b1dfc48b1b118
|
||||
MD5 (TPCommandArchives.proto) = 1cf18ff056e7635f1cfcb68ec943892b
|
||||
MD5 (TPCommandArchives.sos.proto) = 7b627bc1e053a03a762952c5d647a97a
|
||||
MD5 (TSAArchives.proto) = 6a246bb28e425ff796c285c36132b2ef
|
||||
MD5 (TSAArchives.sos.proto) = 8b55a583851f1d11a2af842f20223bc7
|
||||
MD5 (TSACommandArchives.sos.proto) = d0e4741e2fea6dde83b17a7e73d12af7
|
||||
MD5 (TSCEArchives.proto) = abe4e1a7256019562a3790eb58c6b9f9
|
||||
MD5 (TSCH3DArchives.proto) = 2ac8ee10b2e28c5d201524a388a2eb9e
|
||||
MD5 (TSCHArchives.Common.proto) = 6cc9c2954517f29d8247e4d59caa980f
|
||||
MD5 (TSCHArchives.GEN.proto) = 25bd8a10b1646970fb96f411f14821c9
|
||||
MD5 (TSCHArchives.proto) = e4f10e18142ee8b0e2bb6f94b03e05d6
|
||||
MD5 (TSCHArchives.sos.proto) = 4452d0264a26b946524ebbbb2dec56e0
|
||||
MD5 (TSCHCommandArchives.proto) = 75cc30b1ba010f1172d0d08eff8bfa9f
|
||||
MD5 (TSCHPreUFFArchives.proto) = 5a9c935194bd772a30774071ac7a7f64
|
||||
MD5 (TSDArchives.proto) = 5cc5d066f5b394508ec13f64dedba7b7
|
||||
MD5 (TSDArchives.sos.proto) = ccbc06fd5b8db95e78b4b404a0d2177f
|
||||
MD5 (TSDCommandArchives.proto) = 878c419d11b043333e1ed5148a868626
|
||||
MD5 (TSKArchives.proto) = bd76c5489f2bfb5a94750f1a83969549
|
||||
MD5 (TSKArchives.sos.proto) = 5e68b45687d33b9e2cdc0f64d76988f9
|
||||
MD5 (TSPArchiveMessages.proto) = 5bd640aed4df2758a393143096cbaf70
|
||||
MD5 (TSPDatabaseMessages.proto) = ab86cd136e1702555b5080f59609f2f1
|
||||
MD5 (TSPMessages.proto) = 1a33eb51dfb1f8ccbabdc6e236690ce0
|
||||
MD5 (TSSArchives.proto) = 301ea13a293ada201db8edf9b0f83d1d
|
||||
MD5 (TSSArchives.sos.proto) = 951c42b9fd732552ffc4944fe414890a
|
||||
MD5 (TSTArchives.proto) = 59a353cf0dd34b31ee932149c517e80d
|
||||
MD5 (TSTArchives.sos.proto) = d085aa1e4449f85bcc37689c7d6e4c5f
|
||||
MD5 (TSTCommandArchives.proto) = 4c1bbca393199455635dd80a89d9f61b
|
||||
MD5 (TSTStylePropertyArchiving.proto) = 463bb1c64fe02a484b4e9e655d6af391
|
||||
MD5 (TSWPArchives.proto) = 7d171fbf72e184957773c6ad84ff4f09
|
||||
MD5 (TSWPArchives.sos.proto) = 06be0c76ea913408da04153b292f08e0
|
||||
MD5 (TSWPCommandArchives.proto) = dabb9f2e85ffdba52aaca61064303c71
|
||||
```
|
||||
|
||||
|
||||
[](https://github.com/SheetJS/notes)
|
||||
Loading…
Reference in New Issue
Block a user