From 37f7dd1f834bd8afd1eaf70fd338872b621cbee3 Mon Sep 17 00:00:00 2001 From: NikitolProject Date: Thu, 26 Feb 2026 21:25:55 +0300 Subject: [PATCH] feat(07-01): replace worked example with v1.1 directory archive - New worked example: 3 entries (2 dirs + 1 file) totaling 427 bytes - Demonstrates nested dir (project/src), file (project/src/main.rs), empty dir (project/empty) - Entry hex tables show entry_type and permissions fields - Directory entries have all-zero crypto fields (iv, hmac, sha256, sizes) - File entry shows full crypto pipeline with real SHA-256 hash - Archive layout table with verified offsets (header=40, TOC=355, data=32) - Complete annotated hex dump covers all 427 bytes - Shell decode walkthrough handles directory entries (mkdir -p + chmod) Co-Authored-By: Claude Opus 4.6 --- docs/FORMAT.md | 449 +++++++++++++++++++++++++++---------------------- 1 file changed, 250 insertions(+), 199 deletions(-) diff --git a/docs/FORMAT.md b/docs/FORMAT.md index d1510b5..ed1c73e 100644 --- a/docs/FORMAT.md +++ b/docs/FORMAT.md @@ -512,50 +512,81 @@ The following steps MUST be followed in order by all decoders: ## 12. Worked Example -This section constructs a complete 2-file archive byte by byte. All offsets, field sizes, and hex values are internally consistent and can be verified by summing field sizes. This example serves as a **golden reference** for implementation testing. +This section constructs a complete 3-entry directory archive byte by byte, demonstrating the v1.1 format with entry types, permissions, and relative paths. All offsets, field sizes, and hex values are internally consistent and can be verified by summing field sizes. This example serves as a **golden reference** for implementation testing. -### 12.1 Input Files +### 12.1 Input Structure -| File | Name | Content | Size | -|------|------|---------|------| -| 1 | `hello.txt` | ASCII string `Hello` (bytes: `48 65 6C 6C 6F`) | 5 bytes | -| 2 | `data.bin` | 32 bytes of `0x01` repeated | 32 bytes | +``` +project/ + project/src/ (directory, mode 0755) + project/src/main.rs (file, mode 0644, content: "fn main() {}\n" = 14 bytes) + project/empty/ (empty directory, mode 0755) +``` + +This demonstrates: +- A nested directory (`project/src/`) +- A file inside a nested directory (`project/src/main.rs`) +- An empty directory (`project/empty/`) +- Three TOC entries total: 2 directories + 1 file + +| # | Entry Name | Type | Permissions | Content | Size | +|---|------------|------|-------------|---------|------| +| 1 | `project/src` | directory | `0o755` | (none) | 0 bytes | +| 2 | `project/src/main.rs` | file | `0o644` | `fn main() {}\n` | 14 bytes | +| 3 | `project/empty` | directory | `0o755` | (none) | 0 bytes | + +Entries are ordered parent-before-child: `project/src` appears before `project/src/main.rs`. ### 12.2 Parameters - **Key:** 32 bytes: `00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F` - **Flags:** `0x01` (compression enabled, no obfuscation) -- **Version:** `1` +- **Version:** `2` -### 12.3 Per-File Pipeline Walkthrough +### 12.3 Per-Entry Pipeline Walkthrough -#### File 1: `hello.txt` +#### Entry 1: `project/src` (directory) + +Directory entries have no data. All crypto fields are zero-filled: + +- `entry_type`: `0x01` +- `permissions`: `0o755` = `0x01ED` (LE: `ED 01`) +- `original_size`: 0 +- `compressed_size`: 0 +- `encrypted_size`: 0 +- `data_offset`: 0 +- `iv`: zero-filled (16 bytes of `0x00`) +- `hmac`: zero-filled (32 bytes of `0x00`) +- `sha256`: zero-filled (32 bytes of `0x00`) +- `compression_flag`: 0 + +#### Entry 2: `project/src/main.rs` (file) **Step 1: SHA-256 checksum of original content** ``` -SHA-256("Hello") = 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969 +SHA-256("fn main() {}\n") = 536e506bb90914c243a12b397b9a998f85ae2cbd9ba02dfd03a9e155ca5ca0f4 ``` As bytes: ``` -18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 8B 2E 26 -43 06 EC 30 4E DA 51 80 07 D1 76 48 26 38 19 69 +53 6E 50 6B B9 09 14 C2 43 A1 2B 39 7B 9A 99 8F +85 AE 2C BD 9B A0 2D FD 03 A9 E1 55 CA 5C A0 F4 ``` **Step 2: Gzip compression** -Gzip output is implementation-dependent (timestamps, OS flags vary). For this example, we use a representative compressed size of **25 bytes**. The actual gzip output will differ between implementations, but the pipeline and sizes are computed from this value. +Gzip output is implementation-dependent (timestamps, OS flags vary). For this example, we use a representative compressed size of **30 bytes**. The actual gzip output will differ between implementations, but the pipeline and sizes are computed from this value. -- `compressed_size = 25` +- `compressed_size = 30` **Step 3: Compute encrypted_size (PKCS7 padding)** ``` -encrypted_size = ((25 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes +encrypted_size = ((30 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes ``` -PKCS7 padding adds `32 - 25 = 7` bytes of value `0x07`. +PKCS7 padding adds `32 - 30 = 2` bytes of value `0x02`. **Step 4: AES-256-CBC encryption** @@ -571,67 +602,45 @@ HMAC-SHA-256(key, HMAC_input) = <32 bytes> The HMAC value depends on the actual ciphertext; representative bytes (`0xC1` repeated) are used in the hex dump. In a real implementation, this MUST be computed from the actual IV and ciphertext. -#### File 2: `data.bin` +- `entry_type`: `0x00` +- `permissions`: `0o644` = `0x01A4` (LE: `A4 01`) -**Step 1: SHA-256 checksum of original content** +#### Entry 3: `project/empty` (directory) -``` -SHA-256(0x01 * 32) = 72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793 -``` +Directory entries have no data. All crypto fields are zero-filled (identical pattern to Entry 1): -As bytes: -``` -72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D -ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93 -``` - -**Step 2: Gzip compression** - -32 bytes of identical content compresses well. Representative compressed size: **22 bytes**. - -- `compressed_size = 22` - -**Step 3: Compute encrypted_size (PKCS7 padding)** - -``` -encrypted_size = ((22 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes -``` - -PKCS7 padding adds `32 - 22 = 10` bytes of value `0x0A`. - -**Step 4: AES-256-CBC encryption** - -- IV (randomly chosen for this example): `11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00` -- Ciphertext: 32 bytes (representative) - -**Step 5: HMAC-SHA-256** - -``` -HMAC_input = IV (16 bytes) || ciphertext (32 bytes) = 48 bytes total -HMAC-SHA-256(key, HMAC_input) = <32 bytes> -``` - -Representative bytes (`0xD2` repeated) used in the hex dump. +- `entry_type`: `0x01` +- `permissions`: `0o755` = `0x01ED` (LE: `ED 01`) +- All size fields, data_offset, iv, hmac, sha256: zero-filled. ### 12.4 Archive Layout | Region | Start Offset | End Offset | Size | Description | |--------|-------------|------------|------|-------------| -| Header | `0x0000` | `0x0027` | 40 bytes | Fixed header | -| TOC Entry 1 | `0x0028` | `0x0095` | 110 bytes | `hello.txt` metadata | -| TOC Entry 2 | `0x0096` | `0x0102` | 109 bytes | `data.bin` metadata | -| Data Block 1 | `0x0103` | `0x0122` | 32 bytes | `hello.txt` ciphertext | -| Data Block 2 | `0x0123` | `0x0142` | 32 bytes | `data.bin` ciphertext | -| **Total** | | | **323 bytes** | | +| Header | `0x0000` | `0x0027` | 40 bytes | Fixed header (version 2) | +| TOC Entry 1 | `0x0028` | `0x009A` | 115 bytes | `project/src` directory metadata | +| TOC Entry 2 | `0x009B` | `0x0115` | 123 bytes | `project/src/main.rs` file metadata | +| TOC Entry 3 | `0x0116` | `0x018A` | 117 bytes | `project/empty` directory metadata | +| Data Block 1 | `0x018B` | `0x01AA` | 32 bytes | `project/src/main.rs` ciphertext | +| **Total** | | | **427 bytes** | | + +**Note:** Only 1 data block exists because 2 of the 3 entries are directories (no data). + +**Entry size verification:** + +``` +Entry 1: 104 + 11 ("project/src") = 115 bytes CHECK +Entry 2: 104 + 19 ("project/src/main.rs") = 123 bytes CHECK +Entry 3: 104 + 13 ("project/empty") = 117 bytes CHECK +``` **Offset verification:** ``` -TOC offset = header_size = 40 (0x28) CHECK -TOC size = entry1_size + entry2_size = 110 + 109 = 219 (0xDB) CHECK -Data Block 1 = toc_offset + toc_size = 40 + 219 = 259 (0x103) CHECK -Data Block 2 = data_offset_1 + encrypted_size_1 = 259 + 32 = 291 (0x123) CHECK -Archive end = data_offset_2 + encrypted_size_2 = 291 + 32 = 323 (0x143) CHECK +TOC offset = header_size = 40 (0x28) CHECK +TOC size = 115 + 123 + 117 = 355 (0x163) CHECK +Data Block 1 = toc_offset + toc_size = 40 + 355 = 395 (0x18B) CHECK +Archive end = data_offset_1 + encrypted_size_1 = 395 + 32 = 427 (0x1AB) CHECK ``` ### 12.5 Header (Bytes 0x0000 - 0x0027) @@ -639,95 +648,123 @@ Archive end = data_offset_2 + encrypted_size_2 = 291 + 32 = 323 (0x143) | Offset | Hex | Field | Value | |--------|-----|-------|-------| | `0x0000` | `00 EA 72 63` | magic | Custom magic bytes | -| `0x0004` | `01` | version | 1 | +| `0x0004` | `02` | version | 2 (v1.1) | | `0x0005` | `01` | flags | `0x01` = compression enabled | -| `0x0006` | `02 00` | file_count | 2 (LE) | +| `0x0006` | `03 00` | entry_count | 3 (LE) | | `0x0008` | `28 00 00 00` | toc_offset | 40 (LE) | -| `0x000C` | `DB 00 00 00` | toc_size | 219 (LE) | +| `0x000C` | `63 01 00 00` | toc_size | 355 (LE) | | `0x0010` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | toc_iv | Zero-filled (TOC not encrypted) | | `0x0020` | `00 00 00 00 00 00 00 00` | reserved | Zero-filled | -### 12.6 File Table Entry 1: `hello.txt` (Bytes 0x0028 - 0x0095) +### 12.6 TOC Entry 1: `project/src` -- directory (Bytes 0x0028 - 0x009A) | Offset | Hex | Field | Value | |--------|-----|-------|-------| -| `0x0028` | `09 00` | name_length | 9 (LE) | -| `0x002A` | `68 65 6C 6C 6F 2E 74 78 74` | name | "hello.txt" (UTF-8) | -| `0x0033` | `05 00 00 00` | original_size | 5 (LE) | -| `0x0037` | `19 00 00 00` | compressed_size | 25 (LE) | -| `0x003B` | `20 00 00 00` | encrypted_size | 32 (LE) | -| `0x003F` | `03 01 00 00` | data_offset | 259 = 0x103 (LE) | -| `0x0043` | `AA BB CC DD EE FF 00 11 22 33 44 55 66 77 88 99` | iv | Example IV for file 1 | -| `0x0053` | `C1 C1 C1 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | -| `0x0073` | `18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 38 19 69` | sha256 | SHA-256 of "Hello" | -| `0x0093` | `01` | compression_flag | 1 (gzip) | -| `0x0094` | `00 00` | padding_after | 0 (no decoy padding) | +| `0x0028` | `0B 00` | name_length | 11 (LE) | +| `0x002A` | `70 72 6F 6A 65 63 74 2F 73 72 63` | name | "project/src" (UTF-8) | +| `0x0035` | `01` | entry_type | `0x01` = directory | +| `0x0036` | `ED 01` | permissions | `0o755` = `0x01ED` (LE) | +| `0x0038` | `00 00 00 00` | original_size | 0 (directory) | +| `0x003C` | `00 00 00 00` | compressed_size | 0 (directory) | +| `0x0040` | `00 00 00 00` | encrypted_size | 0 (directory) | +| `0x0044` | `00 00 00 00` | data_offset | 0 (directory -- no data block) | +| `0x0048` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | iv | Zero-filled (directory) | +| `0x0058` | `00 ... (32 bytes of 0x00)` | hmac | Zero-filled (directory) | +| `0x0078` | `00 ... (32 bytes of 0x00)` | sha256 | Zero-filled (directory) | +| `0x0098` | `00` | compression_flag | 0 (directory) | +| `0x0099` | `00 00` | padding_after | 0 | -**Entry size verification:** `2 + 9 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 110 bytes`. Offset range: `0x0028` to `0x0095` = 110 bytes. CHECK. +**Entry size verification:** `2 + 11 + 1 + 2 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 115 bytes`. Offset range: `0x0028` to `0x009A` = 115 bytes. CHECK. -### 12.7 File Table Entry 2: `data.bin` (Bytes 0x0096 - 0x0102) +### 12.7 TOC Entry 2: `project/src/main.rs` -- file (Bytes 0x009B - 0x0115) | Offset | Hex | Field | Value | |--------|-----|-------|-------| -| `0x0096` | `08 00` | name_length | 8 (LE) | -| `0x0098` | `64 61 74 61 2E 62 69 6E` | name | "data.bin" (UTF-8) | -| `0x00A0` | `20 00 00 00` | original_size | 32 (LE) | -| `0x00A4` | `16 00 00 00` | compressed_size | 22 (LE) | -| `0x00A8` | `20 00 00 00` | encrypted_size | 32 (LE) | -| `0x00AC` | `23 01 00 00` | data_offset | 291 = 0x123 (LE) | -| `0x00B0` | `11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00` | iv | Example IV for file 2 | -| `0x00C0` | `D2 D2 D2 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | -| `0x00E0` | `72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93` | sha256 | SHA-256 of 32 x 0x01 | -| `0x0100` | `01` | compression_flag | 1 (gzip) | -| `0x0101` | `00 00` | padding_after | 0 (no decoy padding) | +| `0x009B` | `13 00` | name_length | 19 (LE) | +| `0x009D` | `70 72 6F 6A 65 63 74 2F 73 72 63 2F 6D 61 69 6E 2E 72 73` | name | "project/src/main.rs" (UTF-8) | +| `0x00B0` | `00` | entry_type | `0x00` = file | +| `0x00B1` | `A4 01` | permissions | `0o644` = `0x01A4` (LE) | +| `0x00B3` | `0E 00 00 00` | original_size | 14 (LE) | +| `0x00B7` | `1E 00 00 00` | compressed_size | 30 (LE) | +| `0x00BB` | `20 00 00 00` | encrypted_size | 32 (LE) | +| `0x00BF` | `8B 01 00 00` | data_offset | 395 = 0x18B (LE) | +| `0x00C3` | `AA BB CC DD EE FF 00 11 22 33 44 55 66 77 88 99` | iv | Example IV for this file | +| `0x00D3` | `C1 C1 C1 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | +| `0x00F3` | `53 6E 50 6B B9 09 14 C2 43 A1 2B 39 7B 9A 99 8F 85 AE 2C BD 9B A0 2D FD 03 A9 E1 55 CA 5C A0 F4` | sha256 | SHA-256 of "fn main() {}\n" | +| `0x0113` | `01` | compression_flag | 1 (gzip) | +| `0x0114` | `00 00` | padding_after | 0 (no decoy padding) | -**Entry size verification:** `2 + 8 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 109 bytes`. Offset range: `0x0096` to `0x0102` = 109 bytes. CHECK. +**Entry size verification:** `2 + 19 + 1 + 2 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 123 bytes`. Offset range: `0x009B` to `0x0115` = 123 bytes. CHECK. -### 12.8 Data Blocks (Bytes 0x0103 - 0x0142) +### 12.8 TOC Entry 3: `project/empty` -- directory (Bytes 0x0116 - 0x018A) -**Data Block 1** (bytes `0x0103` - `0x0122`, 32 bytes): +| Offset | Hex | Field | Value | +|--------|-----|-------|-------| +| `0x0116` | `0D 00` | name_length | 13 (LE) | +| `0x0118` | `70 72 6F 6A 65 63 74 2F 65 6D 70 74 79` | name | "project/empty" (UTF-8) | +| `0x0125` | `01` | entry_type | `0x01` = directory | +| `0x0126` | `ED 01` | permissions | `0o755` = `0x01ED` (LE) | +| `0x0128` | `00 00 00 00` | original_size | 0 (directory) | +| `0x012C` | `00 00 00 00` | compressed_size | 0 (directory) | +| `0x0130` | `00 00 00 00` | encrypted_size | 0 (directory) | +| `0x0134` | `00 00 00 00` | data_offset | 0 (directory -- no data block) | +| `0x0138` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | iv | Zero-filled (directory) | +| `0x0148` | `00 ... (32 bytes of 0x00)` | hmac | Zero-filled (directory) | +| `0x0168` | `00 ... (32 bytes of 0x00)` | sha256 | Zero-filled (directory) | +| `0x0188` | `00` | compression_flag | 0 (directory) | +| `0x0189` | `00 00` | padding_after | 0 | -Ciphertext of gzip-compressed "Hello", encrypted with AES-256-CBC. Actual bytes depend on the gzip output (which includes timestamps) and the IV. Representative value: 32 bytes of ciphertext. +**Entry size verification:** `2 + 13 + 1 + 2 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 117 bytes`. Offset range: `0x0116` to `0x018A` = 117 bytes. CHECK. -**Data Block 2** (bytes `0x0123` - `0x0142`, 32 bytes): +### 12.9 Data Block (Bytes 0x018B - 0x01AA) -Ciphertext of gzip-compressed `0x01 * 32`, encrypted with AES-256-CBC. Representative value: 32 bytes of ciphertext. +Only one data block exists in this archive -- for `project/src/main.rs` (the only file entry). Both directory entries have no data blocks. -### 12.9 Complete Annotated Hex Dump +**Data Block 1** (bytes `0x018B` - `0x01AA`, 32 bytes): -The following hex dump shows the full 323-byte archive. HMAC values (`C1...` and `D2...`) and ciphertext (`E7...` and `F8...`) are representative placeholders. SHA-256 hashes are real computed values. +Ciphertext of gzip-compressed `"fn main() {}\n"`, encrypted with AES-256-CBC. Actual bytes depend on the gzip output (which includes timestamps) and the IV. Representative value: 32 bytes of ciphertext (`0xE7` repeated). + +### 12.10 Complete Annotated Hex Dump + +The following hex dump shows the full 427-byte archive. HMAC values (`C1...`) and ciphertext (`E7...`) are representative placeholders. The SHA-256 hash is a real computed value. ``` -Offset | Hex | ASCII | Annotation ---------|------------------------------------------------|------------------|------------------------------------------ -0x0000 | 00 EA 72 63 01 01 02 00 28 00 00 00 DB 00 00 00 | ..rc....(...... | Header: magic, ver=1, flags=0x01, count=2, toc_off=40, toc_sz=219 -0x0010 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Header: toc_iv (zero-filled, TOC not encrypted) -0x0020 | 00 00 00 00 00 00 00 00 09 00 68 65 6C 6C 6F 2E | ..........hello. | Header: reserved | TOC Entry 1: name_len=9, name="hello." -0x0030 | 74 78 74 05 00 00 00 19 00 00 00 20 00 00 00 03 | txt........ .... | Entry 1: "txt", orig=5, comp=25, enc=32, data_off= -0x0040 | 01 00 00 AA BB CC DD EE FF 00 11 22 33 44 55 66 | ..........."3DUf | Entry 1: =259(0x103), iv[0..15] -0x0050 | 77 88 99 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | w............... | Entry 1: iv[13..15], hmac[0..12] -0x0060 | C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | ................ | Entry 1: hmac[13..28] -0x0070 | C1 C1 C1 18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 | ...._.."q.%.a... | Entry 1: hmac[29..31], sha256[0..12] -0x0080 | 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 | ..&C..0N.Q...vH& | Entry 1: sha256[13..28] -0x0090 | 38 19 69 01 00 00 08 00 64 61 74 61 2E 62 69 6E | 8.i.....data.bin | Entry 1: sha256[29..31], comp=1, pad=0 | Entry 2: name_len=8, name="data.bin" -0x00A0 | 20 00 00 00 16 00 00 00 20 00 00 00 23 01 00 00 | ....... ...#... | Entry 2: orig=32, comp=22, enc=32, data_off=291(0x123) -0x00B0 | 11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00 | ."3DUfw......... | Entry 2: iv[0..15] -0x00C0 | D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 | ................ | Entry 2: hmac[0..15] -0x00D0 | D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 | ................ | Entry 2: hmac[16..31] -0x00E0 | 72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D | r.n."...m......} | Entry 2: sha256[0..15] -0x00F0 | ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93 | .~........!..67. | Entry 2: sha256[16..31] -0x0100 | 01 00 00 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Entry 2: comp=1, pad=0 | Data Block 1: ciphertext[0..12] -0x0110 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Data Block 1: ciphertext[13..28] -0x0120 | E7 E7 E7 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 | ................ | Data Block 1: ciphertext[29..31] | Data Block 2: ciphertext[0..12] -0x0130 | F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 | ................ | Data Block 2: ciphertext[13..28] -0x0140 | F8 F8 F8 | ... | Data Block 2: ciphertext[29..31] +Offset | Hex | ASCII | Annotation +--------|--------------------------------------------------|------------------|------------------------------------------ +0x0000 | 00 EA 72 63 02 01 03 00 28 00 00 00 63 01 00 00 | ..rc....(...c... | Header: magic, ver=2, flags=0x01, count=3, toc_off=40, toc_sz=355 +0x0010 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Header: toc_iv (zero-filled, TOC not encrypted) +0x0020 | 00 00 00 00 00 00 00 00 0B 00 70 72 6F 6A 65 63 | ..........projec | Header: reserved | Entry 1: name_len=11, name="projec" +0x0030 | 74 2F 73 72 63 01 ED 01 00 00 00 00 00 00 00 00 | t/src........... | Entry 1: name="t/src", type=0x01(dir), perms=0o755, orig=0, comp=0 +0x0040 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: enc=0, data_off=0, iv[0..7] +0x0050 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: iv[8..15], hmac[0..7] +0x0060 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: hmac[8..23] +0x0070 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: hmac[24..31], sha256[0..7] +0x0080 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: sha256[8..23] +0x0090 | 00 00 00 00 00 00 00 00 00 00 00 13 00 70 72 6F | .............pro | Entry 1: sha256[24..31], comp=0, pad=0 | Entry 2: name_len=19, name="pro" +0x00A0 | 6A 65 63 74 2F 73 72 63 2F 6D 61 69 6E 2E 72 73 | ject/src/main.rs | Entry 2: name="ject/src/main.rs" +0x00B0 | 00 A4 01 0E 00 00 00 1E 00 00 00 20 00 00 00 8B | ........... .... | Entry 2: type=0x00(file), perms=0o644, orig=14, comp=30, enc=32, data_off= +0x00C0 | 01 00 00 AA BB CC DD EE FF 00 11 22 33 44 55 66 | ..........."3DUf | Entry 2: =395(0x18B), iv[0..12] +0x00D0 | 77 88 99 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | w............... | Entry 2: iv[13..15], hmac[0..12] +0x00E0 | C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | ................ | Entry 2: hmac[13..28] +0x00F0 | C1 C1 C1 53 6E 50 6B B9 09 14 C2 43 A1 2B 39 7B | ...SnPk....C.+9{ | Entry 2: hmac[29..31], sha256[0..12] +0x0100 | 9A 99 8F 85 AE 2C BD 9B A0 2D FD 03 A9 E1 55 CA | .....,...-....U. | Entry 2: sha256[13..28] +0x0110 | 5C A0 F4 01 00 00 0D 00 70 72 6F 6A 65 63 74 2F | \.......project/ | Entry 2: sha256[29..31], comp=1, pad=0 | Entry 3: name_len=13, name="project/" +0x0120 | 65 6D 70 74 79 01 ED 01 00 00 00 00 00 00 00 00 | empty........... | Entry 3: name="empty", type=0x01(dir), perms=0o755, orig=0, comp=0 +0x0130 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: enc=0, data_off=0, iv[0..7] +0x0140 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: iv[8..15], hmac[0..7] +0x0150 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: hmac[8..23] +0x0160 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: hmac[24..31], sha256[0..7] +0x0170 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: sha256[8..23] +0x0180 | 00 00 00 00 00 00 00 00 00 00 00 E7 E7 E7 E7 E7 | ................ | Entry 3: sha256[24..31], comp=0, pad=0 | Data Block 1: ciphertext[0..4] +0x0190 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Data Block 1: ciphertext[5..20] +0x01A0 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ........... | Data Block 1: ciphertext[21..31] ``` -**Total: 323 bytes (0x143).** +**Total: 427 bytes (0x01AB).** -### 12.10 Step-by-Step Shell Decode Walkthrough +### 12.11 Step-by-Step Shell Decode Walkthrough -The following shell commands demonstrate decoding this archive using only `dd` and `xxd`. The `read_le_u16` and `read_le_u32` functions are defined in the Appendix (Section 13). +The following shell commands demonstrate decoding this archive using only `dd` and `xxd`, showing how the decoder handles both directory and file entries. The `read_le_u16` and `read_le_u32` functions are defined in the Appendix (Section 13). ```sh # ------------------------------------------------------- @@ -740,7 +777,7 @@ dd if=archive.bin bs=1 skip=0 count=4 2>/dev/null | xxd -p # Step 2: Read version # ------------------------------------------------------- dd if=archive.bin bs=1 skip=4 count=1 2>/dev/null | xxd -p -# Expected: 01 +# Expected: 02 (version 2 = v1.1 format) # ------------------------------------------------------- # Step 3: Read flags @@ -749,109 +786,123 @@ dd if=archive.bin bs=1 skip=5 count=1 2>/dev/null | xxd -p # Expected: 01 (compression enabled) # ------------------------------------------------------- -# Step 4: Read file count +# Step 4: Read entry count # ------------------------------------------------------- read_le_u16 archive.bin 6 -# Expected: 2 +# Expected: 3 # ------------------------------------------------------- -# Step 5: Read TOC offset +# Step 5: Read TOC offset and size # ------------------------------------------------------- read_le_u32 archive.bin 8 # Expected: 40 - -# ------------------------------------------------------- -# Step 6: Read TOC size -# ------------------------------------------------------- read_le_u32 archive.bin 12 -# Expected: 219 +# Expected: 355 # ------------------------------------------------------- -# Step 7: Read TOC Entry 1 -- name_length +# Step 6: Parse TOC Entry 1 (offset 40) # ------------------------------------------------------- -read_le_u16 archive.bin 40 -# Expected: 9 +NAME_LEN=$(read_le_u16 archive.bin 40) +# Expected: 11 +dd if=archive.bin bs=1 skip=42 count=11 2>/dev/null +# Expected: project/src + +# Read entry_type (1 byte after name) +ENTRY_TYPE=$(dd if=archive.bin bs=1 skip=53 count=1 2>/dev/null | xxd -p) +# Expected: 01 (directory) + +# Read permissions (2 bytes, LE) +PERMS=$(read_le_u16 archive.bin 54) +# Expected: 493 (= 0o755 = 0x01ED) + +# Directory entry: create directory and set permissions +mkdir -p "output/project/src" +chmod 755 "output/project/src" +# Skip to next entry (no ciphertext to process) # ------------------------------------------------------- -# Step 8: Read TOC Entry 1 -- filename +# Step 7: Parse TOC Entry 2 (offset 155 = 0x9B) # ------------------------------------------------------- -dd if=archive.bin bs=1 skip=42 count=9 2>/dev/null -# Expected: hello.txt +NAME_LEN=$(read_le_u16 archive.bin 155) +# Expected: 19 +dd if=archive.bin bs=1 skip=157 count=19 2>/dev/null +# Expected: project/src/main.rs -# ------------------------------------------------------- -# Step 9: Read TOC Entry 1 -- original_size -# ------------------------------------------------------- -read_le_u32 archive.bin 51 -# Expected: 5 +# Read entry_type +ENTRY_TYPE=$(dd if=archive.bin bs=1 skip=176 count=1 2>/dev/null | xxd -p) +# Expected: 00 (file) -# ------------------------------------------------------- -# Step 10: Read TOC Entry 1 -- compressed_size -# ------------------------------------------------------- -read_le_u32 archive.bin 55 -# Expected: 25 +# Read permissions +PERMS=$(read_le_u16 archive.bin 177) +# Expected: 420 (= 0o644 = 0x01A4) -# ------------------------------------------------------- -# Step 11: Read TOC Entry 1 -- encrypted_size -# ------------------------------------------------------- -read_le_u32 archive.bin 59 -# Expected: 32 +# Read sizes +ORIG_SIZE=$(read_le_u32 archive.bin 179) # Expected: 14 +COMP_SIZE=$(read_le_u32 archive.bin 183) # Expected: 30 +ENC_SIZE=$(read_le_u32 archive.bin 187) # Expected: 32 +DATA_OFF=$(read_le_u32 archive.bin 191) # Expected: 395 -# ------------------------------------------------------- -# Step 12: Read TOC Entry 1 -- data_offset -# ------------------------------------------------------- -read_le_u32 archive.bin 63 -# Expected: 259 - -# ------------------------------------------------------- -# Step 13: Read TOC Entry 1 -- IV (16 bytes) -# ------------------------------------------------------- -dd if=archive.bin bs=1 skip=67 count=16 2>/dev/null | xxd -p +# Read IV (16 bytes at offset 195) +IV_HEX=$(dd if=archive.bin bs=1 skip=195 count=16 2>/dev/null | xxd -p) # Expected: aabbccddeeff00112233445566778899 -# ------------------------------------------------------- -# Step 14: Read TOC Entry 1 -- HMAC (32 bytes) -# ------------------------------------------------------- -dd if=archive.bin bs=1 skip=83 count=32 2>/dev/null | xxd -p -# (32 bytes of HMAC for verification) - -# ------------------------------------------------------- -# Step 15: Extract ciphertext for file 1 -# ------------------------------------------------------- -dd if=archive.bin bs=1 skip=259 count=32 of=/tmp/file1.enc 2>/dev/null - -# ------------------------------------------------------- -# Step 16: Verify HMAC for file 1 -# ------------------------------------------------------- -# Create HMAC input: IV (16 bytes) || ciphertext (32 bytes) -IV_HEX="aabbccddeeff00112233445566778899" KEY_HEX="000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f" -# Extract IV and ciphertext, concatenate, compute HMAC -{ - dd if=archive.bin bs=1 skip=67 count=16 2>/dev/null # IV - dd if=archive.bin bs=1 skip=259 count=32 2>/dev/null # ciphertext -} | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${KEY_HEX}" -hex 2>/dev/null \ - | awk '{print $NF}' -# Compare output with stored HMAC from step 14 +# Read HMAC (32 bytes at offset 211) for verification +STORED_HMAC=$(dd if=archive.bin bs=1 skip=211 count=32 2>/dev/null | xxd -p) -# ------------------------------------------------------- -# Step 17: Decrypt file 1 -# ------------------------------------------------------- +# Verify HMAC: HMAC-SHA-256(key, iv || ciphertext) +COMPUTED_HMAC=$({ + dd if=archive.bin bs=1 skip=195 count=16 2>/dev/null # IV + dd if=archive.bin bs=1 skip=395 count=32 2>/dev/null # ciphertext +} | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${KEY_HEX}" -hex 2>/dev/null \ + | awk '{print $NF}') +# Compare COMPUTED_HMAC with STORED_HMAC + +# Extract and decrypt ciphertext +dd if=archive.bin bs=1 skip=395 count=32 of=/tmp/file.enc 2>/dev/null openssl enc -d -aes-256-cbc -nosalt \ -K "${KEY_HEX}" \ -iv "${IV_HEX}" \ - -in /tmp/file1.enc -out /tmp/file1.gz + -in /tmp/file.enc -out /tmp/file.gz + +# Decompress (compression_flag = 1) +gunzip -c /tmp/file.gz > "output/project/src/main.rs" + +# Set permissions +chmod 644 "output/project/src/main.rs" + +# Verify SHA-256 +sha256sum "output/project/src/main.rs" +# Expected: 536e506bb90914c243a12b397b9a998f85ae2cbd9ba02dfd03a9e155ca5ca0f4 # ------------------------------------------------------- -# Step 18: Decompress file 1 +# Step 8: Parse TOC Entry 3 (offset 278 = 0x116) # ------------------------------------------------------- -gunzip -c /tmp/file1.gz > /tmp/hello.txt +NAME_LEN=$(read_le_u16 archive.bin 278) +# Expected: 13 +dd if=archive.bin bs=1 skip=280 count=13 2>/dev/null +# Expected: project/empty + +ENTRY_TYPE=$(dd if=archive.bin bs=1 skip=293 count=1 2>/dev/null | xxd -p) +# Expected: 01 (directory) + +PERMS=$(read_le_u16 archive.bin 294) +# Expected: 493 (= 0o755) + +# Directory entry: create directory and set permissions +mkdir -p "output/project/empty" +chmod 755 "output/project/empty" +# Done -- no ciphertext to process # ------------------------------------------------------- -# Step 19: Verify SHA-256 of extracted file +# Result: output/ contains the full directory tree # ------------------------------------------------------- -sha256sum /tmp/hello.txt -# Expected: 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969 +# output/ +# project/ +# src/ +# main.rs (14 bytes, mode 644) +# empty/ (empty dir, mode 755) ``` ---