NikitolProject/android-encrypted-archiver
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(07-01): replace worked example with v1.1 directory archive

Browse Source 
- New worked example: 3 entries (2 dirs + 1 file) totaling 427 bytes
- Demonstrates nested dir (project/src), file (project/src/main.rs), empty dir (project/empty)
- Entry hex tables show entry_type and permissions fields
- Directory entries have all-zero crypto fields (iv, hmac, sha256, sizes)
- File entry shows full crypto pipeline with real SHA-256 hash
- Archive layout table with verified offsets (header=40, TOC=355, data=32)
- Complete annotated hex dump covers all 427 bytes
- Shell decode walkthrough handles directory entries (mkdir -p + chmod)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
NikitolProject
2026-02-26 21:25:55 +03:00
parent e7535da7ce
commit 37f7dd1f83
1 changed files with 250 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files

449
docs/FORMAT.md
View File

@@ -512,50 +512,81 @@ The following steps MUST be followed in order by all decoders:
## 12. Worked Example ## 12. Worked Example
This section constructs a complete 2-file archive byte by byte. All offsets, field sizes, and hex values are internally consistent and can be verified by summing field sizes. This example serves as a **golden reference** for implementation testing. This section constructs a complete 3-entry directory archive byte by byte, demonstrating the v1.1 format with entry types, permissions, and relative paths. All offsets, field sizes, and hex values are internally consistent and can be verified by summing field sizes. This example serves as a **golden reference** for implementation testing.
### 12.1 Input Files ### 12.1 Input Structure
| File | Name | Content | Size | ```
|------|------|---------|------| project/
| 1 | `hello.txt` | ASCII string `Hello` (bytes: `48 65 6C 6C 6F`) | 5 bytes | project/src/ (directory, mode 0755)
| 2 | `data.bin` | 32 bytes of `0x01` repeated | 32 bytes | project/src/main.rs (file, mode 0644, content: "fn main() {}\n" = 14 bytes)
project/empty/ (empty directory, mode 0755)
```
This demonstrates:
- A nested directory (`project/src/`)
- A file inside a nested directory (`project/src/main.rs`)
- An empty directory (`project/empty/`)
- Three TOC entries total: 2 directories + 1 file
| # | Entry Name | Type | Permissions | Content | Size |
|---|------------|------|-------------|---------|------|
| 1 | `project/src` | directory | `0o755` | (none) | 0 bytes |
| 2 | `project/src/main.rs` | file | `0o644` | `fn main() {}\n` | 14 bytes |
| 3 | `project/empty` | directory | `0o755` | (none) | 0 bytes |
Entries are ordered parent-before-child: `project/src` appears before `project/src/main.rs`.
### 12.2 Parameters ### 12.2 Parameters
- **Key:** 32 bytes: `00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F` - **Key:** 32 bytes: `00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F`
- **Flags:** `0x01` (compression enabled, no obfuscation) - **Flags:** `0x01` (compression enabled, no obfuscation)
- **Version:** `1` - **Version:** `2`
### 12.3 Per-File Pipeline Walkthrough ### 12.3 Per-Entry Pipeline Walkthrough
#### File 1: `hello.txt` #### Entry 1: `project/src` (directory)
Directory entries have no data. All crypto fields are zero-filled:
- `entry_type`: `0x01`
- `permissions`: `0o755` = `0x01ED` (LE: `ED 01`)
- `original_size`: 0
- `compressed_size`: 0
- `encrypted_size`: 0
- `data_offset`: 0
- `iv`: zero-filled (16 bytes of `0x00`)
- `hmac`: zero-filled (32 bytes of `0x00`)
- `sha256`: zero-filled (32 bytes of `0x00`)
- `compression_flag`: 0
#### Entry 2: `project/src/main.rs` (file)
**Step 1: SHA-256 checksum of original content** **Step 1: SHA-256 checksum of original content**
``` ```
SHA-256("Hello") = 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969 SHA-256("fn main() {}\n") = 536e506bb90914c243a12b397b9a998f85ae2cbd9ba02dfd03a9e155ca5ca0f4
``` ```
As bytes: As bytes:
``` ```
18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 8B 2E 26 53 6E 50 6B B9 09 14 C2 43 A1 2B 39 7B 9A 99 8F
43 06 EC 30 4E DA 51 80 07 D1 76 48 26 38 19 69 85 AE 2C BD 9B A0 2D FD 03 A9 E1 55 CA 5C A0 F4
``` ```
**Step 2: Gzip compression** **Step 2: Gzip compression**
Gzip output is implementation-dependent (timestamps, OS flags vary). For this example, we use a representative compressed size of **25 bytes**. The actual gzip output will differ between implementations, but the pipeline and sizes are computed from this value. Gzip output is implementation-dependent (timestamps, OS flags vary). For this example, we use a representative compressed size of **30 bytes**. The actual gzip output will differ between implementations, but the pipeline and sizes are computed from this value.
- `compressed_size = 25` - `compressed_size = 30`
**Step 3: Compute encrypted_size (PKCS7 padding)** **Step 3: Compute encrypted_size (PKCS7 padding)**
``` ```
encrypted_size = ((25 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes encrypted_size = ((30 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes
``` ```
PKCS7 padding adds `32 - 25 = 7` bytes of value `0x07`. PKCS7 padding adds `32 - 30 = 2` bytes of value `0x02`.
**Step 4: AES-256-CBC encryption** **Step 4: AES-256-CBC encryption**
@@ -571,67 +602,45 @@ HMAC-SHA-256(key, HMAC_input) = <32 bytes>
The HMAC value depends on the actual ciphertext; representative bytes (`0xC1` repeated) are used in the hex dump. In a real implementation, this MUST be computed from the actual IV and ciphertext. The HMAC value depends on the actual ciphertext; representative bytes (`0xC1` repeated) are used in the hex dump. In a real implementation, this MUST be computed from the actual IV and ciphertext.
#### File 2: `data.bin` - `entry_type`: `0x00`
- `permissions`: `0o644` = `0x01A4` (LE: `A4 01`)
**Step 1: SHA-256 checksum of original content** #### Entry 3: `project/empty` (directory)
``` Directory entries have no data. All crypto fields are zero-filled (identical pattern to Entry 1):
SHA-256(0x01 * 32) = 72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793
```
As bytes: - `entry_type`: `0x01`
``` - `permissions`: `0o755` = `0x01ED` (LE: `ED 01`)
72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D - All size fields, data_offset, iv, hmac, sha256: zero-filled.
ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93
```
**Step 2: Gzip compression**
32 bytes of identical content compresses well. Representative compressed size: **22 bytes**.
- `compressed_size = 22`
**Step 3: Compute encrypted_size (PKCS7 padding)**
```
encrypted_size = ((22 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes
```
PKCS7 padding adds `32 - 22 = 10` bytes of value `0x0A`.
**Step 4: AES-256-CBC encryption**
- IV (randomly chosen for this example): `11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00`
- Ciphertext: 32 bytes (representative)
**Step 5: HMAC-SHA-256**
```
HMAC_input = IV (16 bytes) || ciphertext (32 bytes) = 48 bytes total
HMAC-SHA-256(key, HMAC_input) = <32 bytes>
```
Representative bytes (`0xD2` repeated) used in the hex dump.
### 12.4 Archive Layout ### 12.4 Archive Layout
| Region | Start Offset | End Offset | Size | Description | | Region | Start Offset | End Offset | Size | Description |
|--------|-------------|------------|------|-------------| |--------|-------------|------------|------|-------------|
| Header | `0x0000` | `0x0027` | 40 bytes | Fixed header | | Header | `0x0000` | `0x0027` | 40 bytes | Fixed header (version 2) |
| TOC Entry 1 | `0x0028` | `0x0095` | 110 bytes | `hello.txt` metadata | | TOC Entry 1 | `0x0028` | `0x009A` | 115 bytes | `project/src` directory metadata |
| TOC Entry 2 | `0x0096` | `0x0102` | 109 bytes | `data.bin` metadata | | TOC Entry 2 | `0x009B` | `0x0115` | 123 bytes | `project/src/main.rs` file metadata |
| Data Block 1 | `0x0103` | `0x0122` | 32 bytes | `hello.txt` ciphertext | | TOC Entry 3 | `0x0116` | `0x018A` | 117 bytes | `project/empty` directory metadata |
| Data Block 2 | `0x0123` | `0x0142` | 32 bytes | `data.bin` ciphertext | | Data Block 1 | `0x018B` | `0x01AA` | 32 bytes | `project/src/main.rs` ciphertext |
| **Total** | | | **323 bytes** | | | **Total** | | | **427 bytes** | |
**Note:** Only 1 data block exists because 2 of the 3 entries are directories (no data).
**Entry size verification:**
```
Entry 1: 104 + 11 ("project/src") = 115 bytes CHECK
Entry 2: 104 + 19 ("project/src/main.rs") = 123 bytes CHECK
Entry 3: 104 + 13 ("project/empty") = 117 bytes CHECK
```
**Offset verification:** **Offset verification:**
``` ```
TOC offset = header_size = 40 (0x28) CHECK TOC offset = header_size = 40 (0x28) CHECK
TOC size = entry1_size + entry2_size = 110 + 109 = 219 (0xDB) CHECK TOC size = 115 + 123 + 117 = 355 (0x163) CHECK
Data Block 1 = toc_offset + toc_size = 40 + 219 = 259 (0x103) CHECK Data Block 1 = toc_offset + toc_size = 40 + 355 = 395 (0x18B) CHECK
Data Block 2 = data_offset_1 + encrypted_size_1 = 259 + 32 = 291 (0x123) CHECK Archive end = data_offset_1 + encrypted_size_1 = 395 + 32 = 427 (0x1AB) CHECK
Archive end = data_offset_2 + encrypted_size_2 = 291 + 32 = 323 (0x143) CHECK
``` ```
### 12.5 Header (Bytes 0x0000 - 0x0027) ### 12.5 Header (Bytes 0x0000 - 0x0027)
@@ -639,95 +648,123 @@ Archive end = data_offset_2 + encrypted_size_2 = 291 + 32 = 323 (0x143)
| Offset | Hex | Field | Value | | Offset | Hex | Field | Value |
|--------|-----|-------|-------| |--------|-----|-------|-------|
| `0x0000` | `00 EA 72 63` | magic | Custom magic bytes | | `0x0000` | `00 EA 72 63` | magic | Custom magic bytes |
| `0x0004` | `01` | version | 1 | | `0x0004` | `02` | version | 2 (v1.1) |
| `0x0005` | `01` | flags | `0x01` = compression enabled | | `0x0005` | `01` | flags | `0x01` = compression enabled |
| `0x0006` | `02 00` | file_count | 2 (LE) | | `0x0006` | `03 00` | entry_count | 3 (LE) |
| `0x0008` | `28 00 00 00` | toc_offset | 40 (LE) | | `0x0008` | `28 00 00 00` | toc_offset | 40 (LE) |
| `0x000C` | `DB 00 00 00` | toc_size | 219 (LE) | | `0x000C` | `63 01 00 00` | toc_size | 355 (LE) |
| `0x0010` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | toc_iv | Zero-filled (TOC not encrypted) | | `0x0010` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | toc_iv | Zero-filled (TOC not encrypted) |
| `0x0020` | `00 00 00 00 00 00 00 00` | reserved | Zero-filled | | `0x0020` | `00 00 00 00 00 00 00 00` | reserved | Zero-filled |
### 12.6 File Table Entry 1: `hello.txt` (Bytes 0x0028 - 0x0095) ### 12.6 TOC Entry 1: `project/src` -- directory (Bytes 0x0028 - 0x009A)
| Offset | Hex | Field | Value | | Offset | Hex | Field | Value |
|--------|-----|-------|-------| |--------|-----|-------|-------|
| `0x0028` | `09 00` | name_length | 9 (LE) | | `0x0028` | `0B 00` | name_length | 11 (LE) |
| `0x002A` | `68 65 6C 6C 6F 2E 74 78 74` | name | "hello.txt" (UTF-8) | | `0x002A` | `70 72 6F 6A 65 63 74 2F 73 72 63` | name | "project/src" (UTF-8) |
| `0x0033` | `05 00 00 00` | original_size | 5 (LE) | | `0x0035` | `01` | entry_type | `0x01` = directory |
| `0x0037` | `19 00 00 00` | compressed_size | 25 (LE) | | `0x0036` | `ED 01` | permissions | `0o755` = `0x01ED` (LE) |
| `0x003B` | `20 00 00 00` | encrypted_size | 32 (LE) | | `0x0038` | `00 00 00 00` | original_size | 0 (directory) |
| `0x003F` | `03 01 00 00` | data_offset | 259 = 0x103 (LE) | | `0x003C` | `00 00 00 00` | compressed_size | 0 (directory) |
| `0x0043` | `AA BB CC DD EE FF 00 11 22 33 44 55 66 77 88 99` | iv | Example IV for file 1 | | `0x0040` | `00 00 00 00` | encrypted_size | 0 (directory) |
| `0x0053` | `C1 C1 C1 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | | `0x0044` | `00 00 00 00` | data_offset | 0 (directory -- no data block) |
| `0x0073` | `18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 38 19 69` | sha256 | SHA-256 of "Hello" | | `0x0048` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | iv | Zero-filled (directory) |
| `0x0093` | `01` | compression_flag | 1 (gzip) | | `0x0058` | `00 ... (32 bytes of 0x00)` | hmac | Zero-filled (directory) |
| `0x0094` | `00 00` | padding_after | 0 (no decoy padding) | | `0x0078` | `00 ... (32 bytes of 0x00)` | sha256 | Zero-filled (directory) |
| `0x0098` | `00` | compression_flag | 0 (directory) |
| `0x0099` | `00 00` | padding_after | 0 |
**Entry size verification:** `2 + 9 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 110 bytes`. Offset range: `0x0028` to `0x0095` = 110 bytes. CHECK. **Entry size verification:** `2 + 11 + 1 + 2 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 115 bytes`. Offset range: `0x0028` to `0x009A` = 115 bytes. CHECK.
### 12.7 File Table Entry 2: `data.bin` (Bytes 0x0096 - 0x0102) ### 12.7 TOC Entry 2: `project/src/main.rs` -- file (Bytes 0x009B - 0x0115)
| Offset | Hex | Field | Value | | Offset | Hex | Field | Value |
|--------|-----|-------|-------| |--------|-----|-------|-------|
| `0x0096` | `08 00` | name_length | 8 (LE) | | `0x009B` | `13 00` | name_length | 19 (LE) |
| `0x0098` | `64 61 74 61 2E 62 69 6E` | name | "data.bin" (UTF-8) | | `0x009D` | `70 72 6F 6A 65 63 74 2F 73 72 63 2F 6D 61 69 6E 2E 72 73` | name | "project/src/main.rs" (UTF-8) |
| `0x00A0` | `20 00 00 00` | original_size | 32 (LE) | | `0x00B0` | `00` | entry_type | `0x00` = file |
| `0x00A4` | `16 00 00 00` | compressed_size | 22 (LE) | | `0x00B1` | `A4 01` | permissions | `0o644` = `0x01A4` (LE) |
| `0x00A8` | `20 00 00 00` | encrypted_size | 32 (LE) | | `0x00B3` | `0E 00 00 00` | original_size | 14 (LE) |
| `0x00AC` | `23 01 00 00` | data_offset | 291 = 0x123 (LE) | | `0x00B7` | `1E 00 00 00` | compressed_size | 30 (LE) |
| `0x00B0` | `11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00` | iv | Example IV for file 2 | | `0x00BB` | `20 00 00 00` | encrypted_size | 32 (LE) |
| `0x00C0` | `D2 D2 D2 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | | `0x00BF` | `8B 01 00 00` | data_offset | 395 = 0x18B (LE) |
| `0x00E0` | `72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93` | sha256 | SHA-256 of 32 x 0x01 | | `0x00C3` | `AA BB CC DD EE FF 00 11 22 33 44 55 66 77 88 99` | iv | Example IV for this file |
| `0x0100` | `01` | compression_flag | 1 (gzip) | | `0x00D3` | `C1 C1 C1 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) |
| `0x0101` | `00 00` | padding_after | 0 (no decoy padding) | | `0x00F3` | `53 6E 50 6B B9 09 14 C2 43 A1 2B 39 7B 9A 99 8F 85 AE 2C BD 9B A0 2D FD 03 A9 E1 55 CA 5C A0 F4` | sha256 | SHA-256 of "fn main() {}\n" |
| `0x0113` | `01` | compression_flag | 1 (gzip) |
| `0x0114` | `00 00` | padding_after | 0 (no decoy padding) |
**Entry size verification:** `2 + 8 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 109 bytes`. Offset range: `0x0096` to `0x0102` = 109 bytes. CHECK. **Entry size verification:** `2 + 19 + 1 + 2 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 123 bytes`. Offset range: `0x009B` to `0x0115` = 123 bytes. CHECK.
### 12.8 Data Blocks (Bytes 0x0103 - 0x0142) ### 12.8 TOC Entry 3: `project/empty` -- directory (Bytes 0x0116 - 0x018A)
**Data Block 1** (bytes `0x0103` - `0x0122`, 32 bytes): | Offset | Hex | Field | Value |
|--------|-----|-------|-------|
| `0x0116` | `0D 00` | name_length | 13 (LE) |
| `0x0118` | `70 72 6F 6A 65 63 74 2F 65 6D 70 74 79` | name | "project/empty" (UTF-8) |
| `0x0125` | `01` | entry_type | `0x01` = directory |
| `0x0126` | `ED 01` | permissions | `0o755` = `0x01ED` (LE) |
| `0x0128` | `00 00 00 00` | original_size | 0 (directory) |
| `0x012C` | `00 00 00 00` | compressed_size | 0 (directory) |
| `0x0130` | `00 00 00 00` | encrypted_size | 0 (directory) |
| `0x0134` | `00 00 00 00` | data_offset | 0 (directory -- no data block) |
| `0x0138` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | iv | Zero-filled (directory) |
| `0x0148` | `00 ... (32 bytes of 0x00)` | hmac | Zero-filled (directory) |
| `0x0168` | `00 ... (32 bytes of 0x00)` | sha256 | Zero-filled (directory) |
| `0x0188` | `00` | compression_flag | 0 (directory) |
| `0x0189` | `00 00` | padding_after | 0 |
Ciphertext of gzip-compressed "Hello", encrypted with AES-256-CBC. Actual bytes depend on the gzip output (which includes timestamps) and the IV. Representative value: 32 bytes of ciphertext. **Entry size verification:** `2 + 13 + 1 + 2 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 117 bytes`. Offset range: `0x0116` to `0x018A` = 117 bytes. CHECK.
**Data Block 2** (bytes `0x0123` - `0x0142`, 32 bytes): ### 12.9 Data Block (Bytes 0x018B - 0x01AA)
Ciphertext of gzip-compressed `0x01 * 32`, encrypted with AES-256-CBC. Representative value: 32 bytes of ciphertext. Only one data block exists in this archive -- for `project/src/main.rs` (the only file entry). Both directory entries have no data blocks.
### 12.9 Complete Annotated Hex Dump **Data Block 1** (bytes `0x018B` - `0x01AA`, 32 bytes):
The following hex dump shows the full 323-byte archive. HMAC values (`C1...` and `D2...`) and ciphertext (`E7...` and `F8...`) are representative placeholders. SHA-256 hashes are real computed values. Ciphertext of gzip-compressed `"fn main() {}\n"`, encrypted with AES-256-CBC. Actual bytes depend on the gzip output (which includes timestamps) and the IV. Representative value: 32 bytes of ciphertext (`0xE7` repeated).
### 12.10 Complete Annotated Hex Dump
The following hex dump shows the full 427-byte archive. HMAC values (`C1...`) and ciphertext (`E7...`) are representative placeholders. The SHA-256 hash is a real computed value.
``` ```
Offset | Hex | ASCII | Annotation Offset | Hex | ASCII | Annotation
--------|------------------------------------------------|------------------|------------------------------------------ --------|--------------------------------------------------|------------------|------------------------------------------
0x0000 | 00 EA 72 63 01 01 02 00 28 00 00 00 DB 00 00 00 | ..rc....(...... | Header: magic, ver=1, flags=0x01, count=2, toc_off=40, toc_sz=219 0x0000 | 00 EA 72 63 02 01 03 00 28 00 00 00 63 01 00 00 | ..rc....(...c... | Header: magic, ver=2, flags=0x01, count=3, toc_off=40, toc_sz=355
0x0010 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Header: toc_iv (zero-filled, TOC not encrypted) 0x0010 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Header: toc_iv (zero-filled, TOC not encrypted)
0x0020 | 00 00 00 00 00 00 00 00 09 00 68 65 6C 6C 6F 2E | ..........hello. | Header: reserved | TOC Entry 1: name_len=9, name="hello." 0x0020 | 00 00 00 00 00 00 00 00 0B 00 70 72 6F 6A 65 63 | ..........projec | Header: reserved | Entry 1: name_len=11, name="projec"
0x0030 | 74 78 74 05 00 00 00 19 00 00 00 20 00 00 00 03 | txt........ .... | Entry 1: "txt", orig=5, comp=25, enc=32, data_off= 0x0030 | 74 2F 73 72 63 01 ED 01 00 00 00 00 00 00 00 00 | t/src........... | Entry 1: name="t/src", type=0x01(dir), perms=0o755, orig=0, comp=0
0x0040 | 01 00 00 AA BB CC DD EE FF 00 11 22 33 44 55 66 | ..........."3DUf | Entry 1: =259(0x103), iv[0..15] 0x0040 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: enc=0, data_off=0, iv[0..7]
0x0050 | 77 88 99 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | w............... | Entry 1: iv[13..15], hmac[0..12] 0x0050 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: iv[8..15], hmac[0..7]
0x0060 | C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | ................ | Entry 1: hmac[13..28] 0x0060 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: hmac[8..23]
0x0070 | C1 C1 C1 18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 | ...._.."q.%.a... | Entry 1: hmac[29..31], sha256[0..12] 0x0070 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: hmac[24..31], sha256[0..7]
0x0080 | 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 | ..&C..0N.Q...vH& | Entry 1: sha256[13..28] 0x0080 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 1: sha256[8..23]
0x0090 | 38 19 69 01 00 00 08 00 64 61 74 61 2E 62 69 6E | 8.i.....data.bin | Entry 1: sha256[29..31], comp=1, pad=0 | Entry 2: name_len=8, name="data.bin" 0x0090 | 00 00 00 00 00 00 00 00 00 00 00 13 00 70 72 6F | .............pro | Entry 1: sha256[24..31], comp=0, pad=0 | Entry 2: name_len=19, name="pro"
0x00A0 | 20 00 00 00 16 00 00 00 20 00 00 00 23 01 00 00 | ....... ...#... | Entry 2: orig=32, comp=22, enc=32, data_off=291(0x123) 0x00A0 | 6A 65 63 74 2F 73 72 63 2F 6D 61 69 6E 2E 72 73 | ject/src/main.rs | Entry 2: name="ject/src/main.rs"
0x00B0 | 11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00 | ."3DUfw......... | Entry 2: iv[0..15] 0x00B0 | 00 A4 01 0E 00 00 00 1E 00 00 00 20 00 00 00 8B | ........... .... | Entry 2: type=0x00(file), perms=0o644, orig=14, comp=30, enc=32, data_off=
0x00C0 | D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 | ................ | Entry 2: hmac[0..15] 0x00C0 | 01 00 00 AA BB CC DD EE FF 00 11 22 33 44 55 66 | ..........."3DUf | Entry 2: =395(0x18B), iv[0..12]
0x00D0 | D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 | ................ | Entry 2: hmac[16..31] 0x00D0 | 77 88 99 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | w............... | Entry 2: iv[13..15], hmac[0..12]
0x00E0 | 72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D | r.n."...m......} | Entry 2: sha256[0..15] 0x00E0 | C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | ................ | Entry 2: hmac[13..28]
0x00F0 | ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93 | .~........!..67. | Entry 2: sha256[16..31] 0x00F0 | C1 C1 C1 53 6E 50 6B B9 09 14 C2 43 A1 2B 39 7B | ...SnPk....C.+9{ | Entry 2: hmac[29..31], sha256[0..12]
0x0100 | 01 00 00 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Entry 2: comp=1, pad=0 | Data Block 1: ciphertext[0..12] 0x0100 | 9A 99 8F 85 AE 2C BD 9B A0 2D FD 03 A9 E1 55 CA | .....,...-....U. | Entry 2: sha256[13..28]
0x0110 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Data Block 1: ciphertext[13..28] 0x0110 | 5C A0 F4 01 00 00 0D 00 70 72 6F 6A 65 63 74 2F | \.......project/ | Entry 2: sha256[29..31], comp=1, pad=0 | Entry 3: name_len=13, name="project/"
0x0120 | E7 E7 E7 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 | ................ | Data Block 1: ciphertext[29..31] | Data Block 2: ciphertext[0..12] 0x0120 | 65 6D 70 74 79 01 ED 01 00 00 00 00 00 00 00 00 | empty........... | Entry 3: name="empty", type=0x01(dir), perms=0o755, orig=0, comp=0
0x0130 | F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 | ................ | Data Block 2: ciphertext[13..28] 0x0130 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: enc=0, data_off=0, iv[0..7]
0x0140 | F8 F8 F8 | ... | Data Block 2: ciphertext[29..31] 0x0140 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: iv[8..15], hmac[0..7]
0x0150 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: hmac[8..23]
0x0160 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: hmac[24..31], sha256[0..7]
0x0170 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Entry 3: sha256[8..23]
0x0180 | 00 00 00 00 00 00 00 00 00 00 00 E7 E7 E7 E7 E7 | ................ | Entry 3: sha256[24..31], comp=0, pad=0 | Data Block 1: ciphertext[0..4]
0x0190 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Data Block 1: ciphertext[5..20]
0x01A0 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ........... | Data Block 1: ciphertext[21..31]
``` ```
**Total: 323 bytes (0x143).** **Total: 427 bytes (0x01AB).**
### 12.10 Step-by-Step Shell Decode Walkthrough ### 12.11 Step-by-Step Shell Decode Walkthrough
The following shell commands demonstrate decoding this archive using only `dd` and `xxd`. The `read_le_u16` and `read_le_u32` functions are defined in the Appendix (Section 13). The following shell commands demonstrate decoding this archive using only `dd` and `xxd`, showing how the decoder handles both directory and file entries. The `read_le_u16` and `read_le_u32` functions are defined in the Appendix (Section 13).
```sh ```sh
# ------------------------------------------------------- # -------------------------------------------------------
@@ -740,7 +777,7 @@ dd if=archive.bin bs=1 skip=0 count=4 2>/dev/null | xxd -p
# Step 2: Read version # Step 2: Read version
# ------------------------------------------------------- # -------------------------------------------------------
dd if=archive.bin bs=1 skip=4 count=1 2>/dev/null | xxd -p dd if=archive.bin bs=1 skip=4 count=1 2>/dev/null | xxd -p
# Expected: 01 # Expected: 02 (version 2 = v1.1 format)
# ------------------------------------------------------- # -------------------------------------------------------
# Step 3: Read flags # Step 3: Read flags
@@ -749,109 +786,123 @@ dd if=archive.bin bs=1 skip=5 count=1 2>/dev/null | xxd -p
# Expected: 01 (compression enabled) # Expected: 01 (compression enabled)
# ------------------------------------------------------- # -------------------------------------------------------
# Step 4: Read file count # Step 4: Read entry count
# ------------------------------------------------------- # -------------------------------------------------------
read_le_u16 archive.bin 6 read_le_u16 archive.bin 6
# Expected: 2 # Expected: 3
# ------------------------------------------------------- # -------------------------------------------------------
# Step 5: Read TOC offset # Step 5: Read TOC offset and size
# ------------------------------------------------------- # -------------------------------------------------------
read_le_u32 archive.bin 8 read_le_u32 archive.bin 8
# Expected: 40 # Expected: 40
# -------------------------------------------------------
# Step 6: Read TOC size
# -------------------------------------------------------
read_le_u32 archive.bin 12 read_le_u32 archive.bin 12
# Expected: 219 # Expected: 355
# ------------------------------------------------------- # -------------------------------------------------------
# Step 7: Read TOC Entry 1 -- name_length # Step 6: Parse TOC Entry 1 (offset 40)
# ------------------------------------------------------- # -------------------------------------------------------
read_le_u16 archive.bin 40 NAME_LEN=$(read_le_u16 archive.bin 40)
# Expected: 9 # Expected: 11
dd if=archive.bin bs=1 skip=42 count=11 2>/dev/null
# Expected: project/src
# Read entry_type (1 byte after name)
ENTRY_TYPE=$(dd if=archive.bin bs=1 skip=53 count=1 2>/dev/null | xxd -p)
# Expected: 01 (directory)
# Read permissions (2 bytes, LE)
PERMS=$(read_le_u16 archive.bin 54)
# Expected: 493 (= 0o755 = 0x01ED)
# Directory entry: create directory and set permissions
mkdir -p "output/project/src"
chmod 755 "output/project/src"
# Skip to next entry (no ciphertext to process)
# ------------------------------------------------------- # -------------------------------------------------------
# Step 8: Read TOC Entry 1 -- filename # Step 7: Parse TOC Entry 2 (offset 155 = 0x9B)
# ------------------------------------------------------- # -------------------------------------------------------
dd if=archive.bin bs=1 skip=42 count=9 2>/dev/null NAME_LEN=$(read_le_u16 archive.bin 155)
# Expected: hello.txt # Expected: 19
dd if=archive.bin bs=1 skip=157 count=19 2>/dev/null
# Expected: project/src/main.rs
# ------------------------------------------------------- # Read entry_type
# Step 9: Read TOC Entry 1 -- original_size ENTRY_TYPE=$(dd if=archive.bin bs=1 skip=176 count=1 2>/dev/null | xxd -p)
# ------------------------------------------------------- # Expected: 00 (file)
read_le_u32 archive.bin 51
# Expected: 5
# ------------------------------------------------------- # Read permissions
# Step 10: Read TOC Entry 1 -- compressed_size PERMS=$(read_le_u16 archive.bin 177)
# ------------------------------------------------------- # Expected: 420 (= 0o644 = 0x01A4)
read_le_u32 archive.bin 55
# Expected: 25
# ------------------------------------------------------- # Read sizes
# Step 11: Read TOC Entry 1 -- encrypted_size ORIG_SIZE=$(read_le_u32 archive.bin 179) # Expected: 14
# ------------------------------------------------------- COMP_SIZE=$(read_le_u32 archive.bin 183) # Expected: 30
read_le_u32 archive.bin 59 ENC_SIZE=$(read_le_u32 archive.bin 187) # Expected: 32
# Expected: 32 DATA_OFF=$(read_le_u32 archive.bin 191) # Expected: 395
# ------------------------------------------------------- # Read IV (16 bytes at offset 195)
# Step 12: Read TOC Entry 1 -- data_offset IV_HEX=$(dd if=archive.bin bs=1 skip=195 count=16 2>/dev/null | xxd -p)
# -------------------------------------------------------
read_le_u32 archive.bin 63
# Expected: 259
# -------------------------------------------------------
# Step 13: Read TOC Entry 1 -- IV (16 bytes)
# -------------------------------------------------------
dd if=archive.bin bs=1 skip=67 count=16 2>/dev/null | xxd -p
# Expected: aabbccddeeff00112233445566778899 # Expected: aabbccddeeff00112233445566778899
# -------------------------------------------------------
# Step 14: Read TOC Entry 1 -- HMAC (32 bytes)
# -------------------------------------------------------
dd if=archive.bin bs=1 skip=83 count=32 2>/dev/null | xxd -p
# (32 bytes of HMAC for verification)
# -------------------------------------------------------
# Step 15: Extract ciphertext for file 1
# -------------------------------------------------------
dd if=archive.bin bs=1 skip=259 count=32 of=/tmp/file1.enc 2>/dev/null
# -------------------------------------------------------
# Step 16: Verify HMAC for file 1
# -------------------------------------------------------
# Create HMAC input: IV (16 bytes) || ciphertext (32 bytes)
IV_HEX="aabbccddeeff00112233445566778899"
KEY_HEX="000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f" KEY_HEX="000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"
# Extract IV and ciphertext, concatenate, compute HMAC # Read HMAC (32 bytes at offset 211) for verification
{ STORED_HMAC=$(dd if=archive.bin bs=1 skip=211 count=32 2>/dev/null | xxd -p)
dd if=archive.bin bs=1 skip=67 count=16 2>/dev/null # IV
dd if=archive.bin bs=1 skip=259 count=32 2>/dev/null # ciphertext
} | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${KEY_HEX}" -hex 2>/dev/null \
| awk '{print $NF}'
# Compare output with stored HMAC from step 14
# ------------------------------------------------------- # Verify HMAC: HMAC-SHA-256(key, iv || ciphertext)
# Step 17: Decrypt file 1 COMPUTED_HMAC=$({
# ------------------------------------------------------- dd if=archive.bin bs=1 skip=195 count=16 2>/dev/null # IV
dd if=archive.bin bs=1 skip=395 count=32 2>/dev/null # ciphertext
} | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${KEY_HEX}" -hex 2>/dev/null \
| awk '{print $NF}')
# Compare COMPUTED_HMAC with STORED_HMAC
# Extract and decrypt ciphertext
dd if=archive.bin bs=1 skip=395 count=32 of=/tmp/file.enc 2>/dev/null
openssl enc -d -aes-256-cbc -nosalt \ openssl enc -d -aes-256-cbc -nosalt \
-K "${KEY_HEX}" \ -K "${KEY_HEX}" \
-iv "${IV_HEX}" \ -iv "${IV_HEX}" \
-in /tmp/file1.enc -out /tmp/file1.gz -in /tmp/file.enc -out /tmp/file.gz
# Decompress (compression_flag = 1)
gunzip -c /tmp/file.gz > "output/project/src/main.rs"
# Set permissions
chmod 644 "output/project/src/main.rs"
# Verify SHA-256
sha256sum "output/project/src/main.rs"
# Expected: 536e506bb90914c243a12b397b9a998f85ae2cbd9ba02dfd03a9e155ca5ca0f4
# ------------------------------------------------------- # -------------------------------------------------------
# Step 18: Decompress file 1 # Step 8: Parse TOC Entry 3 (offset 278 = 0x116)
# ------------------------------------------------------- # -------------------------------------------------------
gunzip -c /tmp/file1.gz > /tmp/hello.txt NAME_LEN=$(read_le_u16 archive.bin 278)
# Expected: 13
dd if=archive.bin bs=1 skip=280 count=13 2>/dev/null
# Expected: project/empty
ENTRY_TYPE=$(dd if=archive.bin bs=1 skip=293 count=1 2>/dev/null | xxd -p)
# Expected: 01 (directory)
PERMS=$(read_le_u16 archive.bin 294)
# Expected: 493 (= 0o755)
# Directory entry: create directory and set permissions
mkdir -p "output/project/empty"
chmod 755 "output/project/empty"
# Done -- no ciphertext to process
# ------------------------------------------------------- # -------------------------------------------------------
# Step 19: Verify SHA-256 of extracted file # Result: output/ contains the full directory tree
# ------------------------------------------------------- # -------------------------------------------------------
sha256sum /tmp/hello.txt # output/
# Expected: 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969 # project/
# src/
# main.rs (14 bytes, mode 644)
# empty/ (empty dir, mode 755)
``` ```
--- ---