# Encrypted Archive Binary Format Specification **Version:** 1.0 **Date:** 2026-02-24 **Status:** Normative --- ## Table of Contents 1. [Overview and Design Goals](#1-overview-and-design-goals) 2. [Notation Conventions](#2-notation-conventions) 3. [Archive Structure Diagram](#3-archive-structure-diagram) 4. [Archive Header Definition](#4-archive-header-definition) 5. [File Table Entry Definition](#5-file-table-entry-definition) 6. [Data Block Layout](#6-data-block-layout) 7. [Encryption and Authentication Details](#7-encryption-and-authentication-details) 8. [Compression Details](#8-compression-details) 9. [Obfuscation Features](#9-obfuscation-features) 10. [Decode Order of Operations](#10-decode-order-of-operations) 11. [Version Compatibility Rules](#11-version-compatibility-rules) 12. [Worked Example](#12-worked-example) 13. [Appendix: Shell Decoder Reference](#13-appendix-shell-decoder-reference) --- ## 1. Overview and Design Goals This document specifies the binary format for `encrypted_archive` -- a custom archive container designed to be **unrecognizable by standard tools**. Standard utilities (`file`, `binwalk`, `7z`, `tar`, `unzip`) must not be able to identify or extract the contents of an archive produced in this format. ### Target Decoders Three independent implementations will build against this specification: 1. **Rust CLI archiver** (`encrypted_archive pack`/`unpack`) -- the reference encoder and primary decoder, runs on Linux/macOS. 2. **Kotlin Android decoder** -- runs on Android 13 (Qualcomm SoC) using only `javax.crypto` and `java.util.zip`. Primary extraction path on the target device. 3. **Busybox shell decoder** -- a fallback shell script using only standard busybox commands: `dd`, `xxd`, `openssl`, `gunzip`, and `sh`. Must work without external dependencies. ### Core Constraint The shell decoder must be able to parse the archive format using `dd` (for byte extraction), `xxd` (for hex conversion), and `openssl enc` (for AES-CBC decryption with raw key mode: `-K`/`-iv`/`-nosalt`). This constraint drives several design choices: - Fixed-size header at a known offset (no variable-length preamble before the TOC pointer) - Absolute offsets (no relative offset chains that require cumulative addition) - IVs stored in the file table, not embedded in data blocks (single `dd` call per extraction) - Little-endian integers (native byte order on ARM and x86) --- ## 2. Notation Conventions | Convention | Meaning | |------------|---------| | **LE** | Little-endian byte order | | **u8** | Unsigned 8-bit integer (1 byte) | | **u16** | Unsigned 16-bit integer (2 bytes) | | **u32** | Unsigned 32-bit integer (4 bytes) | | **bytes** | Raw byte sequence (no endianness) | | Offset `0xNN` | Absolute byte offset from archive byte 0 | | Size | Always in bytes unless stated otherwise | | `\|\|` | Concatenation of byte sequences | - All multi-byte integers are **little-endian (LE)**. - All sizes are in **bytes** unless stated otherwise. - All offsets are **absolute** from archive byte 0 (the first byte of the file). - Filenames are **UTF-8 encoded**, length-prefixed with a u16 byte count (NOT null-terminated). - Reserved fields are **zero-filled** and MUST be written as `0x00` bytes. --- ## 3. Archive Structure Diagram ``` +=======================================+ | ARCHIVE HEADER | Fixed 40 bytes | magic(4) | ver(1) | flags(1) | | file_count(2) | toc_offset(4) | | toc_size(4) | toc_iv(16) | | reserved(8) | +=======================================+ | FILE TABLE (TOC) | Variable size | Entry 1: name, sizes, offset, | Optionally encrypted | iv, hmac, sha256, flags | (see Section 9.2) | Entry 2: ... | | ... | | Entry N: ... | +=======================================+ | DATA BLOCK 1 | encrypted_size bytes | [ciphertext] | +---------------------------------------+ | [DECOY PADDING 1] | Optional (see Section 9.3) +---------------------------------------+ | DATA BLOCK 2 | encrypted_size bytes | [ciphertext] | +---------------------------------------+ | [DECOY PADDING 2] | Optional (see Section 9.3) +---------------------------------------+ | ... | +=======================================+ ``` The archive consists of three contiguous regions: 1. **Header** (fixed 40 bytes) -- contains magic bytes, version, flags, and a pointer to the file table. 2. **File Table (TOC)** (variable size) -- contains one entry per archived file with all metadata needed for extraction. 3. **Data Blocks** (variable size) -- contains the encrypted (and optionally compressed) file contents, one block per file, optionally separated by decoy padding. --- ## 4. Archive Header Definition The header is a fixed-size 40-byte structure at offset 0x00. | Offset | Size | Type | Endian | Field | Description | |--------|------|------|--------|-------|-------------| | `0x00` | 4 | bytes | - | `magic` | Custom magic bytes: `0x00 0xEA 0x72 0x63`. The leading `0x00` signals binary content; the remaining bytes (`0xEA 0x72 0x63`) do not match any known file signature. | | `0x04` | 1 | u8 | - | `version` | Format version. Value `1` for this specification (v1). | | `0x05` | 1 | u8 | - | `flags` | Feature flags bitfield (see below). | | `0x06` | 2 | u16 | LE | `file_count` | Number of files stored in the archive. | | `0x08` | 4 | u32 | LE | `toc_offset` | Absolute byte offset of the file table from archive start. | | `0x0C` | 4 | u32 | LE | `toc_size` | Size of the file table in bytes (if TOC encryption is on, this is the encrypted size including PKCS7 padding). | | `0x10` | 16 | bytes | - | `toc_iv` | Initialization vector for encrypted TOC. Zero-filled (`0x00` x 16) when TOC encryption flag (bit 1) is off. | | `0x20` | 8 | bytes | - | `reserved` | Reserved for future use. MUST be zero-filled. | **Total header size: 40 bytes (0x28).** ### Flags Bitfield | Bit | Mask | Name | Description | |-----|------|------|-------------| | 0 | `0x01` | `compression` | Per-file compression enabled. When set, files MAY be individually gzip-compressed (per-file `compression_flag` controls each file). When clear, all files are stored raw. | | 1 | `0x02` | `toc_encrypted` | File table is encrypted with AES-256-CBC using `toc_iv`. When clear, file table is stored as plaintext. | | 2 | `0x04` | `xor_header` | Header bytes are XOR-obfuscated (see Section 9.1). When clear, header is stored as-is. | | 3 | `0x08` | `decoy_padding` | Random decoy bytes are inserted after data blocks (see Section 9.3). When clear, `padding_after` in every file table entry is 0. | | 4-7 | `0xF0` | reserved | Reserved. MUST be `0`. | --- ## 5. File Table Entry Definition The file table (TOC) is a contiguous sequence of variable-length entries, one per file. Entries are stored in the order files were added to the archive. There is no per-entry delimiter; entries are read sequentially using the `name_length` field to determine where each entry's variable-length name ends. ### Entry Field Table | Field | Size | Type | Endian | Description | |-------|------|------|--------|-------------| | `name_length` | 2 | u16 | LE | Filename length in bytes (UTF-8 encoded byte count). | | `name` | `name_length` | bytes | - | Filename as UTF-8 bytes. NOT null-terminated. May contain path separators (`/`). | | `original_size` | 4 | u32 | LE | Original file size in bytes (before compression). | | `compressed_size` | 4 | u32 | LE | Size after gzip compression. Equals `original_size` if `compression_flag` is 0 (no compression). | | `encrypted_size` | 4 | u32 | LE | Size after AES-256-CBC encryption with PKCS7 padding. Formula: `((compressed_size / 16) + 1) * 16`. | | `data_offset` | 4 | u32 | LE | Absolute byte offset of this file's data block from archive start. | | `iv` | 16 | bytes | - | Random AES-256-CBC initialization vector for this file. | | `hmac` | 32 | bytes | - | HMAC-SHA-256 over `iv || ciphertext`. See Section 7 for details. | | `sha256` | 32 | bytes | - | SHA-256 hash of the original file content (before compression and encryption). | | `compression_flag` | 1 | u8 | - | `0` = raw (no compression), `1` = gzip compressed. | | `padding_after` | 2 | u16 | LE | Number of decoy padding bytes after this file's data block. Always `0` when flags bit 3 (decoy_padding) is off. | ### Entry Size Formula Each file table entry has a total size of: ``` entry_size = 2 + name_length + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 101 + name_length bytes ``` ### File Table Total Size The total file table size is the sum of all entry sizes: ``` toc_size = SUM(101 + name_length_i) for i in 0..file_count-1 ``` When TOC encryption (flags bit 1) is active, the encrypted TOC size includes PKCS7 padding: ``` encrypted_toc_size = ((toc_size / 16) + 1) * 16 ``` The `toc_size` field in the header stores the **actual size on disk** (encrypted size if TOC encryption is on, plaintext size if off). --- ## 6. Data Block Layout Each file has a single contiguous data block containing **only the ciphertext** (the AES-256-CBC encrypted output). ``` [ciphertext: encrypted_size bytes] ``` **Important design decisions:** - The **IV is stored only in the file table entry**, not duplicated at the start of the data block. The data block contains only ciphertext. This simplifies `dd` extraction in the shell decoder: a single `dd` call with the correct offset and size extracts the complete ciphertext. - The **HMAC is stored only in the file table entry**, not appended to the data block. The decoder reads the HMAC from the TOC, then verifies against the data block contents. - If decoy padding is enabled (flags bit 3), `padding_after` bytes of random data follow the ciphertext. The decoder MUST skip these bytes. The next file's data block starts at offset `data_offset + encrypted_size + padding_after`. ### Data Block Ordering Data blocks appear in the same order as file table entries. For file entry `i`: ``` data_offset_0 = toc_offset + toc_size data_offset_i = data_offset_{i-1} + encrypted_size_{i-1} + padding_after_{i-1} ``` --- ## 7. Encryption and Authentication Details ### Pipeline Each file is processed through the following pipeline, in order: ``` original_file | v [1. SHA-256 checksum] --> stored in file table entry as `sha256` | v [2. Gzip compress] (if compression_flag = 1) --> compressed_data | (size = compressed_size) v [3. PKCS7 pad] --> padded_data | (size = encrypted_size) v [4. AES-256-CBC encrypt] (with random IV) --> ciphertext | (size = encrypted_size) v [5. HMAC-SHA-256] (over IV || ciphertext) --> stored in file table entry as `hmac` ``` ### AES-256-CBC - **Key:** 32 bytes (256 bits), hardcoded and shared across all three decoders. - **IV:** 16 bytes, randomly generated for each file. Stored in the file table entry `iv` field. - **Block size:** 16 bytes. - **Mode:** CBC (Cipher Block Chaining). - The same 32-byte key is used for all files in the archive. ### PKCS7 Padding PKCS7 padding is applied to the compressed (or raw) data before encryption. PKCS7 **always adds at least 1 byte** of padding. If the input length is already a multiple of 16, a full 16-byte padding block is added. **Formula:** ``` encrypted_size = ((compressed_size / 16) + 1) * 16 ``` Where `/` is integer division (floor). **Examples:** | `compressed_size` | Padding bytes | `encrypted_size` | |-------------------|---------------|------------------| | 0 | 16 | 16 | | 1 | 15 | 16 | | 15 | 1 | 16 | | 16 | 16 | 32 | | 17 | 15 | 32 | | 31 | 1 | 32 | | 32 | 16 | 48 | | 100 | 12 | 112 | ### HMAC-SHA-256 - **Key:** The same 32-byte key used for AES-256-CBC encryption. (v1 uses a single key for both encryption and authentication. v2 will derive separate subkeys using HKDF.) - **Input:** The concatenation of the 16-byte IV and the ciphertext: ``` HMAC_input = IV (16 bytes) || ciphertext (encrypted_size bytes) Total HMAC input length = 16 + encrypted_size bytes ``` - **Output:** 32 bytes, stored in the file table entry `hmac` field. ### Encrypt-then-MAC This format uses the **Encrypt-then-MAC** construction: 1. The HMAC is computed **after** encryption, over the IV and ciphertext. 2. The decoder **MUST verify the HMAC before attempting decryption**. If the HMAC does not match, the decoder MUST reject the file without decrypting. This prevents padding oracle attacks and avoids processing tampered data. ### SHA-256 Integrity Checksum - **Input:** The original file content (before compression, before encryption). - **Output:** 32 bytes, stored in the file table entry `sha256` field. - **Verification:** After the decoder decrypts and decompresses a file, it computes SHA-256 of the result and compares it to the stored `sha256`. A mismatch indicates data corruption or an incorrect key. --- ## 8. Compression Details - **Algorithm:** Standard gzip (DEFLATE, RFC 1952). - **Granularity:** Per-file. Each file has its own `compression_flag` in the file table entry. - **Global flag:** The header flags bit 0 (`compression`) enables per-file compression. When this bit is clear, ALL files are stored raw regardless of individual `compression_flag` values. - **Recommendation:** Already-compressed files (APK, ZIP, PNG, JPEG) should use `compression_flag = 0` (raw) to avoid size inflation. ### Size Tracking - `original_size`: Size of the file before any processing. - `compressed_size`: Size after gzip compression. If `compression_flag = 0`, then `compressed_size = original_size`. - `encrypted_size`: Size after AES-256-CBC with PKCS7 padding. Always `>= compressed_size`. ### Decompression in Each Decoder | Decoder | Library/Command | |---------|-----------------| | Rust | `flate2` crate (`GzDecoder`) | | Kotlin | `java.util.zip.GZIPInputStream` | | Shell | `gunzip` (busybox) | --- ## 9. Obfuscation Features These features are defined fully in this v1 specification but are intended for implementation in Phase 6 (after all three decoders work without obfuscation). Each feature is controlled by a flag bit in the header and can be activated independently. ### 9.1 XOR Header Obfuscation (flags bit 2, mask `0x04`) When flags bit 2 is set, the entire 40-byte header is XOR-obfuscated with a fixed repeating 8-byte key. **XOR Key:** `0xA5 0x3C 0x96 0x0F 0xE1 0x7B 0x4D 0xC8` (8 bytes, repeating) **XOR Range:** Bytes `0x00` through `0x27` (the entire 40-byte header). **Application:** - XOR is applied **after** the header is fully constructed (all fields written). - The 8-byte key repeats cyclically across the 40 bytes: byte `i` of the header is XORed with `key[i % 8]`. **Decoding:** - The decoder reads the first 40 bytes and XORs them with the same repeating key (XOR is its own inverse). - After de-XOR, the decoder reads header fields normally. **Bootstrapping problem:** When XOR obfuscation is active, the flags byte itself is XORed. The decoder MUST: 1. Always attempt de-XOR on the first 40 bytes. 2. Read the flags byte from the de-XORed header. 3. Check if bit 2 is set. If it is, the de-XOR was correct. If it is not, re-read the header from the original (un-XORed) bytes. Alternatively, the decoder can check the magic bytes: if the first 4 bytes are `0x00 0xEA 0x72 0x63`, the header is not XOR-obfuscated. If they are not, attempt de-XOR and re-check. **When flags bit 2 is 0:** The header is stored as-is (no XOR). ### 9.2 TOC Encryption (flags bit 1, mask `0x02`) When flags bit 1 is set, the entire file table is encrypted with AES-256-CBC. - **Key:** The same 32-byte key used for file encryption. - **IV:** The `toc_iv` field in the header (16 bytes, randomly generated). - **Input:** The serialized file table (all entries concatenated). - **Padding:** PKCS7 padding is applied to the entire serialized TOC. - **`toc_size` in header:** Stores the **encrypted** TOC size (including PKCS7 padding), not the plaintext size. **Decoding:** 1. Read `toc_offset`, `toc_size`, and `toc_iv` from the (de-XORed) header. 2. Read `toc_size` bytes starting at `toc_offset`. 3. Decrypt with AES-256-CBC using `toc_iv` and the 32-byte key. 4. Remove PKCS7 padding. 5. Parse file table entries from the decrypted plaintext. **When flags bit 1 is 0:** The file table is stored as plaintext. `toc_iv` is zero-filled but unused. ### 9.3 Decoy Padding (flags bit 3, mask `0x08`) When flags bit 3 is set, random bytes are inserted after each file's data block. - The number of random padding bytes for each file is stored in the file table entry `padding_after` field (u16 LE). - Padding bytes are cryptographically random and carry no meaningful data. - The decoder MUST skip `padding_after` bytes after reading the ciphertext of each file. - The padding disrupts size-based analysis: an observer cannot determine individual file sizes from the data block layout. **Next data block offset:** ``` next_data_offset = data_offset + encrypted_size + padding_after ``` **When flags bit 3 is 0:** `padding_after` is `0` for every file table entry. No padding bytes exist between data blocks. --- ## 10. Decode Order of Operations The following steps MUST be followed in order by all decoders: ``` 1. Read 40 bytes from offset 0x00. 2. Attempt XOR de-obfuscation: a. Check if bytes 0x00-0x03 equal magic (0x00 0xEA 0x72 0x63). b. If YES: header is not XOR-obfuscated. Use as-is. c. If NO: XOR bytes 0x00-0x27 with key (0xA5 0x3C 0x96 0x0F 0xE1 0x7B 0x4D 0xC8), repeating cyclically. Re-check magic. If still wrong, reject archive. 3. Parse header fields: - Verify magic == 0x00 0xEA 0x72 0x63 - Read version (must be 1) - Read flags - Check for unknown flag bits (bits 4-7 must be 0; reject if not) - Read file_count - Read toc_offset, toc_size, toc_iv 4. Read TOC: a. Seek to toc_offset. b. Read toc_size bytes. c. If flags bit 1 (toc_encrypted) is set: - Decrypt TOC with AES-256-CBC using toc_iv and the 32-byte key. - Remove PKCS7 padding. d. Parse file_count entries sequentially from the (decrypted) TOC bytes. 5. For each file entry (i = 0 to file_count - 1): a. Read ciphertext: - Seek to data_offset. - Read encrypted_size bytes. b. Verify HMAC: - Compute HMAC-SHA-256(key, iv || ciphertext). - Compare with stored hmac (32 bytes). - If mismatch: REJECT this file. Do NOT attempt decryption. c. Decrypt: - Decrypt ciphertext with AES-256-CBC using entry's iv and the 32-byte key. - Remove PKCS7 padding. - Result = compressed_data (or raw data if compression_flag = 0). d. Decompress (if compression_flag = 1): - Decompress with gzip. - Result = original file content. e. Verify integrity: - Compute SHA-256 of the decompressed/raw result. - Compare with stored sha256 (32 bytes). - If mismatch: WARN (data corruption or wrong key). f. Write to output: - Create output file using stored name. - Write the verified content. ``` --- ## 11. Version Compatibility Rules 1. **Version field:** The `version` field at offset `0x04` identifies the format version. This specification defines version `1`. 2. **Forward compatibility:** Decoders MUST reject archives with `version` greater than their supported version. A v1 decoder encountering `version = 2` MUST fail with a clear error message. 3. **Unknown flags:** Decoders MUST reject archives that have any reserved flag bits (bits 4-7) set to `1`. Unknown flags indicate features the decoder does not understand and cannot safely skip. Silent ignoring of unknown flags is prohibited. 4. **Future versions:** Version 2+ MAY: - Add fields after the `reserved` bytes in the header (growing header size). - Define new flag bits (bits 4-7). - Change the `reserved` field to carry metadata. - Introduce HKDF-derived per-file keys (replacing single shared key). 5. **Backward compatibility:** Future versions SHOULD maintain the same magic bytes and the same position of the `version` field (offset `0x04`) so that decoders can read the version before deciding how to proceed. --- ## 12. Worked Example This section constructs a complete 2-file archive byte by byte. All offsets, field sizes, and hex values are internally consistent and can be verified by summing field sizes. This example serves as a **golden reference** for implementation testing. ### 12.1 Input Files | File | Name | Content | Size | |------|------|---------|------| | 1 | `hello.txt` | ASCII string `Hello` (bytes: `48 65 6C 6C 6F`) | 5 bytes | | 2 | `data.bin` | 32 bytes of `0x01` repeated | 32 bytes | ### 12.2 Parameters - **Key:** 32 bytes: `00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F` - **Flags:** `0x01` (compression enabled, no obfuscation) - **Version:** `1` ### 12.3 Per-File Pipeline Walkthrough #### File 1: `hello.txt` **Step 1: SHA-256 checksum of original content** ``` SHA-256("Hello") = 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969 ``` As bytes: ``` 18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 38 19 69 ``` **Step 2: Gzip compression** Gzip output is implementation-dependent (timestamps, OS flags vary). For this example, we use a representative compressed size of **25 bytes**. The actual gzip output will differ between implementations, but the pipeline and sizes are computed from this value. - `compressed_size = 25` **Step 3: Compute encrypted_size (PKCS7 padding)** ``` encrypted_size = ((25 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes ``` PKCS7 padding adds `32 - 25 = 7` bytes of value `0x07`. **Step 4: AES-256-CBC encryption** - IV (randomly chosen for this example): `AA BB CC DD EE FF 00 11 22 33 44 55 66 77 88 99` - Ciphertext: 32 bytes (actual value depends on the gzip output and IV; representative bytes used in the hex dump below) **Step 5: HMAC-SHA-256** ``` HMAC_input = IV (16 bytes) || ciphertext (32 bytes) = 48 bytes total HMAC-SHA-256(key, HMAC_input) = <32 bytes> ``` The HMAC value depends on the actual ciphertext; representative bytes (`0xC1` repeated) are used in the hex dump. In a real implementation, this MUST be computed from the actual IV and ciphertext. #### File 2: `data.bin` **Step 1: SHA-256 checksum of original content** ``` SHA-256(0x01 * 32) = 72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793 ``` As bytes: ``` 72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93 ``` **Step 2: Gzip compression** 32 bytes of identical content compresses well. Representative compressed size: **22 bytes**. - `compressed_size = 22` **Step 3: Compute encrypted_size (PKCS7 padding)** ``` encrypted_size = ((22 / 16) + 1) * 16 = ((1) + 1) * 16 = 32 bytes ``` PKCS7 padding adds `32 - 22 = 10` bytes of value `0x0A`. **Step 4: AES-256-CBC encryption** - IV (randomly chosen for this example): `11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00` - Ciphertext: 32 bytes (representative) **Step 5: HMAC-SHA-256** ``` HMAC_input = IV (16 bytes) || ciphertext (32 bytes) = 48 bytes total HMAC-SHA-256(key, HMAC_input) = <32 bytes> ``` Representative bytes (`0xD2` repeated) used in the hex dump. ### 12.4 Archive Layout | Region | Start Offset | End Offset | Size | Description | |--------|-------------|------------|------|-------------| | Header | `0x0000` | `0x0027` | 40 bytes | Fixed header | | TOC Entry 1 | `0x0028` | `0x0095` | 110 bytes | `hello.txt` metadata | | TOC Entry 2 | `0x0096` | `0x0102` | 109 bytes | `data.bin` metadata | | Data Block 1 | `0x0103` | `0x0122` | 32 bytes | `hello.txt` ciphertext | | Data Block 2 | `0x0123` | `0x0142` | 32 bytes | `data.bin` ciphertext | | **Total** | | | **323 bytes** | | **Offset verification:** ``` TOC offset = header_size = 40 (0x28) CHECK TOC size = entry1_size + entry2_size = 110 + 109 = 219 (0xDB) CHECK Data Block 1 = toc_offset + toc_size = 40 + 219 = 259 (0x103) CHECK Data Block 2 = data_offset_1 + encrypted_size_1 = 259 + 32 = 291 (0x123) CHECK Archive end = data_offset_2 + encrypted_size_2 = 291 + 32 = 323 (0x143) CHECK ``` ### 12.5 Header (Bytes 0x0000 - 0x0027) | Offset | Hex | Field | Value | |--------|-----|-------|-------| | `0x0000` | `00 EA 72 63` | magic | Custom magic bytes | | `0x0004` | `01` | version | 1 | | `0x0005` | `01` | flags | `0x01` = compression enabled | | `0x0006` | `02 00` | file_count | 2 (LE) | | `0x0008` | `28 00 00 00` | toc_offset | 40 (LE) | | `0x000C` | `DB 00 00 00` | toc_size | 219 (LE) | | `0x0010` | `00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00` | toc_iv | Zero-filled (TOC not encrypted) | | `0x0020` | `00 00 00 00 00 00 00 00` | reserved | Zero-filled | ### 12.6 File Table Entry 1: `hello.txt` (Bytes 0x0028 - 0x0095) | Offset | Hex | Field | Value | |--------|-----|-------|-------| | `0x0028` | `09 00` | name_length | 9 (LE) | | `0x002A` | `68 65 6C 6C 6F 2E 74 78 74` | name | "hello.txt" (UTF-8) | | `0x0033` | `05 00 00 00` | original_size | 5 (LE) | | `0x0037` | `19 00 00 00` | compressed_size | 25 (LE) | | `0x003B` | `20 00 00 00` | encrypted_size | 32 (LE) | | `0x003F` | `03 01 00 00` | data_offset | 259 = 0x103 (LE) | | `0x0043` | `AA BB CC DD EE FF 00 11 22 33 44 55 66 77 88 99` | iv | Example IV for file 1 | | `0x0053` | `C1 C1 C1 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | | `0x0073` | `18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 38 19 69` | sha256 | SHA-256 of "Hello" | | `0x0093` | `01` | compression_flag | 1 (gzip) | | `0x0094` | `00 00` | padding_after | 0 (no decoy padding) | **Entry size verification:** `2 + 9 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 110 bytes`. Offset range: `0x0028` to `0x0095` = 110 bytes. CHECK. ### 12.7 File Table Entry 2: `data.bin` (Bytes 0x0096 - 0x0102) | Offset | Hex | Field | Value | |--------|-----|-------|-------| | `0x0096` | `08 00` | name_length | 8 (LE) | | `0x0098` | `64 61 74 61 2E 62 69 6E` | name | "data.bin" (UTF-8) | | `0x00A0` | `20 00 00 00` | original_size | 32 (LE) | | `0x00A4` | `16 00 00 00` | compressed_size | 22 (LE) | | `0x00A8` | `20 00 00 00` | encrypted_size | 32 (LE) | | `0x00AC` | `23 01 00 00` | data_offset | 291 = 0x123 (LE) | | `0x00B0` | `11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00` | iv | Example IV for file 2 | | `0x00C0` | `D2 D2 D2 ... (32 bytes)` | hmac | Representative HMAC (actual depends on ciphertext) | | `0x00E0` | `72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93` | sha256 | SHA-256 of 32 x 0x01 | | `0x0100` | `01` | compression_flag | 1 (gzip) | | `0x0101` | `00 00` | padding_after | 0 (no decoy padding) | **Entry size verification:** `2 + 8 + 4 + 4 + 4 + 4 + 16 + 32 + 32 + 1 + 2 = 109 bytes`. Offset range: `0x0096` to `0x0102` = 109 bytes. CHECK. ### 12.8 Data Blocks (Bytes 0x0103 - 0x0142) **Data Block 1** (bytes `0x0103` - `0x0122`, 32 bytes): Ciphertext of gzip-compressed "Hello", encrypted with AES-256-CBC. Actual bytes depend on the gzip output (which includes timestamps) and the IV. Representative value: 32 bytes of ciphertext. **Data Block 2** (bytes `0x0123` - `0x0142`, 32 bytes): Ciphertext of gzip-compressed `0x01 * 32`, encrypted with AES-256-CBC. Representative value: 32 bytes of ciphertext. ### 12.9 Complete Annotated Hex Dump The following hex dump shows the full 323-byte archive. HMAC values (`C1...` and `D2...`) and ciphertext (`E7...` and `F8...`) are representative placeholders. SHA-256 hashes are real computed values. ``` Offset | Hex | ASCII | Annotation --------|------------------------------------------------|------------------|------------------------------------------ 0x0000 | 00 EA 72 63 01 01 02 00 28 00 00 00 DB 00 00 00 | ..rc....(...... | Header: magic, ver=1, flags=0x01, count=2, toc_off=40, toc_sz=219 0x0010 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ | Header: toc_iv (zero-filled, TOC not encrypted) 0x0020 | 00 00 00 00 00 00 00 00 09 00 68 65 6C 6C 6F 2E | ..........hello. | Header: reserved | TOC Entry 1: name_len=9, name="hello." 0x0030 | 74 78 74 05 00 00 00 19 00 00 00 20 00 00 00 03 | txt........ .... | Entry 1: "txt", orig=5, comp=25, enc=32, data_off= 0x0040 | 01 00 00 AA BB CC DD EE FF 00 11 22 33 44 55 66 | ..........."3DUf | Entry 1: =259(0x103), iv[0..15] 0x0050 | 77 88 99 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | w............... | Entry 1: iv[13..15], hmac[0..12] 0x0060 | C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 C1 | ................ | Entry 1: hmac[13..28] 0x0070 | C1 C1 C1 18 5F 8D B3 22 71 FE 25 F5 61 A6 FC 93 | ...._.."q.%.a... | Entry 1: hmac[29..31], sha256[0..12] 0x0080 | 8B 2E 26 43 06 EC 30 4E DA 51 80 07 D1 76 48 26 | ..&C..0N.Q...vH& | Entry 1: sha256[13..28] 0x0090 | 38 19 69 01 00 00 08 00 64 61 74 61 2E 62 69 6E | 8.i.....data.bin | Entry 1: sha256[29..31], comp=1, pad=0 | Entry 2: name_len=8, name="data.bin" 0x00A0 | 20 00 00 00 16 00 00 00 20 00 00 00 23 01 00 00 | ....... ...#... | Entry 2: orig=32, comp=22, enc=32, data_off=291(0x123) 0x00B0 | 11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF 00 | ."3DUfw......... | Entry 2: iv[0..15] 0x00C0 | D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 | ................ | Entry 2: hmac[0..15] 0x00D0 | D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 D2 | ................ | Entry 2: hmac[16..31] 0x00E0 | 72 CD 6E 84 22 C4 07 FB 6D 09 86 90 F1 13 0B 7D | r.n."...m......} | Entry 2: sha256[0..15] 0x00F0 | ED 7E C2 F7 F5 E1 D3 0B D9 D5 21 F0 15 36 37 93 | .~........!..67. | Entry 2: sha256[16..31] 0x0100 | 01 00 00 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Entry 2: comp=1, pad=0 | Data Block 1: ciphertext[0..12] 0x0110 | E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 E7 | ................ | Data Block 1: ciphertext[13..28] 0x0120 | E7 E7 E7 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 | ................ | Data Block 1: ciphertext[29..31] | Data Block 2: ciphertext[0..12] 0x0130 | F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 F8 | ................ | Data Block 2: ciphertext[13..28] 0x0140 | F8 F8 F8 | ... | Data Block 2: ciphertext[29..31] ``` **Total: 323 bytes (0x143).** ### 12.10 Step-by-Step Shell Decode Walkthrough The following shell commands demonstrate decoding this archive using only `dd` and `xxd`. The `read_le_u16` and `read_le_u32` functions are defined in the Appendix (Section 13). ```sh # ------------------------------------------------------- # Step 1: Read and verify magic bytes # ------------------------------------------------------- dd if=archive.bin bs=1 skip=0 count=4 2>/dev/null | xxd -p # Expected: 00ea7263 # ------------------------------------------------------- # Step 2: Read version # ------------------------------------------------------- dd if=archive.bin bs=1 skip=4 count=1 2>/dev/null | xxd -p # Expected: 01 # ------------------------------------------------------- # Step 3: Read flags # ------------------------------------------------------- dd if=archive.bin bs=1 skip=5 count=1 2>/dev/null | xxd -p # Expected: 01 (compression enabled) # ------------------------------------------------------- # Step 4: Read file count # ------------------------------------------------------- read_le_u16 archive.bin 6 # Expected: 2 # ------------------------------------------------------- # Step 5: Read TOC offset # ------------------------------------------------------- read_le_u32 archive.bin 8 # Expected: 40 # ------------------------------------------------------- # Step 6: Read TOC size # ------------------------------------------------------- read_le_u32 archive.bin 12 # Expected: 219 # ------------------------------------------------------- # Step 7: Read TOC Entry 1 -- name_length # ------------------------------------------------------- read_le_u16 archive.bin 40 # Expected: 9 # ------------------------------------------------------- # Step 8: Read TOC Entry 1 -- filename # ------------------------------------------------------- dd if=archive.bin bs=1 skip=42 count=9 2>/dev/null # Expected: hello.txt # ------------------------------------------------------- # Step 9: Read TOC Entry 1 -- original_size # ------------------------------------------------------- read_le_u32 archive.bin 51 # Expected: 5 # ------------------------------------------------------- # Step 10: Read TOC Entry 1 -- compressed_size # ------------------------------------------------------- read_le_u32 archive.bin 55 # Expected: 25 # ------------------------------------------------------- # Step 11: Read TOC Entry 1 -- encrypted_size # ------------------------------------------------------- read_le_u32 archive.bin 59 # Expected: 32 # ------------------------------------------------------- # Step 12: Read TOC Entry 1 -- data_offset # ------------------------------------------------------- read_le_u32 archive.bin 63 # Expected: 259 # ------------------------------------------------------- # Step 13: Read TOC Entry 1 -- IV (16 bytes) # ------------------------------------------------------- dd if=archive.bin bs=1 skip=67 count=16 2>/dev/null | xxd -p # Expected: aabbccddeeff00112233445566778899 # ------------------------------------------------------- # Step 14: Read TOC Entry 1 -- HMAC (32 bytes) # ------------------------------------------------------- dd if=archive.bin bs=1 skip=83 count=32 2>/dev/null | xxd -p # (32 bytes of HMAC for verification) # ------------------------------------------------------- # Step 15: Extract ciphertext for file 1 # ------------------------------------------------------- dd if=archive.bin bs=1 skip=259 count=32 of=/tmp/file1.enc 2>/dev/null # ------------------------------------------------------- # Step 16: Verify HMAC for file 1 # ------------------------------------------------------- # Create HMAC input: IV (16 bytes) || ciphertext (32 bytes) IV_HEX="aabbccddeeff00112233445566778899" KEY_HEX="000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f" # Extract IV and ciphertext, concatenate, compute HMAC { dd if=archive.bin bs=1 skip=67 count=16 2>/dev/null # IV dd if=archive.bin bs=1 skip=259 count=32 2>/dev/null # ciphertext } | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${KEY_HEX}" -hex 2>/dev/null \ | awk '{print $NF}' # Compare output with stored HMAC from step 14 # ------------------------------------------------------- # Step 17: Decrypt file 1 # ------------------------------------------------------- openssl enc -d -aes-256-cbc -nosalt \ -K "${KEY_HEX}" \ -iv "${IV_HEX}" \ -in /tmp/file1.enc -out /tmp/file1.gz # ------------------------------------------------------- # Step 18: Decompress file 1 # ------------------------------------------------------- gunzip -c /tmp/file1.gz > /tmp/hello.txt # ------------------------------------------------------- # Step 19: Verify SHA-256 of extracted file # ------------------------------------------------------- sha256sum /tmp/hello.txt # Expected: 185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969 ``` --- ## 13. Appendix: Shell Decoder Reference This appendix provides reference shell functions for decoding archives using only standard busybox commands. ### 13.1 Little-Endian Integer Reading ```sh # Read a little-endian u16 from a binary file at a byte offset. # Usage: read_le_u16 # Output: decimal integer value read_le_u16() { local file="$1" offset="$2" local hex=$(dd if="$file" bs=1 skip="$offset" count=2 2>/dev/null | xxd -p) local b0=${hex:0:2} b1=${hex:2:2} printf '%d' "0x${b1}${b0}" } # Read a little-endian u32 from a binary file at a byte offset. # Usage: read_le_u32 # Output: decimal integer value read_le_u32() { local file="$1" offset="$2" local hex=$(dd if="$file" bs=1 skip="$offset" count=4 2>/dev/null | xxd -p) local b0=${hex:0:2} b1=${hex:2:2} b2=${hex:4:2} b3=${hex:6:2} printf '%d' "0x${b3}${b2}${b1}${b0}" } ``` **Busybox compatibility note:** If `xxd` is not available, use `od` as a fallback: ```sh # Fallback using od instead of xxd read_le_u32_od() { local file="$1" offset="$2" local bytes=$(dd if="$file" bs=1 skip="$offset" count=4 2>/dev/null \ | od -A n -t x1 | tr -d ' \n') local b0=${bytes:0:2} b1=${bytes:2:2} b2=${bytes:4:2} b3=${bytes:6:2} printf '%d' "0x${b3}${b2}${b1}${b0}" } ``` ### 13.2 Read Raw Bytes as Hex ```sh # Read N bytes from file at offset as hex string (no spaces) # Usage: read_hex read_hex() { local file="$1" offset="$2" count="$3" dd if="$file" bs=1 skip="$offset" count="$count" 2>/dev/null | xxd -p | tr -d '\n' } ``` ### 13.3 HMAC-SHA-256 Verification ```sh # Verify HMAC-SHA-256 of IV || ciphertext. # Usage: verify_hmac # Returns: 0 if HMAC matches, 1 if not verify_hmac() { local file="$1" local iv_offset="$2" iv_length="$3" local data_offset="$4" data_length="$5" local expected="$6" key="$7" local actual=$( { dd if="$file" bs=1 skip="$iv_offset" count="$iv_length" 2>/dev/null dd if="$file" bs=1 skip="$data_offset" count="$data_length" 2>/dev/null } | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${key}" -hex 2>/dev/null \ | awk '{print $NF}' ) [ "$actual" = "$expected" ] } ``` **Graceful degradation:** If the target busybox `openssl` does not support `-mac HMAC -macopt`, the shell decoder MAY skip HMAC verification. In this case, print a warning: ```sh # Check if openssl HMAC is available if ! echo -n "test" | openssl dgst -sha256 -mac HMAC -macopt hexkey:00 >/dev/null 2>&1; then echo "WARNING: openssl HMAC not available, skipping integrity verification" SKIP_HMAC=1 fi ``` ### 13.4 Single-File Decryption ```sh # Decrypt a single file from the archive. # Usage: decrypt_file decrypt_file() { local archive="$1" local data_offset="$2" encrypted_size="$3" local iv_hex="$4" key_hex="$5" local output="$6" is_compressed="$7" # Extract ciphertext dd if="$archive" bs=1 skip="$data_offset" count="$encrypted_size" 2>/dev/null \ | openssl enc -d -aes-256-cbc -nosalt -K "$key_hex" -iv "$iv_hex" \ > /tmp/_decrypted_$$ # Decompress if needed if [ "$is_compressed" = "1" ]; then gunzip -c /tmp/_decrypted_$$ > "$output" else mv /tmp/_decrypted_$$ "$output" fi rm -f /tmp/_decrypted_$$ } ``` ### 13.5 SHA-256 Verification ```sh # Verify SHA-256 of an extracted file. # Usage: verify_sha256 # Returns: 0 if matches, 1 if not verify_sha256() { local file="$1" expected="$2" local actual=$(sha256sum "$file" | awk '{print $1}') [ "$actual" = "$expected" ] } ``` ### 13.6 Kotlin Decoder Reference For Android implementations using `javax.crypto`: ```kotlin import java.io.ByteArrayInputStream import java.security.MessageDigest import java.util.zip.GZIPInputStream import javax.crypto.Cipher import javax.crypto.Mac import javax.crypto.spec.IvParameterSpec import javax.crypto.spec.SecretKeySpec /** * Decrypt a single file entry from the archive. * * @param ciphertext The encrypted data (encrypted_size bytes from the data block) * @param iv The 16-byte IV from the file table entry * @param key The 32-byte AES key * @return Decrypted data (after PKCS7 unpadding, which is automatic) */ fun decryptFileEntry(ciphertext: ByteArray, iv: ByteArray, key: ByteArray): ByteArray { val cipher = Cipher.getInstance("AES/CBC/PKCS5Padding") // Note: PKCS5Padding in Java/Android == PKCS7 for 16-byte blocks val secretKey = SecretKeySpec(key, "AES") val ivSpec = IvParameterSpec(iv) cipher.init(Cipher.DECRYPT_MODE, secretKey, ivSpec) return cipher.doFinal(ciphertext) } /** * Verify HMAC-SHA-256 of IV || ciphertext. * * @param iv The 16-byte IV * @param ciphertext The encrypted data * @param key The 32-byte key (same as AES key in v1) * @param expectedHmac The 32-byte HMAC from the file table entry * @return true if HMAC matches */ fun verifyHmac(iv: ByteArray, ciphertext: ByteArray, key: ByteArray, expectedHmac: ByteArray): Boolean { val mac = Mac.getInstance("HmacSHA256") mac.init(SecretKeySpec(key, "HmacSHA256")) mac.update(iv) mac.update(ciphertext) val computed = mac.doFinal() return computed.contentEquals(expectedHmac) } /** * Decompress gzip data. * * @param compressed Gzip-compressed data * @return Decompressed data */ fun decompressGzip(compressed: ByteArray): ByteArray { return GZIPInputStream(ByteArrayInputStream(compressed)).readBytes() } /** * Verify SHA-256 checksum of extracted content. * * @param data The decompressed file content * @param expectedSha256 The 32-byte SHA-256 from the file table entry * @return true if checksum matches */ fun verifySha256(data: ByteArray, expectedSha256: ByteArray): Boolean { val digest = MessageDigest.getInstance("SHA-256") val computed = digest.digest(data) return computed.contentEquals(expectedSha256) } ``` **Full decode flow in Kotlin:** ```kotlin // For each file entry: // 1. Read ciphertext from data_offset (encrypted_size bytes) // 2. Verify HMAC BEFORE decryption if (!verifyHmac(entry.iv, ciphertext, key, entry.hmac)) { throw SecurityException("HMAC verification failed for ${entry.name}") } // 3. Decrypt val compressed = decryptFileEntry(ciphertext, entry.iv, key) // 4. Decompress if needed val original = if (entry.compressionFlag == 1) decompressGzip(compressed) else compressed // 5. Verify SHA-256 if (!verifySha256(original, entry.sha256)) { throw SecurityException("SHA-256 verification failed for ${entry.name}") } // 6. Write to file File(outputDir, entry.name).writeBytes(original) ```