android-encrypted-archiver/.planning/ROADMAP.md

# Roadmap: Encrypted Archive

## Milestones

- **v1.0 Core Archive** - Phases 1-6 (shipped 2026-02-25)
- **v1.1 Directory Support** - Phases 7-11 (in progress)

## Overview

Build a custom encrypted archive format that standard tools cannot recognize or extract. v1.0 delivered the complete pipeline: format spec, Rust archiver with crypto, round-trip tests, Kotlin decoder, shell decoder, and obfuscation hardening. v1.1 adds recursive directory archival with path preservation, Unix permissions, and empty directory support across all three decoders.

## Phases

**Phase Numbering:**
- Integer phases (1, 2, 3): Planned milestone work
- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED)

Decimal phases appear between their surrounding integers in numeric order.

<details>
<summary>v1.0 Core Archive (Phases 1-6) - SHIPPED 2026-02-25</summary>

- [x] **Phase 1: Format Specification** - Document the complete binary format before writing any code (completed 2026-02-24)
- [x] **Phase 2: Core Archiver** - Rust CLI that compresses, encrypts, and packs files into the custom format (completed 2026-02-24)
- [x] **Phase 3: Round-Trip Verification** - Rust unpack command + golden test vectors + unit tests proving byte-identical round-trips (completed 2026-02-24)
- [x] **Phase 4: Kotlin Decoder** - Android 13 decoder using javax.crypto and java.util.zip (primary extraction path) (completed 2026-02-25)
- [x] **Phase 5: Shell Decoder** - Busybox shell script decoder using dd/xxd/openssl/gunzip (fallback extraction) (completed 2026-02-25)
- [x] **Phase 6: Obfuscation Hardening** - XOR-obfuscated headers, encrypted file table, decoy padding to defeat casual analysis (completed 2026-02-25)

</details>

### v1.1 Directory Support (In Progress)

- [ ] **Phase 7: Format Spec Update** - Extend FORMAT.md with entry type, permission bits, and relative path fields in TOC
- [ ] **Phase 8: Rust Directory Archiver** - Recursive directory traversal, path-preserving pack/unpack, empty dirs, and mode bits in Rust CLI
- [ ] **Phase 9: Kotlin Decoder Update** - Kotlin decoder parses new TOC, creates directory hierarchy, and sets permissions
- [ ] **Phase 10: Shell Decoder Update** - Shell decoder parses new TOC, mkdir -p for hierarchy, chmod for permissions
- [ ] **Phase 11: Directory Cross-Validation** - Round-trip tests with nested dirs, empty dirs, mode bits, and cross-decoder verification

## Phase Details

<details>
<summary>v1.0 Core Archive (Phases 1-6) - SHIPPED 2026-02-25</summary>

### Phase 1: Format Specification
**Goal**: A complete, unambiguous binary format document that all three implementations can build against
**Depends on**: Nothing (first phase)
**Requirements**: FMT-05
**Success Criteria** (what must be TRUE):
  1. Format spec document exists with byte-level field definitions (offsets, sizes, types) for header, file table, and data blocks
  2. Spec defines magic bytes, version field, encryption parameters (AES-256-CBC, IV storage, HMAC placement), and compression flags
  3. Spec includes a worked example: a concrete archive with 2 files showing exact byte layout
  4. Spec addresses all obfuscation features (XOR headers, encrypted TOC, decoy padding) even though implementation is Phase 6
**Plans**: 1 plan

Plans:
- [x] 01-01-PLAN.md -- Write complete binary format specification with byte-level field definitions, worked example, and shell reference appendix

### Phase 2: Core Archiver
**Goal**: A working Rust CLI that takes input files and produces a valid encrypted archive
**Depends on**: Phase 1
**Requirements**: FMT-01, FMT-02, FMT-03, FMT-04, ENC-01, ENC-02, ENC-03, ENC-04, ENC-05, CMP-01, CMP-02, INT-01, CLI-01, CLI-02, CLI-03
**Success Criteria** (what must be TRUE):
  1. Running `encrypted_archive pack file1.txt file2.apk -o archive.bin` produces a single binary file
  2. The output file is not recognized by `file`, `binwalk`, or `7z` (custom magic bytes, no standard signatures)
  3. Each file in the archive is independently compressed (gzip) and encrypted (AES-256-CBC) with a unique random IV
  4. HMAC-SHA256 is computed over IV+ciphertext for each file (encrypt-then-MAC)
  5. Running `encrypted_archive inspect archive.bin` shows file count, names, and sizes without decrypting content
**Plans**: 2 plans

Plans:
- [x] 02-01-PLAN.md -- Project scaffolding, binary format types, crypto pipeline, and compression module
- [x] 02-02-PLAN.md -- Pack, inspect, and unpack commands with full archive orchestration

### Phase 3: Round-Trip Verification
**Goal**: Proven byte-identical round-trips through the Rust unpack command, backed by golden test vectors
**Depends on**: Phase 2
**Requirements**: INT-02, TST-01, TST-02, TST-03
**Success Criteria** (what must be TRUE):
  1. Running `encrypted_archive unpack archive.bin -o output_dir/` extracts all files byte-identical to originals (verified by SHA-256 comparison)
  2. Golden test vectors exist: a known plaintext + known key + known IV produces an expected ciphertext (checked in CI)
  3. Unit tests cover each pipeline stage independently: compression, encryption, HMAC, format serialization/deserialization
  4. Round-trip succeeds for edge cases: empty file, large APK (>10MB), file with non-ASCII name (Cyrillic)
**Plans**: 2 plans

Plans:
- [x] 03-01-PLAN.md -- Library crate structure, dev-dependencies, and unit tests for crypto/compression/format modules
- [x] 03-02-PLAN.md -- Golden test vectors with fixed IV/key and CLI round-trip integration tests

### Phase 4: Kotlin Decoder
**Goal**: Android 13 Kotlin code that extracts files from the custom archive using only Android SDK built-ins
**Depends on**: Phase 3
**Requirements**: KOT-01, KOT-02, KOT-03, KOT-04
**Success Criteria** (what must be TRUE):
  1. Kotlin decoder extracts all files from an archive created by the Rust archiver, producing byte-identical output
  2. Decoder uses only javax.crypto (AES/CBC/PKCS5Padding) and java.util.zip.GZIPInputStream -- no native libraries or third-party dependencies
  3. Decoder verifies HMAC-SHA256 before attempting decryption (fails fast on tampered data)
  4. Decoder verifies SHA-256 checksum after decompression (catches decompression corruption)
**Plans**: 1 plan

Plans:
- [x] 04-01-PLAN.md -- Kotlin ArchiveDecoder with full decode pipeline and cross-validation test script

### Phase 5: Shell Decoder
**Goal**: A busybox-compatible shell script that extracts files from the custom archive as a fallback when Kotlin is unavailable
**Depends on**: Phase 3
**Requirements**: SHL-01, SHL-02, SHL-03
**Success Criteria** (what must be TRUE):
  1. Shell script extracts all files from an archive created by the Rust archiver, producing byte-identical output
  2. Script uses only standard busybox commands: dd, xxd, openssl (with -K/-iv/-nosalt for raw key mode), gunzip
  3. Script correctly handles files with non-ASCII names (Cyrillic characters)
**Plans**: 2 plans

Plans:
- [x] 05-01-PLAN.md -- Shell decode.sh: busybox-compatible decoder with full pipeline (dd/xxd/openssl/gunzip)
- [x] 05-02-PLAN.md -- Cross-validation test script and end-to-end verification (6 test cases)

### Phase 6: Obfuscation Hardening
**Goal**: Archive format resists casual analysis -- binwalk, file, strings, and hex editors reveal nothing useful
**Depends on**: Phase 4, Phase 5
**Requirements**: FMT-06, FMT-07, FMT-08
**Success Criteria** (what must be TRUE):
  1. File table (names, sizes, offsets) is encrypted with its own IV -- hex dump of archive reveals no plaintext metadata
  2. All headers are XOR-obfuscated with a fixed key -- no recognizable structure patterns in first 256 bytes
  3. Random decoy padding exists between data blocks -- file boundaries are not detectable by size analysis
  4. All three decoders (Rust, Kotlin, Shell) still produce byte-identical output after obfuscation is applied
**Plans**: 2 plans

Plans:
- [x] 06-01-PLAN.md -- Rust archiver/unpacker obfuscation (XOR header + encrypted TOC + decoy padding + updated tests)
- [x] 06-02-PLAN.md -- Kotlin and Shell decoder obfuscation support + cross-validation tests

</details>

### Phase 7: Format Spec Update
**Goal**: FORMAT.md fully documents the v1.1 TOC entry layout with entry type, permission bits, and relative path semantics -- all three decoders can build against it
**Depends on**: Phase 6 (v1.0 complete)
**Requirements**: FMT-09, FMT-10, FMT-11, FMT-12
**Success Criteria** (what must be TRUE):
  1. FORMAT.md defines the entry type field (1 byte) in TOC entries, distinguishing files from directories
  2. FORMAT.md defines the Unix permissions field (2 bytes, u16 little-endian) in TOC entries with bit layout matching POSIX mode_t lower 12 bits
  3. FORMAT.md specifies that entry names are relative paths using `/` as separator (e.g., `dir/subdir/file.txt`), replacing the previous filename-only convention
  4. FORMAT.md includes an updated worked example showing a directory archive with at least one nested directory, one file, and one empty directory
**Plans**: 1 plan

Plans:
- [x] 07-01-PLAN.md -- Update TOC entry definition (entry_type, permissions, path semantics) and worked example with directory archive

### Phase 8: Rust Directory Archiver
**Goal**: `pack` accepts directories and recursively archives them with full path hierarchy and permissions; `unpack` restores the complete directory tree
**Depends on**: Phase 7
**Requirements**: DIR-01, DIR-02, DIR-03, DIR-04, DIR-05
**Success Criteria** (what must be TRUE):
  1. Running `encrypted_archive pack mydir/ -o archive.bin` recursively includes all files and subdirectories, preserving relative paths from the given root
  2. Running `encrypted_archive pack file.txt mydir/ another.apk -o archive.bin` handles mixed file and directory arguments in a single invocation
  3. Empty directories within the input are stored as TOC entries of type "directory" with zero-length data and are recreated on unpack
  4. Running `encrypted_archive unpack archive.bin -o output/` creates the full directory hierarchy and restores Unix mode bits (e.g., a file packed with 0755 is extracted with 0755)
  5. Running `encrypted_archive inspect archive.bin` shows entry type (file/dir), relative paths, and permissions for each TOC entry
**Plans**: TBD

### Phase 9: Kotlin Decoder Update
**Goal**: Kotlin decoder extracts directory archives created by the updated Rust archiver, preserving hierarchy and permissions on Android
**Depends on**: Phase 8
**Requirements**: KOT-05, KOT-06, KOT-07
**Success Criteria** (what must be TRUE):
  1. Kotlin decoder parses the updated TOC format including entry type and permission fields without errors
  2. Kotlin decoder creates the full directory hierarchy (nested directories) before extracting files into them
  3. Kotlin decoder restores permissions on extracted files and directories using File.setReadable/setWritable/setExecutable
  4. Kotlin decoder handles empty directory entries by creating the directory without attempting to decrypt data
**Plans**: TBD

### Phase 10: Shell Decoder Update
**Goal**: Shell decoder extracts directory archives, creating hierarchy with mkdir -p and restoring permissions with chmod
**Depends on**: Phase 8
**Requirements**: SHL-04, SHL-05, SHL-06
**Success Criteria** (what must be TRUE):
  1. Shell decoder parses the updated TOC format including entry type byte and permission field
  2. Shell decoder uses `mkdir -p` to create the full directory hierarchy (including empty directories) before extracting files
  3. Shell decoder applies `chmod` with the octal mode from the TOC to every extracted file and directory
  4. Shell decoder handles entries with relative paths containing `/` separators correctly (no path traversal issues)
**Plans**: TBD

### Phase 11: Directory Cross-Validation
**Goal**: All three decoders produce identical output for directory archives, verified by automated tests covering edge cases
**Depends on**: Phase 9, Phase 10
**Requirements**: TST-04, TST-05, TST-06, TST-07
**Success Criteria** (what must be TRUE):
  1. Round-trip test passes with 3+ levels of nested directories (e.g., `a/b/c/file.txt`) -- Rust pack then Rust unpack produces byte-identical files
  2. Round-trip test passes with empty directories at multiple levels -- they exist in the unpacked output
  3. Mode bits survive the round-trip: a file packed with mode 0755 is extracted with mode 0755; a file with 0644 is extracted with 0644
  4. Cross-decoder test: archive created by Rust is extracted identically by Kotlin decoder and Shell decoder (same file contents, same directory structure)
**Plans**: TBD

## Progress

**Execution Order:**
Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 -> 11
(Phases 9 and 10 can execute in parallel after Phase 8)

| Phase | Milestone | Plans Complete | Status | Completed |
|-------|-----------|----------------|--------|-----------|
| 1. Format Specification | v1.0 | 1/1 | Complete | 2026-02-24 |
| 2. Core Archiver | v1.0 | 2/2 | Complete | 2026-02-24 |
| 3. Round-Trip Verification | v1.0 | 2/2 | Complete | 2026-02-24 |
| 4. Kotlin Decoder | v1.0 | 1/1 | Complete | 2026-02-25 |
| 5. Shell Decoder | v1.0 | 2/2 | Complete | 2026-02-25 |
| 6. Obfuscation Hardening | v1.0 | 2/2 | Complete | 2026-02-25 |
| 7. Format Spec Update | v1.1 | 1/1 | Complete | 2026-02-26 |
| 8. Rust Directory Archiver | v1.1 | 0/TBD | Not started | - |
| 9. Kotlin Decoder Update | v1.1 | 0/TBD | Not started | - |
| 10. Shell Decoder Update | v1.1 | 0/TBD | Not started | - |
| 11. Directory Cross-Validation | v1.1 | 0/TBD | Not started | - |