android-encrypted-archiver/.planning/ROADMAP.md

Roadmap: Encrypted Archive

Milestones

• v1.0 Core Archive - Phases 1-6 (shipped 2026-02-25)
• v1.1 Directory Support - Phases 7-11 (in progress)

Overview

Build a custom encrypted archive format that standard tools cannot recognize or extract. v1.0 delivered the complete pipeline: format spec, Rust archiver with crypto, round-trip tests, Kotlin decoder, shell decoder, and obfuscation hardening. v1.1 adds recursive directory archival with path preservation, Unix permissions, and empty directory support across all three decoders.

Phases

Phase Numbering:

• Integer phases (1, 2, 3): Planned milestone work
• Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED)

Decimal phases appear between their surrounding integers in numeric order.

v1.0 Core Archive (Phases 1-6) - SHIPPED 2026-02-25

• Phase 1: Format Specification - Document the complete binary format before writing any code (completed 2026-02-24)
• Phase 2: Core Archiver - Rust CLI that compresses, encrypts, and packs files into the custom format (completed 2026-02-24)
• Phase 3: Round-Trip Verification - Rust unpack command + golden test vectors + unit tests proving byte-identical round-trips (completed 2026-02-24)
• Phase 4: Kotlin Decoder - Android 13 decoder using javax.crypto and java.util.zip (primary extraction path) (completed 2026-02-25)
• Phase 5: Shell Decoder - Busybox shell script decoder using dd/xxd/openssl/gunzip (fallback extraction) (completed 2026-02-25)
• Phase 6: Obfuscation Hardening - XOR-obfuscated headers, encrypted file table, decoy padding to defeat casual analysis (completed 2026-02-25)

v1.1 Directory Support (In Progress)

• Phase 7: Format Spec Update - Extend FORMAT.md with entry type, permission bits, and relative path fields in TOC
• Phase 8: Rust Directory Archiver - Recursive directory traversal, path-preserving pack/unpack, empty dirs, and mode bits in Rust CLI
• Phase 9: Kotlin Decoder Update - Kotlin decoder parses new TOC, creates directory hierarchy, and sets permissions
• Phase 10: Shell Decoder Update - Shell decoder parses new TOC, mkdir -p for hierarchy, chmod for permissions
• Phase 11: Directory Cross-Validation - Round-trip tests with nested dirs, empty dirs, mode bits, and cross-decoder verification

Phase Details

v1.0 Core Archive (Phases 1-6) - SHIPPED 2026-02-25

Phase 1: Format Specification

Goal: A complete, unambiguous binary format document that all three implementations can build against Depends on: Nothing (first phase) Requirements: FMT-05 Success Criteria (what must be TRUE):

1. Format spec document exists with byte-level field definitions (offsets, sizes, types) for header, file table, and data blocks
2. Spec defines magic bytes, version field, encryption parameters (AES-256-CBC, IV storage, HMAC placement), and compression flags
3. Spec includes a worked example: a concrete archive with 2 files showing exact byte layout
4. Spec addresses all obfuscation features (XOR headers, encrypted TOC, decoy padding) even though implementation is Phase 6 Plans: 1 plan

Plans:

• 01-01-PLAN.md -- Write complete binary format specification with byte-level field definitions, worked example, and shell reference appendix

Phase 2: Core Archiver

Goal: A working Rust CLI that takes input files and produces a valid encrypted archive Depends on: Phase 1 Requirements: FMT-01, FMT-02, FMT-03, FMT-04, ENC-01, ENC-02, ENC-03, ENC-04, ENC-05, CMP-01, CMP-02, INT-01, CLI-01, CLI-02, CLI-03 Success Criteria (what must be TRUE):

1. Running encrypted_archive pack file1.txt file2.apk -o archive.bin produces a single binary file
2. The output file is not recognized by file, binwalk, or 7z (custom magic bytes, no standard signatures)
3. Each file in the archive is independently compressed (gzip) and encrypted (AES-256-CBC) with a unique random IV
4. HMAC-SHA256 is computed over IV+ciphertext for each file (encrypt-then-MAC)
5. Running encrypted_archive inspect archive.bin shows file count, names, and sizes without decrypting content Plans: 2 plans

Plans:

• 02-01-PLAN.md -- Project scaffolding, binary format types, crypto pipeline, and compression module
• 02-02-PLAN.md -- Pack, inspect, and unpack commands with full archive orchestration

Phase 3: Round-Trip Verification

Goal: Proven byte-identical round-trips through the Rust unpack command, backed by golden test vectors Depends on: Phase 2 Requirements: INT-02, TST-01, TST-02, TST-03 Success Criteria (what must be TRUE):

1. Running encrypted_archive unpack archive.bin -o output_dir/ extracts all files byte-identical to originals (verified by SHA-256 comparison)
2. Golden test vectors exist: a known plaintext + known key + known IV produces an expected ciphertext (checked in CI)
3. Unit tests cover each pipeline stage independently: compression, encryption, HMAC, format serialization/deserialization
4. Round-trip succeeds for edge cases: empty file, large APK (>10MB), file with non-ASCII name (Cyrillic) Plans: 2 plans

Plans:

• 03-01-PLAN.md -- Library crate structure, dev-dependencies, and unit tests for crypto/compression/format modules
• 03-02-PLAN.md -- Golden test vectors with fixed IV/key and CLI round-trip integration tests

Phase 4: Kotlin Decoder

Goal: Android 13 Kotlin code that extracts files from the custom archive using only Android SDK built-ins Depends on: Phase 3 Requirements: KOT-01, KOT-02, KOT-03, KOT-04 Success Criteria (what must be TRUE):

1. Kotlin decoder extracts all files from an archive created by the Rust archiver, producing byte-identical output
2. Decoder uses only javax.crypto (AES/CBC/PKCS5Padding) and java.util.zip.GZIPInputStream -- no native libraries or third-party dependencies
3. Decoder verifies HMAC-SHA256 before attempting decryption (fails fast on tampered data)
4. Decoder verifies SHA-256 checksum after decompression (catches decompression corruption) Plans: 1 plan

Plans:

• 04-01-PLAN.md -- Kotlin ArchiveDecoder with full decode pipeline and cross-validation test script

Phase 5: Shell Decoder

Goal: A busybox-compatible shell script that extracts files from the custom archive as a fallback when Kotlin is unavailable Depends on: Phase 3 Requirements: SHL-01, SHL-02, SHL-03 Success Criteria (what must be TRUE):

1. Shell script extracts all files from an archive created by the Rust archiver, producing byte-identical output
2. Script uses only standard busybox commands: dd, xxd, openssl (with -K/-iv/-nosalt for raw key mode), gunzip
3. Script correctly handles files with non-ASCII names (Cyrillic characters) Plans: 2 plans

Plans:

• 05-01-PLAN.md -- Shell decode.sh: busybox-compatible decoder with full pipeline (dd/xxd/openssl/gunzip)
• 05-02-PLAN.md -- Cross-validation test script and end-to-end verification (6 test cases)

Phase 6: Obfuscation Hardening

Goal: Archive format resists casual analysis -- binwalk, file, strings, and hex editors reveal nothing useful Depends on: Phase 4, Phase 5 Requirements: FMT-06, FMT-07, FMT-08 Success Criteria (what must be TRUE):

1. File table (names, sizes, offsets) is encrypted with its own IV -- hex dump of archive reveals no plaintext metadata
2. All headers are XOR-obfuscated with a fixed key -- no recognizable structure patterns in first 256 bytes
3. Random decoy padding exists between data blocks -- file boundaries are not detectable by size analysis
4. All three decoders (Rust, Kotlin, Shell) still produce byte-identical output after obfuscation is applied Plans: 2 plans

Plans:

• 06-01-PLAN.md -- Rust archiver/unpacker obfuscation (XOR header + encrypted TOC + decoy padding + updated tests)
• 06-02-PLAN.md -- Kotlin and Shell decoder obfuscation support + cross-validation tests

Phase 7: Format Spec Update

Goal: FORMAT.md fully documents the v1.1 TOC entry layout with entry type, permission bits, and relative path semantics -- all three decoders can build against it Depends on: Phase 6 (v1.0 complete) Requirements: FMT-09, FMT-10, FMT-11, FMT-12 Success Criteria (what must be TRUE):

1. FORMAT.md defines the entry type field (1 byte) in TOC entries, distinguishing files from directories
2. FORMAT.md defines the Unix permissions field (2 bytes, u16 little-endian) in TOC entries with bit layout matching POSIX mode_t lower 12 bits
3. FORMAT.md specifies that entry names are relative paths using / as separator (e.g., dir/subdir/file.txt), replacing the previous filename-only convention
4. FORMAT.md includes an updated worked example showing a directory archive with at least one nested directory, one file, and one empty directory Plans: 1 plan

Plans:

• 07-01-PLAN.md -- Update TOC entry definition (entry_type, permissions, path semantics) and worked example with directory archive

Phase 8: Rust Directory Archiver

Goal: pack accepts directories and recursively archives them with full path hierarchy and permissions; unpack restores the complete directory tree Depends on: Phase 7 Requirements: DIR-01, DIR-02, DIR-03, DIR-04, DIR-05 Success Criteria (what must be TRUE):

1. Running encrypted_archive pack mydir/ -o archive.bin recursively includes all files and subdirectories, preserving relative paths from the given root
2. Running encrypted_archive pack file.txt mydir/ another.apk -o archive.bin handles mixed file and directory arguments in a single invocation
3. Empty directories within the input are stored as TOC entries of type "directory" with zero-length data and are recreated on unpack
4. Running encrypted_archive unpack archive.bin -o output/ creates the full directory hierarchy and restores Unix mode bits (e.g., a file packed with 0755 is extracted with 0755)
5. Running encrypted_archive inspect archive.bin shows entry type (file/dir), relative paths, and permissions for each TOC entry Plans: 1 plan

Plans:

• 08-01-PLAN.md -- Update format.rs (v1.1 TocEntry), archive.rs (recursive dir pack/unpack/inspect), and integration tests

Phase 9: Kotlin Decoder Update

Goal: Kotlin decoder extracts directory archives created by the updated Rust archiver, preserving hierarchy and permissions on Android Depends on: Phase 8 Requirements: KOT-05, KOT-06, KOT-07 Success Criteria (what must be TRUE):

1. Kotlin decoder parses the updated TOC format including entry type and permission fields without errors
2. Kotlin decoder creates the full directory hierarchy (nested directories) before extracting files into them
3. Kotlin decoder restores permissions on extracted files and directories using File.setReadable/setWritable/setExecutable
4. Kotlin decoder handles empty directory entries by creating the directory without attempting to decrypt data Plans: TBD

Phase 10: Shell Decoder Update

Goal: Shell decoder extracts directory archives, creating hierarchy with mkdir -p and restoring permissions with chmod Depends on: Phase 8 Requirements: SHL-04, SHL-05, SHL-06 Success Criteria (what must be TRUE):

1. Shell decoder parses the updated TOC format including entry type byte and permission field
2. Shell decoder uses mkdir -p to create the full directory hierarchy (including empty directories) before extracting files
3. Shell decoder applies chmod with the octal mode from the TOC to every extracted file and directory
4. Shell decoder handles entries with relative paths containing / separators correctly (no path traversal issues) Plans: TBD

Phase 11: Directory Cross-Validation

Goal: All three decoders produce identical output for directory archives, verified by automated tests covering edge cases Depends on: Phase 9, Phase 10 Requirements: TST-04, TST-05, TST-06, TST-07 Success Criteria (what must be TRUE):

1. Round-trip test passes with 3+ levels of nested directories (e.g., a/b/c/file.txt) -- Rust pack then Rust unpack produces byte-identical files
2. Round-trip test passes with empty directories at multiple levels -- they exist in the unpacked output
3. Mode bits survive the round-trip: a file packed with mode 0755 is extracted with mode 0755; a file with 0644 is extracted with 0644
4. Cross-decoder test: archive created by Rust is extracted identically by Kotlin decoder and Shell decoder (same file contents, same directory structure) Plans: TBD

Progress

Execution Order: Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 -> 11 (Phases 9 and 10 can execute in parallel after Phase 8)

Phase	Milestone	Plans Complete	Status	Completed
1. Format Specification	v1.0	1/1	Complete	2026-02-24
2. Core Archiver	v1.0	2/2	Complete	2026-02-24
3. Round-Trip Verification	v1.0	2/2	Complete	2026-02-24
4. Kotlin Decoder	v1.0	1/1	Complete	2026-02-25
5. Shell Decoder	v1.0	2/2	Complete	2026-02-25
6. Obfuscation Hardening	v1.0	2/2	Complete	2026-02-25
7. Format Spec Update	v1.1	1/1	Complete	2026-02-26
8. Rust Directory Archiver	v1.1	0/1	Not started	-
9. Kotlin Decoder Update	v1.1	0/TBD	Not started	-
10. Shell Decoder Update	v1.1	0/TBD	Not started	-
11. Directory Cross-Validation	v1.1	0/TBD	Not started	-