From a716d09178b13d6ad41f3f56342a5c847e821aca Mon Sep 17 00:00:00 2001 From: NikitolProject Date: Wed, 25 Feb 2026 04:10:09 +0300 Subject: [PATCH] docs: create milestone v1.1 roadmap (5 phases) --- .planning/REQUIREMENTS.md | 48 ++++++++--------- .planning/ROADMAP.md | 107 ++++++++++++++++++++++++++++++++++---- .planning/STATE.md | 57 ++++++-------------- 3 files changed, 135 insertions(+), 77 deletions(-) diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 2fb37f0..807ac20 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -161,32 +161,32 @@ | TST-01 | Phase 3 | Complete | | TST-02 | Phase 3 | Complete | | TST-03 | Phase 3 | Complete | -| FMT-09 | — | Pending | -| FMT-10 | — | Pending | -| FMT-11 | — | Pending | -| FMT-12 | — | Pending | -| DIR-01 | — | Pending | -| DIR-02 | — | Pending | -| DIR-03 | — | Pending | -| DIR-04 | — | Pending | -| DIR-05 | — | Pending | -| KOT-05 | — | Pending | -| KOT-06 | — | Pending | -| KOT-07 | — | Pending | -| SHL-04 | — | Pending | -| SHL-05 | — | Pending | -| SHL-06 | — | Pending | -| TST-04 | — | Pending | -| TST-05 | — | Pending | -| TST-06 | — | Pending | -| TST-07 | — | Pending | +| FMT-09 | Phase 7 | Pending | +| FMT-10 | Phase 7 | Pending | +| FMT-11 | Phase 7 | Pending | +| FMT-12 | Phase 7 | Pending | +| DIR-01 | Phase 8 | Pending | +| DIR-02 | Phase 8 | Pending | +| DIR-03 | Phase 8 | Pending | +| DIR-04 | Phase 8 | Pending | +| DIR-05 | Phase 8 | Pending | +| KOT-05 | Phase 9 | Pending | +| KOT-06 | Phase 9 | Pending | +| KOT-07 | Phase 9 | Pending | +| SHL-04 | Phase 10 | Pending | +| SHL-05 | Phase 10 | Pending | +| SHL-06 | Phase 10 | Pending | +| TST-04 | Phase 11 | Pending | +| TST-05 | Phase 11 | Pending | +| TST-06 | Phase 11 | Pending | +| TST-07 | Phase 11 | Pending | **Coverage:** -- v1.0 requirements: 30 total — all Complete -- v1.1 requirements: 19 total -- Mapped to phases: 0 (pending roadmap) -- Unmapped: 19 +- v1.0 requirements: 30 total -- all Complete +- v1.1 requirements: 19 total -- all mapped to phases 7-11 +- Mapped to phases: 19/19 +- Unmapped: 0 --- *Requirements defined: 2026-02-24* -*Last updated: 2026-02-25 after milestone v1.1 requirements definition* +*Last updated: 2026-02-25 after v1.1 roadmap creation* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 2cc6821..fa7c8d2 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -1,8 +1,13 @@ # Roadmap: Encrypted Archive +## Milestones + +- **v1.0 Core Archive** - Phases 1-6 (shipped 2026-02-25) +- **v1.1 Directory Support** - Phases 7-11 (in progress) + ## Overview -Build a custom encrypted archive format that standard tools cannot recognize or extract. The format spec comes first (it governs all three implementations), then the Rust archiver with full crypto pipeline, then round-trip verification to catch format bugs early, then Kotlin decoder (primary extraction on Android), then shell decoder (busybox fallback), and finally obfuscation hardening to defeat binwalk/file/strings analysis. +Build a custom encrypted archive format that standard tools cannot recognize or extract. v1.0 delivered the complete pipeline: format spec, Rust archiver with crypto, round-trip tests, Kotlin decoder, shell decoder, and obfuscation hardening. v1.1 adds recursive directory archival with path preservation, Unix permissions, and empty directory support across all three decoders. ## Phases @@ -12,6 +17,9 @@ Build a custom encrypted archive format that standard tools cannot recognize or Decimal phases appear between their surrounding integers in numeric order. +
+v1.0 Core Archive (Phases 1-6) - SHIPPED 2026-02-25 + - [x] **Phase 1: Format Specification** - Document the complete binary format before writing any code (completed 2026-02-24) - [x] **Phase 2: Core Archiver** - Rust CLI that compresses, encrypts, and packs files into the custom format (completed 2026-02-24) - [x] **Phase 3: Round-Trip Verification** - Rust unpack command + golden test vectors + unit tests proving byte-identical round-trips (completed 2026-02-24) @@ -19,8 +27,21 @@ Decimal phases appear between their surrounding integers in numeric order. - [x] **Phase 5: Shell Decoder** - Busybox shell script decoder using dd/xxd/openssl/gunzip (fallback extraction) (completed 2026-02-25) - [x] **Phase 6: Obfuscation Hardening** - XOR-obfuscated headers, encrypted file table, decoy padding to defeat casual analysis (completed 2026-02-25) +
+ +### v1.1 Directory Support (In Progress) + +- [ ] **Phase 7: Format Spec Update** - Extend FORMAT.md with entry type, permission bits, and relative path fields in TOC +- [ ] **Phase 8: Rust Directory Archiver** - Recursive directory traversal, path-preserving pack/unpack, empty dirs, and mode bits in Rust CLI +- [ ] **Phase 9: Kotlin Decoder Update** - Kotlin decoder parses new TOC, creates directory hierarchy, and sets permissions +- [ ] **Phase 10: Shell Decoder Update** - Shell decoder parses new TOC, mkdir -p for hierarchy, chmod for permissions +- [ ] **Phase 11: Directory Cross-Validation** - Round-trip tests with nested dirs, empty dirs, mode bits, and cross-decoder verification + ## Phase Details +
+v1.0 Core Archive (Phases 1-6) - SHIPPED 2026-02-25 + ### Phase 1: Format Specification **Goal**: A complete, unambiguous binary format document that all three implementations can build against **Depends on**: Nothing (first phase) @@ -33,7 +54,7 @@ Decimal phases appear between their surrounding integers in numeric order. **Plans**: 1 plan Plans: -- [ ] 01-01-PLAN.md -- Write complete binary format specification with byte-level field definitions, worked example, and shell reference appendix +- [x] 01-01-PLAN.md -- Write complete binary format specification with byte-level field definitions, worked example, and shell reference appendix ### Phase 2: Core Archiver **Goal**: A working Rust CLI that takes input files and produces a valid encrypted archive @@ -109,16 +130,80 @@ Plans: - [x] 06-01-PLAN.md -- Rust archiver/unpacker obfuscation (XOR header + encrypted TOC + decoy padding + updated tests) - [x] 06-02-PLAN.md -- Kotlin and Shell decoder obfuscation support + cross-validation tests +
+ +### Phase 7: Format Spec Update +**Goal**: FORMAT.md fully documents the v1.1 TOC entry layout with entry type, permission bits, and relative path semantics -- all three decoders can build against it +**Depends on**: Phase 6 (v1.0 complete) +**Requirements**: FMT-09, FMT-10, FMT-11, FMT-12 +**Success Criteria** (what must be TRUE): + 1. FORMAT.md defines the entry type field (1 byte) in TOC entries, distinguishing files from directories + 2. FORMAT.md defines the Unix permissions field (2 bytes, u16 little-endian) in TOC entries with bit layout matching POSIX mode_t lower 12 bits + 3. FORMAT.md specifies that entry names are relative paths using `/` as separator (e.g., `dir/subdir/file.txt`), replacing the previous filename-only convention + 4. FORMAT.md includes an updated worked example showing a directory archive with at least one nested directory, one file, and one empty directory +**Plans**: TBD + +### Phase 8: Rust Directory Archiver +**Goal**: `pack` accepts directories and recursively archives them with full path hierarchy and permissions; `unpack` restores the complete directory tree +**Depends on**: Phase 7 +**Requirements**: DIR-01, DIR-02, DIR-03, DIR-04, DIR-05 +**Success Criteria** (what must be TRUE): + 1. Running `encrypted_archive pack mydir/ -o archive.bin` recursively includes all files and subdirectories, preserving relative paths from the given root + 2. Running `encrypted_archive pack file.txt mydir/ another.apk -o archive.bin` handles mixed file and directory arguments in a single invocation + 3. Empty directories within the input are stored as TOC entries of type "directory" with zero-length data and are recreated on unpack + 4. Running `encrypted_archive unpack archive.bin -o output/` creates the full directory hierarchy and restores Unix mode bits (e.g., a file packed with 0755 is extracted with 0755) + 5. Running `encrypted_archive inspect archive.bin` shows entry type (file/dir), relative paths, and permissions for each TOC entry +**Plans**: TBD + +### Phase 9: Kotlin Decoder Update +**Goal**: Kotlin decoder extracts directory archives created by the updated Rust archiver, preserving hierarchy and permissions on Android +**Depends on**: Phase 8 +**Requirements**: KOT-05, KOT-06, KOT-07 +**Success Criteria** (what must be TRUE): + 1. Kotlin decoder parses the updated TOC format including entry type and permission fields without errors + 2. Kotlin decoder creates the full directory hierarchy (nested directories) before extracting files into them + 3. Kotlin decoder restores permissions on extracted files and directories using File.setReadable/setWritable/setExecutable + 4. Kotlin decoder handles empty directory entries by creating the directory without attempting to decrypt data +**Plans**: TBD + +### Phase 10: Shell Decoder Update +**Goal**: Shell decoder extracts directory archives, creating hierarchy with mkdir -p and restoring permissions with chmod +**Depends on**: Phase 8 +**Requirements**: SHL-04, SHL-05, SHL-06 +**Success Criteria** (what must be TRUE): + 1. Shell decoder parses the updated TOC format including entry type byte and permission field + 2. Shell decoder uses `mkdir -p` to create the full directory hierarchy (including empty directories) before extracting files + 3. Shell decoder applies `chmod` with the octal mode from the TOC to every extracted file and directory + 4. Shell decoder handles entries with relative paths containing `/` separators correctly (no path traversal issues) +**Plans**: TBD + +### Phase 11: Directory Cross-Validation +**Goal**: All three decoders produce identical output for directory archives, verified by automated tests covering edge cases +**Depends on**: Phase 9, Phase 10 +**Requirements**: TST-04, TST-05, TST-06, TST-07 +**Success Criteria** (what must be TRUE): + 1. Round-trip test passes with 3+ levels of nested directories (e.g., `a/b/c/file.txt`) -- Rust pack then Rust unpack produces byte-identical files + 2. Round-trip test passes with empty directories at multiple levels -- they exist in the unpacked output + 3. Mode bits survive the round-trip: a file packed with mode 0755 is extracted with mode 0755; a file with 0644 is extracted with 0644 + 4. Cross-decoder test: archive created by Rust is extracted identically by Kotlin decoder and Shell decoder (same file contents, same directory structure) +**Plans**: TBD + ## Progress **Execution Order:** -Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 +Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 -> 11 +(Phases 9 and 10 can execute in parallel after Phase 8) -| Phase | Plans Complete | Status | Completed | -|-------|----------------|--------|-----------| -| 1. Format Specification | 1/1 | Complete | 2026-02-24 | -| 2. Core Archiver | 2/2 | Complete | 2026-02-24 | -| 3. Round-Trip Verification | 2/2 | Complete | 2026-02-24 | -| 4. Kotlin Decoder | 1/1 | Complete | 2026-02-24 | -| 5. Shell Decoder | 2/2 | Complete | 2026-02-24 | -| 6. Obfuscation Hardening | 2/2 | Complete | 2026-02-24 | +| Phase | Milestone | Plans Complete | Status | Completed | +|-------|-----------|----------------|--------|-----------| +| 1. Format Specification | v1.0 | 1/1 | Complete | 2026-02-24 | +| 2. Core Archiver | v1.0 | 2/2 | Complete | 2026-02-24 | +| 3. Round-Trip Verification | v1.0 | 2/2 | Complete | 2026-02-24 | +| 4. Kotlin Decoder | v1.0 | 1/1 | Complete | 2026-02-25 | +| 5. Shell Decoder | v1.0 | 2/2 | Complete | 2026-02-25 | +| 6. Obfuscation Hardening | v1.0 | 2/2 | Complete | 2026-02-25 | +| 7. Format Spec Update | v1.1 | 0/TBD | Not started | - | +| 8. Rust Directory Archiver | v1.1 | 0/TBD | Not started | - | +| 9. Kotlin Decoder Update | v1.1 | 0/TBD | Not started | - | +| 10. Shell Decoder Update | v1.1 | 0/TBD | Not started | - | +| 11. Directory Cross-Validation | v1.1 | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 775b3d6..7a0590c 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,14 +5,16 @@ See: .planning/PROJECT.md (updated 2026-02-25) **Core value:** Archive impossible to unpack without knowing the format -- standard tools (7z, tar, unzip, binwalk) cannot recognize or extract contents -**Current focus:** Milestone v1.1 Directory Support +**Current focus:** Milestone v1.1 Directory Support -- Phase 7: Format Spec Update ## Current Position -Phase: Not started (defining requirements) -Plan: — -Status: Defining requirements -Last activity: 2026-02-25 — Milestone v1.1 started +Phase: 7 of 11 (Format Spec Update) -- first phase of v1.1 +Plan: Not yet planned +Status: Ready to plan +Last activity: 2026-02-25 -- v1.1 roadmap created (phases 7-11) + +Progress: [##########..........] 55% (10/~18 plans estimated) ## Performance Metrics @@ -28,42 +30,13 @@ Last activity: 2026-02-25 — Milestone v1.1 started Decisions are logged in PROJECT.md Key Decisions table. Recent decisions affecting current work: -- Roadmap: Format spec must precede all implementation (all three decoders build against same spec) -- Roadmap: Obfuscation (XOR headers, encrypted TOC, decoy padding) deferred to Phase 6 after all decoders work without it -- Phase 1: IV stored only in TOC, not duplicated in data blocks (simplifies shell dd extraction) -- Phase 1: Same 32-byte key for AES-256-CBC and HMAC-SHA-256 in v1 (v2 will use HKDF) -- Phase 1: Magic bytes 0x00 0xEA 0x72 0x63 (leading null signals binary) -- Phase 1: HMAC scope = IV (16 bytes) || ciphertext (encrypted_size bytes) -- Phase 2: Used rand::Fill::fill() for IV generation (correct rand 0.9 API) -- Phase 2: Manual binary serialization with to_le_bytes/from_le_bytes (no serde/bincode) -- Phase 2: Filename-only entry names (not full paths) for archive portability -- Phase 2: HMAC failure skips file and continues; SHA-256 mismatch warns but writes -- Phase 2: Flags bit 0 set only when at least one file is actually compressed -- Phase 3: Library crate with pub mod re-exports for all 6 modules -- Phase 3: Unit tests embedded in modules via #[cfg(test)] (not separate files) -- Phase 3: hex-literal v1.1 for compile-time SHA-256 known-value assertions -- Phase 3: Corrected HMAC golden vector (openssl pipe+xxd produced wrong value; verified with file input and Python) -- Phase 3: cargo_bin! macro for non-deprecated assert_cmd binary resolution -- Phase 3: 11MB deterministic pseudo-random data for large file test (wrapping_mul Knuth hash) -- Phase 4: Single-file Kotlin decoder (ArchiveDecoder.kt) for simplicity and Android embeddability -- Phase 4: RandomAccessFile for seeking to data blocks instead of reading entire archive into memory -- Phase 4: HMAC failure skips file, SHA-256 mismatch warns but writes (matching Rust behavior) -- Phase 4: Kotlin signed byte handling with .toByte() for literals > 0x7F, contentEquals() for ByteArray comparison -- Phase 5: POSIX sh (not bash) for maximum busybox compatibility -- Phase 5: xxd/od auto-detection at startup for hex conversion -- Phase 5: Graceful HMAC degradation when openssl lacks -mac support -- Phase 5: Extract ciphertext to temp file before decryption (avoids pipe buffering issues) -- Phase 5: LC_ALL=C for predictable byte handling across locales -- Phase 5: All 6 cross-validation tests passed on first run -- decode.sh was correct as written -- Phase 5: Used sh (not bash) to invoke decode.sh in tests for POSIX compatibility validation -- Phase 6: Always enable all 3 obfuscation features (no --no-obfuscate flag in v1) -- Phase 6: Decoy padding range 64-4096 bytes per file (FORMAT.md allows up to 65535) -- Phase 6: Shared read_archive_metadata() helper for unpack/inspect de-obfuscation -- Phase 6: Two-pass TOC serialization for correct data_offsets with encrypted TOC size -- Phase 6: XOR bootstrapping in Kotlin uses and 0xFF masking on BOTH operands for signed byte safety -- Phase 6: Shell decoder writes de-XORed header to temp file, reuses existing read_hex/read_le_u16/read_le_u32 -- Phase 6: Shell decoder TOC_FILE/TOC_BASE_OFFSET abstraction for encrypted vs plaintext TOC -- Phase 6: Shell decoder HMAC constructs IV from parsed hex via hex_to_bin (not archive position) +- v1.0: IV stored only in TOC, not duplicated in data blocks +- v1.0: Manual binary serialization with to_le_bytes/from_le_bytes (no serde/bincode) +- v1.0: Filename-only entry names -- v1.1 changes this to relative paths with `/` separator +- v1.0: Always enable all 3 obfuscation features (no flags) +- v1.0: Two-pass TOC serialization for correct data_offsets with encrypted TOC size +- v1.1: No backward compatibility with v1.0 archives (format version bump) +- v1.1: Only mode bits (no uid/gid, no timestamps, no symlinks) ### Pending Todos @@ -76,5 +49,5 @@ None. ## Session Continuity Last session: 2026-02-25 -Stopped at: Starting milestone v1.1 — defining requirements +Stopped at: v1.1 roadmap created -- ready to plan Phase 7 Resume file: None