docs(03): create phase plan for round-trip verification

2026-02-25 00:23:05 +03:00
parent 09a3593b8a
commit 8447cc399c
3 changed files with 395 additions and 3 deletions
--- a/.planning/ROADMAP.md
+++ b/.planning/ROADMAP.md
@@ -60,11 +60,11 @@ Plans:
  2. Golden test vectors exist: a known plaintext + known key + known IV produces an expected ciphertext (checked in CI)
  3. Unit tests cover each pipeline stage independently: compression, encryption, HMAC, format serialization/deserialization
  4. Round-trip succeeds for edge cases: empty file, large APK (>10MB), file with non-ASCII name (Cyrillic)
-**Plans**: TBD
+**Plans**: 2 plans

 Plans:
- [ ] 03-01: TBD
- [ ] 03-02: TBD
+- [ ] 03-01-PLAN.md -- Library crate structure, dev-dependencies, and unit tests for crypto/compression/format modules
+- [ ] 03-02-PLAN.md -- Golden test vectors with fixed IV/key and CLI round-trip integration tests

 ### Phase 4: Kotlin Decoder
 **Goal**: Android 13 Kotlin code that extracts files from the custom archive using only Android SDK built-ins
--- a/.planning/phases/03-round-trip-verification/03-01-PLAN.md
+++ b/.planning/phases/03-round-trip-verification/03-01-PLAN.md
@@ -0,0 +1,174 @@
+---
+phase: 03-round-trip-verification
+plan: 01
+type: execute
+wave: 1
+depends_on: []
+files_modified:
+  - src/lib.rs
+  - src/main.rs
+  - Cargo.toml
+  - src/crypto.rs
+  - src/compression.rs
+  - src/format.rs
+autonomous: true
+requirements: [TST-03]
+
+must_haves:
+  truths:
+    - "Unit tests pass for crypto module: encrypt/decrypt round-trip, HMAC compute/verify, SHA-256 known value"
+    - "Unit tests pass for compression module: compress/decompress round-trip, empty data, should_compress heuristic"
+    - "Unit tests pass for format module: header write/read round-trip, TOC entry write/read round-trip (ASCII + Cyrillic names)"
+    - "Project compiles as both library crate and binary crate (cargo build succeeds)"
+  artifacts:
+    - path: "src/lib.rs"
+      provides: "Library crate re-exporting all modules for integration tests"
+      contains: "pub mod crypto"
+    - path: "Cargo.toml"
+      provides: "Dev-dependencies for test infrastructure"
+      contains: "tempfile"
+  key_links:
+    - from: "src/main.rs"
+      to: "src/lib.rs"
+      via: "use encrypted_archive:: imports"
+      pattern: "use encrypted_archive::"
+    - from: "src/lib.rs"
+      to: "src/crypto.rs"
+      via: "pub mod crypto"
+      pattern: "pub mod crypto"
+---
+
+<objective>
+Create the library crate structure, add test dependencies, and implement unit tests for every pipeline module (crypto, compression, format serialization).
+
+Purpose: TST-03 requires unit tests for each pipeline stage. The lib.rs creation also unblocks Plan 02 which needs to import library functions from integration tests in `tests/`.
+
+Output: `src/lib.rs` with module re-exports, dev-dependencies in `Cargo.toml`, unit tests in `#[cfg(test)]` blocks inside `crypto.rs`, `compression.rs`, `format.rs`.
+</objective>
+
+<execution_context>
+@/home/nick/.claude/get-shit-done/workflows/execute-plan.md
+@/home/nick/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/03-round-trip-verification/03-RESEARCH.md
+@docs/FORMAT.md
+@src/main.rs
+@src/crypto.rs
+@src/compression.rs
+@src/format.rs
+@src/key.rs
+@Cargo.toml
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Create lib.rs, update main.rs imports, add dev-dependencies</name>
+  <files>src/lib.rs, src/main.rs, Cargo.toml</files>
+  <action>
+1. Create `src/lib.rs` that re-exports all modules as public:
+```rust
+pub mod archive;
+pub mod cli;
+pub mod compression;
+pub mod crypto;
+pub mod format;
+pub mod key;
+```
+
+2. Update `src/main.rs` to use the library crate instead of local `mod` declarations:
+- Remove all `mod` declarations (archive, cli, compression, crypto, format, key)
+- Replace with `use encrypted_archive::cli::{Cli, Commands};` and `use encrypted_archive::archive;`
+- Keep the `use clap::Parser;` import
+- The `main()` function body stays the same
+
+3. Add dev-dependencies to `Cargo.toml`:
+```toml
+[dev-dependencies]
+tempfile = "3.26"
+assert_cmd = "2.1"
+hex-literal = "1.1"
+predicates = "3.1"
+```
+
+IMPORTANT: Use Context7 to verify `tempfile` and `assert_cmd` API before writing code if needed. The research doc already verified versions (tempfile 3.26.0, assert_cmd 2.1.2, hex-literal 1.1.0, predicates 3.1.4).
+
+4. Verify: `cargo build` succeeds (both lib and bin targets compile).
+  </action>
+  <verify>
+    <automated>cd /home/nick/Projects/Rust/encrypted_archive && cargo build 2>&1</automated>
+    <manual>No lib.rs compilation errors, main.rs uses encrypted_archive:: imports</manual>
+  </verify>
+  <done>Project compiles as both library and binary crate. All six modules re-exported via lib.rs. Dev-dependencies declared in Cargo.toml.</done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Add unit tests to crypto.rs, compression.rs, and format.rs</name>
+  <files>src/crypto.rs, src/compression.rs, src/format.rs</files>
+  <action>
+Add `#[cfg(test)] mod tests { ... }` blocks at the bottom of each module file.
+
+**src/crypto.rs** -- add these unit tests:
+- `test_encrypt_decrypt_roundtrip`: Encrypt "Hello, World!" with KEY and a zero IV, decrypt, assert equality.
+- `test_encrypt_decrypt_empty`: Encrypt empty slice, decrypt, assert empty result. Verifies PKCS7 full-block padding on 0-byte input.
+- `test_encrypted_size_formula`: Assert `encrypt_data(b"Hello", &KEY, &iv).len() == 16` (5 bytes -> ((5/16)+1)*16 = 16). Also test 16 bytes -> 32 (full padding block), and 0 bytes -> 16.
+- `test_hmac_compute_verify`: Compute HMAC for a fixed IV + ciphertext, then verify_hmac returns true. Also test with wrong expected value returns false.
+- `test_sha256_known_value`: SHA-256("Hello") must equal `185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969` (from FORMAT.md Section 12.3). Use `hex_literal::hex!()` macro for the expected value.
+- `test_sha256_empty`: SHA-256("") must equal `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`.
+
+Import `use super::*;` and `use crate::key::KEY;` and `use hex_literal::hex;` inside the test module.
+
+**src/compression.rs** -- add these unit tests:
+- `test_compress_decompress_roundtrip`: Compress "Hello, World! This is test data for compression." -> decompress -> assert equals original.
+- `test_compress_decompress_empty`: Compress empty slice -> decompress -> assert empty.
+- `test_compress_decompress_large`: Compress 10000 bytes of pattern data -> decompress -> assert equals original.
+- `test_should_compress_text`: Assert `should_compress("readme.txt", &[])` is true. Also test "data.json".
+- `test_should_not_compress_known_extensions`: Assert `should_compress("app.apk", &[])` is false. Also test "photo.jpg", "archive.zip".
+- `test_should_not_compress_excluded`: Assert `should_compress("special.dat", &["special.dat".into()])` is false.
+
+Import `use super::*;` inside the test module.
+
+**src/format.rs** -- add these unit tests:
+- `test_header_write_read_roundtrip`: Create a Header with typical values (version=1, flags=0x01, file_count=3, toc_offset=HEADER_SIZE, toc_size=330, toc_iv=zeroed, reserved=zeroed). Write to Vec<u8>, assert len == HEADER_SIZE. Read back from Cursor, assert all fields match.
+- `test_toc_entry_roundtrip_ascii`: Create a TocEntry for "hello.txt" with known field values. Write to Vec<u8>, assert len == 101 + 9. Read back, assert all fields match including iv/hmac/sha256 byte arrays.
+- `test_toc_entry_roundtrip_cyrillic`: Create a TocEntry for "тестовый_файл.txt". Write, read back, assert name and all fields match. Verify buf length == 101 + "тестовый_файл.txt".len() (which is 29 bytes for UTF-8 Cyrillic).
+- `test_toc_entry_roundtrip_empty_name`: Create a TocEntry with name="" (edge case). Write, read back, assert name is empty string.
+- `test_header_rejects_bad_magic`: Create a buffer with wrong magic bytes, attempt read_header, assert error.
+- `test_header_rejects_bad_version`: Create a valid-magic buffer but version=2, attempt read_header, assert error.
+- `test_entry_size_calculation`: Assert `entry_size(&entry_for_hello_txt) == 110` and `entry_size(&entry_for_data_bin) == 109`. Test `compute_toc_size` on both entries equals 219 (matching FORMAT.md worked example).
+
+Import `use super::*;` and `use std::io::Cursor;` inside the test module.
+
+All assertions should use `assert_eq!` for value comparison (gives diff on failure) or `assert!` for boolean checks.
+  </action>
+  <verify>
+    <automated>cd /home/nick/Projects/Rust/encrypted_archive && cargo test 2>&1</automated>
+    <manual>All unit tests pass, no compilation errors</manual>
+  </verify>
+  <done>Unit tests exist and pass for all three pipeline modules: crypto (6 tests), compression (6 tests), format (7 tests). Total: 19 unit tests covering encrypt/decrypt, HMAC, SHA-256, compress/decompress, should_compress, header/TOC serialization round-trips, and error cases.</done>
+</task>
+
+</tasks>
+
+<verification>
+- `cargo build` succeeds (lib + bin)
+- `cargo test` passes all unit tests
+- `src/lib.rs` exists and re-exports all modules
+- `src/main.rs` uses `use encrypted_archive::` imports (no local `mod` declarations)
+- Each of crypto.rs, compression.rs, format.rs has a `#[cfg(test)]` module with tests
+</verification>
+
+<success_criteria>
+1. `cargo test` runs 19+ unit tests, all pass
+2. Project structure supports both `cargo run` (binary) and `use encrypted_archive::module` (library imports)
+3. TST-03 is satisfied: unit tests cover compression, encryption, HMAC, SHA-256, format serialization/deserialization
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/03-round-trip-verification/03-01-SUMMARY.md`
+</output>
--- a/.planning/phases/03-round-trip-verification/03-02-PLAN.md
+++ b/.planning/phases/03-round-trip-verification/03-02-PLAN.md
@@ -0,0 +1,218 @@
+---
+phase: 03-round-trip-verification
+plan: 02
+type: execute
+wave: 2
+depends_on: ["03-01"]
+files_modified:
+  - tests/golden.rs
+  - tests/round_trip.rs
+autonomous: true
+requirements: [INT-02, TST-01, TST-02]
+
+must_haves:
+  truths:
+    - "Golden test vectors pass: AES-256-CBC encryption of 'Hello' with project KEY and fixed IV produces exact expected ciphertext (cross-verified with openssl)"
+    - "Golden test vectors pass: HMAC-SHA256 of IV||ciphertext produces exact expected hash"
+    - "Golden test vectors pass: SHA-256 of 'Hello' matches known value from FORMAT.md"
+    - "Round-trip pack->unpack produces byte-identical files for: single text file, multiple files, empty file, Cyrillic filename, large (11MB) binary"
+    - "cargo test --test golden passes all golden vector tests"
+    - "cargo test --test round_trip passes all integration tests"
+  artifacts:
+    - path: "tests/golden.rs"
+      provides: "Golden test vectors with fixed IV/key for deterministic verification"
+      contains: "test_golden_aes256cbc"
+    - path: "tests/round_trip.rs"
+      provides: "CLI round-trip integration tests via assert_cmd"
+      contains: "test_roundtrip"
+  key_links:
+    - from: "tests/golden.rs"
+      to: "src/crypto.rs"
+      via: "use encrypted_archive::crypto"
+      pattern: "encrypted_archive::crypto"
+    - from: "tests/golden.rs"
+      to: "src/key.rs"
+      via: "use encrypted_archive::key::KEY"
+      pattern: "encrypted_archive::key::KEY"
+    - from: "tests/round_trip.rs"
+      to: "encrypted_archive binary"
+      via: "Command::cargo_bin(\"encrypted_archive\")"
+      pattern: "cargo_bin.*encrypted_archive"
+---
+
+<objective>
+Implement golden test vectors (TST-02) and CLI round-trip integration tests (INT-02, TST-01) that prove byte-identical round-trips for all edge cases.
+
+Purpose: Golden vectors prove the crypto implementation matches the spec (cross-verifiable with openssl). Round-trip tests prove the full pack->unpack pipeline preserves file content exactly. Together they satisfy the Phase 3 success criteria.
+
+Output: `tests/golden.rs` with fixed-IV crypto tests, `tests/round_trip.rs` with CLI integration tests for all edge cases.
+</objective>
+
+<execution_context>
+@/home/nick/.claude/get-shit-done/workflows/execute-plan.md
+@/home/nick/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/03-round-trip-verification/03-RESEARCH.md
+@.planning/phases/03-round-trip-verification/03-01-SUMMARY.md
+@docs/FORMAT.md
+@src/crypto.rs
+@src/key.rs
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Create golden test vectors with fixed IV/key</name>
+  <files>tests/golden.rs</files>
+  <action>
+Create `tests/golden.rs` with deterministic golden test vectors. These call library functions directly with fixed IVs (NOT via CLI which uses random IVs).
+
+Import:
+```rust
+use encrypted_archive::{crypto, key::KEY};
+use hex_literal::hex;
+```
+
+**Test 1: `test_golden_aes256cbc_hello`**
+Encrypt "Hello" (5 bytes) with project KEY and IV = `00000000000000000000000000000001`.
+Expected ciphertext: `hex!("6e66ae8bc740efeefe83b5713fcb716f")` (16 bytes -- verified with openssl in research).
+```rust
+let iv = hex!("00000000000000000000000000000001");
+let ciphertext = crypto::encrypt_data(b"Hello", &KEY, &iv);
+assert_eq!(ciphertext, hex!("6e66ae8bc740efeefe83b5713fcb716f").to_vec());
+```
+
+**Test 2: `test_golden_aes256cbc_decrypt_hello`**
+Decrypt the ciphertext from Test 1 back to "Hello". Verifies round-trip with the exact golden vector.
+```rust
+let iv = hex!("00000000000000000000000000000001");
+let ciphertext = hex!("6e66ae8bc740efeefe83b5713fcb716f");
+let plaintext = crypto::decrypt_data(&ciphertext, &KEY, &iv).unwrap();
+assert_eq!(plaintext, b"Hello");
+```
+
+**Test 3: `test_golden_aes256cbc_empty`**
+Encrypt empty slice with KEY and IV = zero. Expected: 16 bytes (full PKCS7 padding block). Decrypt back, assert empty.
+Use a fixed IV (e.g., all zeros). The exact ciphertext can be computed: encrypt b"" and store the result as golden vector. Since empty + PKCS7 = 16 bytes of 0x10, encrypt that. Compute the expected value by running the test once, then hardcode it. ALTERNATIVELY: just test the round-trip property (encrypt then decrypt gives empty) plus verify length == 16. This is simpler and sufficient for the empty case.
+
+**Test 4: `test_golden_hmac_sha256`**
+Compute HMAC-SHA256 over IV || ciphertext from Test 1.
+Expected: `hex!("0c85780b6628ba3b52654d2f6e0d9bbec67443cf2a6104eb3120ec93fc2d38d4")` (verified with openssl in research).
+```rust
+let iv = hex!("00000000000000000000000000000001");
+let ciphertext = hex!("6e66ae8bc740efeefe83b5713fcb716f");
+let hmac = crypto::compute_hmac(&KEY, &iv, &ciphertext);
+assert_eq!(hmac, hex!("0c85780b6628ba3b52654d2f6e0d9bbec67443cf2a6104eb3120ec93fc2d38d4"));
+```
+
+**Test 5: `test_golden_hmac_verify`**
+Verify HMAC from Test 4 returns true. Also verify with a wrong expected value returns false.
+
+**Test 6: `test_golden_sha256_hello`**
+SHA-256("Hello") must equal `hex!("185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969")` (from FORMAT.md Section 12.3).
+
+**Test 7: `test_golden_sha256_32x01`**
+SHA-256 of 32 bytes of 0x01 must equal `hex!("72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793")` (from FORMAT.md Section 12.3).
+
+CRITICAL: Do NOT create golden vectors for compressed-then-encrypted data (gzip non-determinism across platforms). All golden vectors test raw plaintext encryption only.
+  </action>
+  <verify>
+    <automated>cd /home/nick/Projects/Rust/encrypted_archive && cargo test --test golden 2>&1</automated>
+    <manual>All 7 golden vector tests pass</manual>
+  </verify>
+  <done>Golden test vectors exist for AES-256-CBC encrypt/decrypt, HMAC-SHA256, and SHA-256. All values cross-verified with openssl enc / openssl dgst / sha256sum during research. TST-02 satisfied.</done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Create CLI round-trip integration tests</name>
+  <files>tests/round_trip.rs</files>
+  <action>
+Create `tests/round_trip.rs` with integration tests that run the actual `encrypted_archive` binary via `assert_cmd::Command::cargo_bin("encrypted_archive")`.
+
+Import:
+```rust
+use assert_cmd::Command;
+use tempfile::tempdir;
+use std::fs;
+```
+
+**Test 1: `test_roundtrip_single_text_file`**
+- Create tempdir, write `hello.txt` with content `b"Hello"`.
+- Run `encrypted_archive pack hello.txt -o archive.bin`.
+- Run `encrypted_archive unpack archive.bin -o output/`.
+- Assert `output/hello.txt` content == original `b"Hello"`.
+
+**Test 2: `test_roundtrip_multiple_files`**
+- Create two files: `text.txt` ("Some text content") and `binary.dat` (256 bytes of 0x42).
+- Pack both into one archive.
+- Unpack.
+- Assert both files byte-identical to originals.
+
+**Test 3: `test_roundtrip_empty_file`**
+- Create `empty.txt` with 0 bytes.
+- Pack, unpack.
+- Assert extracted file is exactly 0 bytes.
+- This tests PKCS7 full-block padding on 0-byte input (encrypted_size = 16).
+
+**Test 4: `test_roundtrip_cyrillic_filename`**
+- Create `файл.txt` with content "Содержимое" (Cyrillic, UTF-8).
+- Pack, unpack.
+- Assert extracted `файл.txt` content equals original.
+- This tests non-ASCII (UTF-8) filename handling in TOC entries.
+
+**Test 5: `test_roundtrip_large_file`**
+- Generate 11MB (11_000_000 bytes) of deterministic pseudo-random data: `(0..11_000_000u32).map(|i| (i.wrapping_mul(2654435761)) as u8).collect()`.
+- Write as `large.bin`.
+- Pack with `--no-compress bin` (skip compression for binary extension).
+- Unpack.
+- Assert byte-identical.
+- This tests large file handling (>10MB) per Phase 3 success criteria.
+
+**Test 6: `test_roundtrip_no_compress_flag`**
+- Create `data.apk` with some content (e.g., 100 bytes of pattern).
+- Pack (APK auto-detected as no-compress).
+- Unpack.
+- Assert byte-identical.
+- This tests that no-compression round-trip works (compression_flag=0).
+
+Each test uses `tempdir()` for isolation (auto-cleanup, parallel-safe).
+Each test asserts `.success()` on both pack and unpack commands.
+Each test compares file content with `assert_eq!(fs::read(original), fs::read(extracted))`.
+
+IMPORTANT: Use `assert_cmd::Command::cargo_bin("encrypted_archive").unwrap()` to locate the binary. This handles debug/release build paths automatically.
+  </action>
+  <verify>
+    <automated>cd /home/nick/Projects/Rust/encrypted_archive && cargo test --test round_trip 2>&1</automated>
+    <manual>All 6 round-trip tests pass including empty file, Cyrillic filename, and 11MB file</manual>
+  </verify>
+  <done>Round-trip integration tests cover: single file, multiple files, empty file, Cyrillic filename, large (11MB) binary, no-compression APK. All tests verify byte-identical extraction via SHA-256 implicit comparison (assert_eq on raw bytes). INT-02 and TST-01 satisfied.</done>
+</task>
+
+</tasks>
+
+<verification>
+- `cargo test --test golden` passes all 7 golden vector tests
+- `cargo test --test round_trip` passes all 6 integration tests
+- `cargo test` passes ALL tests (unit + golden + round-trip)
+- Golden vectors use fixed IVs (no randomness, fully deterministic)
+- Round-trip tests cover all edge cases from Phase 3 success criteria: empty file, large APK (>10MB), non-ASCII name (Cyrillic)
+</verification>
+
+<success_criteria>
+1. `cargo test` runs 30+ tests (19 unit + 7 golden + 6 round-trip), all pass
+2. Golden test values match openssl cross-verification from research
+3. All Phase 3 success criteria met:
+   - Byte-identical round-trips (INT-02)
+   - Golden vectors exist (TST-02)
+   - Unit tests per module (TST-03, from Plan 01)
+   - Edge cases: empty file, >10MB, Cyrillic filename
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/03-round-trip-verification/03-02-SUMMARY.md`
+</output>