--- phase: 05-shell-decoder plan: 02 type: execute wave: 2 depends_on: - "05-01" files_modified: - shell/test_decoder.sh autonomous: true requirements: - SHL-01 - SHL-02 - SHL-03 must_haves: truths: - "Shell decoder extracts single text file byte-identical to original" - "Shell decoder extracts multiple files (text + binary) byte-identical to originals" - "Shell decoder extracts files with Cyrillic UTF-8 filenames correctly" - "Shell decoder handles no-compress mode (raw encrypted, not gzip-compressed)" - "Shell decoder handles empty (0-byte) files" - "Shell decoder handles large files (100 KB+)" artifacts: - path: "shell/test_decoder.sh" provides: "Cross-validation test script: Rust pack -> Shell decode -> SHA-256 verify" min_lines: 150 contains: "sha256sum" key_links: - from: "shell/test_decoder.sh" to: "shell/decode.sh" via: "Invokes decode.sh to decode Rust-created archives" pattern: "decode\\.sh" - from: "shell/test_decoder.sh" to: "target/release/encrypted_archive" via: "Uses Rust archiver to create test archives" pattern: "encrypted_archive.*pack" --- Create cross-validation test script that verifies the shell decoder produces byte-identical output to originals for all edge cases. Purpose: Prove the shell decoder correctly implements the archive format by testing against Rust-created archives (same pattern as kotlin/test_decoder.sh). Output: `shell/test_decoder.sh` -- a test script that creates archives with the Rust CLI, decodes with decode.sh, and verifies byte-identical output via SHA-256 comparison. @/home/nick/.claude/get-shit-done/workflows/execute-plan.md @/home/nick/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/05-shell-decoder/05-01-SUMMARY.md @kotlin/test_decoder.sh @shell/decode.sh Task 1: Create shell/test_decoder.sh cross-validation test script shell/test_decoder.sh Create `shell/test_decoder.sh` modeled after `kotlin/test_decoder.sh` but adapted for the shell decoder. This script uses bash (it runs on the development machine, not the target device) but invokes `decode.sh` with `sh` to ensure POSIX compatibility. **Script structure (follow kotlin/test_decoder.sh pattern):** 1. **Header:** - `#!/usr/bin/env bash` - `set -euo pipefail` - Comment: Cross-validation test script for Shell decoder 2. **Variables and helpers (copy pattern from kotlin/test_decoder.sh):** - SCRIPT_DIR, PROJECT_DIR, TMPDIR, PASS_COUNT, FAIL_COUNT, TOTAL_COUNT - Colors (GREEN, RED, YELLOW, BOLD, NC) with terminal detection - `cleanup()` function with trap - `pass()`, `fail()`, `sha256_of()`, `verify_file()` helper functions (identical to kotlin version) 3. **Prerequisites check:** - Check for `cargo` (to build Rust archiver) - Check for `openssl` (needed by decode.sh) - Check for `sh` (always available, but verify) - Print versions: cargo, openssl, sh 4. **Build Rust archiver:** ```bash (cd "$PROJECT_DIR" && cargo build --release -q) ARCHIVER="$PROJECT_DIR/target/release/encrypted_archive" ``` 5. **Decoder path:** ```bash DECODER="$SCRIPT_DIR/decode.sh" ``` Verify it exists and is executable. 6. **Test cases (6 tests matching kotlin/test_decoder.sh):** **Test 1: Single text file** - Create `hello.txt` with content "Hello, World!" (no newline: `echo -n`) - Pack with Rust archiver - Decode with: `sh "$DECODER" "$TMPDIR/test1.archive" "$TMPDIR/output1/"` - verify_file hello.txt **Test 2: Multiple files (text + binary)** - Create text.txt with 3 lines - Create binary.bin with 10 KB random data (`dd if=/dev/urandom bs=1024 count=10`) - Pack both files - Decode with sh - verify_file text.txt, binary.bin **Test 3: No-compress mode** - Create fake.dat with 2.5 KB random data - Pack with `--no-compress "fake.dat"` - Decode with sh - verify_file fake.dat **Test 4: Empty file** - Create empty.txt with `touch` - Pack with Rust archiver - Decode with sh - Verify output file exists and is 0 bytes (same logic as kotlin test) **Test 5: Large file (100 KB)** - Create large.bin with 100 KB random data - Pack with Rust archiver - Decode with sh - verify_file large.bin **Test 6: Cyrillic UTF-8 filename (SHL-03 validation)** - Create a file with a Cyrillic name: `echo -n "Тестовое содержимое" > "$TMPDIR/файл.txt"` - Pack with Rust archiver - Decode with sh - verify_file that the Cyrillic-named file exists and matches SHA-256 - This test specifically validates SHL-03 7. **Summary:** - Print results: "N passed, M failed out of T tests" - Exit 1 if any failures, exit 0 if all pass **Key difference from kotlin test:** Instead of `java -jar "$JAR"`, use `sh "$DECODER"` to invoke the shell decoder. This ensures the decoder is tested under POSIX sh, not bash. Make the script executable: `chmod +x shell/test_decoder.sh` cd /home/nick/Projects/Rust/encrypted_archive && test -f shell/test_decoder.sh && test -x shell/test_decoder.sh && bash -n shell/test_decoder.sh && echo "SYNTAX OK" && grep -q 'decode.sh' shell/test_decoder.sh && grep -q 'verify_file' shell/test_decoder.sh && echo "STRUCTURE OK" Run `bash shell/test_decoder.sh` and verify all 6 tests pass shell/test_decoder.sh exists, is executable, passes bash -n, and contains all 6 test cases including Cyrillic filename test Task 2: Run cross-validation tests and fix any decode.sh issues shell/decode.sh, shell/test_decoder.sh Run the cross-validation test script and fix any issues discovered: ```bash cd /home/nick/Projects/Rust/encrypted_archive bash shell/test_decoder.sh ``` **Expected:** All 6 tests pass (PASS for each test case). **If tests fail:** - Read the error output carefully - Common issues to look for: 1. **Hex case mismatch:** HMAC or SHA-256 comparison fails because of uppercase vs lowercase hex. Fix: normalize with `tr 'A-F' 'a-f'` 2. **dd offset errors:** Wrong offset calculation in TOC parsing. Fix: verify field order matches FORMAT.md Section 5 exactly 3. **openssl pipe issues:** Decryption produces garbage. Fix: extract ciphertext to temp file first, then decrypt from file (not pipe) 4. **Empty file handling:** gunzip fails on empty input. Fix: check original_size=0 before decompression 5. **Cyrillic filename:** garbled characters. Fix: ensure LC_ALL=C and no text processing on filename bytes 6. **HMAC scope error:** Wrong bytes fed to HMAC. Fix: HMAC = IV (from archive file, 16 bytes) || ciphertext (encrypted_size bytes) Fix decode.sh until all tests pass. Do NOT modify the test script to make tests pass -- fix the decoder. **After all tests pass**, run one more verification: ```bash # Verify Rust round-trip still works cd /home/nick/Projects/Rust/encrypted_archive && cargo test --release 2>&1 | tail -5 ``` cd /home/nick/Projects/Rust/encrypted_archive && bash shell/test_decoder.sh 2>&1 | tail -20 && echo "---" && cargo test --release 2>&1 | tail -3 All 6 shell decoder tests pass, including Cyrillic filename. Rust tests still pass. All 6 cross-validation tests pass (single file, multiple files, no-compress, empty file, large file, Cyrillic filename). Existing Rust tests still pass. 1. `bash shell/test_decoder.sh` runs all 6 tests and reports "ALL TESTS PASSED" 2. Test 6 specifically validates Cyrillic filename extraction (SHL-03) 3. Test 3 validates no-compress mode with openssl raw decryption (SHL-02) 4. All tests use Rust archiver to pack and shell decoder to unpack (SHL-01) 5. `cargo test --release` still passes (no regressions) - shell/test_decoder.sh exists and is executable - All 6 test cases pass: single file, multiple files, no-compress, empty file, large file, Cyrillic filename - Shell decoder produces byte-identical output verified by SHA-256 comparison - Existing Rust test suite still passes After completion, create `.planning/phases/05-shell-decoder/05-02-SUMMARY.md`