docs(06-obfuscation-hardening): create phase plan

2026-02-25 02:12:16 +03:00
parent 361f9bfb6b
commit 0cd76d7a32
3 changed files with 481 additions and 3 deletions
--- a/.planning/phases/06-obfuscation-hardening/06-02-PLAN.md
+++ b/.planning/phases/06-obfuscation-hardening/06-02-PLAN.md
@@ -0,0 +1,291 @@
+---
+phase: 06-obfuscation-hardening
+plan: 02
+type: execute
+wave: 2
+depends_on:
+  - "06-01"
+files_modified:
+  - kotlin/ArchiveDecoder.kt
+  - shell/decode.sh
+  - kotlin/test_decoder.sh
+  - shell/test_decoder.sh
+autonomous: true
+requirements:
+  - FMT-06
+  - FMT-07
+  - FMT-08
+
+must_haves:
+  truths:
+    - "Kotlin decoder extracts files from obfuscated archives (XOR header + encrypted TOC + decoy padding) producing byte-identical output"
+    - "Shell decoder extracts files from obfuscated archives producing byte-identical output"
+    - "All 6 Kotlin cross-validation tests pass (Rust pack with obfuscation -> Kotlin decode -> SHA-256 match)"
+    - "All 6 Shell cross-validation tests pass (Rust pack with obfuscation -> Shell decode -> SHA-256 match)"
+    - "Both decoders handle XOR bootstrapping (check magic, if mismatch XOR 40 bytes and re-check)"
+    - "Both decoders decrypt encrypted TOC before parsing entries when flags bit 1 is set"
+  artifacts:
+    - path: "kotlin/ArchiveDecoder.kt"
+      provides: "XOR_KEY constant, xorHeader() function, TOC decryption, updated decode() with obfuscation support"
+      contains: "XOR_KEY"
+    - path: "shell/decode.sh"
+      provides: "XOR de-obfuscation loop, TOC decryption via openssl, updated TOC parsing from decrypted temp file"
+      contains: "XOR_KEY_HEX"
+    - path: "kotlin/test_decoder.sh"
+      provides: "Cross-validation tests using obfuscated archives"
+    - path: "shell/test_decoder.sh"
+      provides: "Cross-validation tests using obfuscated archives"
+  key_links:
+    - from: "kotlin/ArchiveDecoder.kt decode()"
+      to: "xorHeader()"
+      via: "XOR bootstrapping on header bytes before parseHeader"
+      pattern: "xorHeader"
+    - from: "kotlin/ArchiveDecoder.kt decode()"
+      to: "decryptAesCbc()"
+      via: "Encrypted TOC bytes decrypted with toc_iv before parseToc"
+      pattern: "decryptAesCbc.*toc"
+    - from: "shell/decode.sh"
+      to: "openssl enc -d"
+      via: "Encrypted TOC extracted to temp file, decrypted, then parsed from decrypted file"
+      pattern: "openssl enc.*toc"
+---
+
+<objective>
+Update Kotlin and Shell decoders to handle obfuscated archives (XOR header + encrypted TOC + decoy padding) and verify all three decoders produce byte-identical output via cross-validation tests.
+
+Purpose: Complete the obfuscation hardening by ensuring all decoder implementations correctly handle the new format. This is the final piece -- the Rust archiver (Plan 01) produces obfuscated archives, and now all decoders must read them.
+
+Output: Updated ArchiveDecoder.kt and decode.sh with obfuscation support. All cross-validation tests pass.
+</objective>
+
+<execution_context>
+@/home/nick/.claude/get-shit-done/workflows/execute-plan.md
+@/home/nick/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/06-obfuscation-hardening/06-RESEARCH.md
+@.planning/phases/06-obfuscation-hardening/06-01-SUMMARY.md
+@docs/FORMAT.md (Sections 9.1-9.3 and Section 10)
+@kotlin/ArchiveDecoder.kt
+@shell/decode.sh
+@kotlin/test_decoder.sh
+@shell/test_decoder.sh
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Update Kotlin decoder with XOR header + encrypted TOC support</name>
+  <files>kotlin/ArchiveDecoder.kt, kotlin/test_decoder.sh</files>
+  <action>
+Update ArchiveDecoder.kt to handle obfuscated archives. Follow the decoder order from FORMAT.md Section 10 and 06-RESEARCH.md patterns.
+
+**Add XOR_KEY constant and xorHeader() function:**
+```kotlin
+val XOR_KEY = byteArrayOf(
+    0xA5.toByte(), 0x3C, 0x96.toByte(), 0x0F,
+    0xE1.toByte(), 0x7B, 0x4D, 0xC8.toByte()
+)
+
+fun xorHeader(buf: ByteArray) {
+    for (i in 0 until minOf(buf.size, 40)) {
+        buf[i] = ((buf[i].toInt() and 0xFF) xor (XOR_KEY[i % 8].toInt() and 0xFF)).toByte()
+    }
+}
+```
+Note: MUST use `and 0xFF` on BOTH operands to avoid Kotlin signed byte issues (06-RESEARCH.md Pitfall 4).
+
+**Update decode() function:**
+
+1. **XOR bootstrapping** (after reading 40-byte headerBytes):
+   - Check if first 4 bytes match MAGIC.
+   - If NO match: call `xorHeader(headerBytes)`.
+   - Then call `parseHeader(headerBytes)` (which validates magic).
+
+2. **TOC decryption** (before parsing TOC entries):
+   - After parsing header, check `header.flags and 0x02 != 0` (bit 1 = TOC encrypted).
+   - If set: seek to `header.tocOffset`, read `header.tocSize.toInt()` bytes, decrypt with `decryptAesCbc(encryptedToc, header.tocIv, KEY)`.
+   - Parse TOC from decrypted bytes: `parseToc(decryptedToc, header.fileCount)`.
+   - If NOT set (backward compat): read raw TOC bytes as before and parse directly.
+
+3. **parseToc() adjustment for encrypted TOC:**
+   - Currently parseToc() asserts `pos == data.size`. After TOC encryption, the decrypted buffer may have PKCS7 padding bytes stripped, so the size should match the sum of entry sizes. Keep the assertion -- it validates that the decrypted plaintext is correct.
+
+4. **Decoy padding** requires NO decoder changes -- decoders already use absolute `data_offset` from TOC entries to seek to each file's ciphertext. Padding is naturally skipped.
+
+**Re-run cross-validation tests** (kotlin/test_decoder.sh). The test script already:
+- Builds the Rust archiver (`cargo build --release`)
+- Creates test files, packs with Rust, decodes with Kotlin, compares SHA-256
+- Now the Rust archiver produces obfuscated archives, so the Kotlin decoder must handle them.
+
+No changes needed to test_decoder.sh unless the test script has hardcoded assumptions about archive format. Read it first and verify.
+  </action>
+  <verify>
+    <automated>cd /home/nick/Projects/Rust/encrypted_archive && bash kotlin/test_decoder.sh 2>&1 | tail -10</automated>
+    <manual>Check that kotlin/ArchiveDecoder.kt contains xorHeader function and TOC decryption logic</manual>
+  </verify>
+  <done>Kotlin decoder handles XOR-obfuscated headers, encrypted TOC, and archives with decoy padding. All 6 cross-validation tests pass (Rust pack -> Kotlin decode -> SHA-256 match).</done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Update Shell decoder with XOR header + encrypted TOC support</name>
+  <files>shell/decode.sh, shell/test_decoder.sh</files>
+  <action>
+Update decode.sh to handle obfuscated archives. This is the most complex change because shell has no native XOR and TOC parsing must switch from reading the archive file to reading a decrypted temp file.
+
+**1. Add XOR de-obfuscation (after reading magic, before parsing header fields):**
+
+Replace the current magic check block (lines ~108-113) with XOR bootstrapping:
+
+```sh
+XOR_KEY_HEX="a53c960fe17b4dc8"
+
+# Read 40-byte header as hex string (80 hex chars)
+raw_header_hex=$(read_hex "$ARCHIVE" 0 40)
+magic_hex=$(printf '%.8s' "$raw_header_hex")
+
+if [ "$magic_hex" != "00ea7263" ]; then
+    # Attempt XOR de-obfuscation
+    header_hex=""
+    byte_idx=0
+    while [ "$byte_idx" -lt 40 ]; do
+        hex_pos=$((byte_idx * 2))
+        # Extract this byte from raw header (2 hex chars)
+        raw_byte=$(printf '%s' "$raw_header_hex" | cut -c$((hex_pos + 1))-$((hex_pos + 2)))
+        # Extract key byte (cyclic)
+        key_pos=$(( (byte_idx % 8) * 2 ))
+        key_byte=$(printf '%s' "$XOR_KEY_HEX" | cut -c$((key_pos + 1))-$((key_pos + 2)))
+        # XOR
+        xored=$(printf '%02x' "$(( 0x$raw_byte ^ 0x$key_byte ))")
+        header_hex="${header_hex}${xored}"
+        byte_idx=$((byte_idx + 1))
+    done
+
+    # Verify magic after XOR
+    magic_hex=$(printf '%.8s' "$header_hex")
+    if [ "$magic_hex" != "00ea7263" ]; then
+        printf 'Invalid archive: bad magic bytes\n' >&2
+        exit 1
+    fi
+else
+    header_hex="$raw_header_hex"
+fi
+
+# Write de-XORed header to temp file for field parsing
+printf '%s' "$header_hex" | xxd -r -p > "$TMPDIR/header.bin"
+```
+
+If xxd is not available (HAS_XXD=0), use an od-based approach to write the binary header from hex. For the `xxd -r -p` replacement when only od is available, use printf with octal escapes or a python one-liner. However, since the existing code already checks for xxd availability and falls back to od for reading, check if `xxd -r -p` is available. If not, use:
+```sh
+# Fallback: write binary from hex using printf with octal
+i=0
+: > "$TMPDIR/header.bin"
+while [ $i -lt 80 ]; do
+    byte_hex=$(printf '%s' "$header_hex" | cut -c$((i + 1))-$((i + 2)))
+    printf "\\$(printf '%03o' "0x$byte_hex")" >> "$TMPDIR/header.bin"
+    i=$((i + 2))
+done
+```
+
+**2. Parse header fields from temp file instead of archive:**
+
+Change all header field reads to use `$TMPDIR/header.bin`:
+```sh
+version_hex=$(read_hex "$TMPDIR/header.bin" 4 1)
+version=$(printf '%d' "0x${version_hex}")
+flags_hex=$(read_hex "$TMPDIR/header.bin" 5 1)
+flags=$(printf '%d' "0x${flags_hex}")
+file_count=$(read_le_u16 "$TMPDIR/header.bin" 6)
+toc_offset=$(read_le_u32 "$TMPDIR/header.bin" 8)
+toc_size=$(read_le_u32 "$TMPDIR/header.bin" 12)
+toc_iv_hex=$(read_hex "$TMPDIR/header.bin" 16 16)
+```
+
+**3. TOC decryption (when flags bit 1 is set):**
+
+After reading header fields, check TOC encryption flag:
+```sh
+toc_encrypted=$(( flags & 2 ))
+
+if [ "$toc_encrypted" -ne 0 ]; then
+    # Extract encrypted TOC to temp file
+    dd if="$ARCHIVE" bs=1 skip="$toc_offset" count="$toc_size" of="$TMPDIR/toc_enc.bin" 2>/dev/null
+
+    # Decrypt TOC
+    openssl enc -d -aes-256-cbc -nosalt \
+        -K "$KEY_HEX" -iv "$toc_iv_hex" \
+        -in "$TMPDIR/toc_enc.bin" -out "$TMPDIR/toc_dec.bin"
+
+    TOC_FILE="$TMPDIR/toc_dec.bin"
+    TOC_BASE_OFFSET=0
+else
+    TOC_FILE="$ARCHIVE"
+    TOC_BASE_OFFSET=$toc_offset
+fi
+```
+
+**4. Update TOC parsing loop to use TOC_FILE and TOC_BASE_OFFSET:**
+
+Change `pos=$toc_offset` to `pos=$TOC_BASE_OFFSET`.
+
+Change ALL references to `"$ARCHIVE"` in the TOC field reads to `"$TOC_FILE"`:
+- `read_le_u16 "$TOC_FILE" "$pos"` instead of `read_le_u16 "$ARCHIVE" "$pos"`
+- `dd if="$TOC_FILE" ...` for filename read
+- `read_le_u32 "$TOC_FILE" "$pos"` for all u32 fields
+- `read_hex "$TOC_FILE" "$pos" N` for IV, HMAC, SHA-256, compression_flag
+
+This is the biggest refactor (06-RESEARCH.md Pitfall 1). Every field read in the TOC loop (lines ~141-183) must change from `$ARCHIVE` to `$TOC_FILE`.
+
+**IMPORTANT HMAC exception:** The HMAC verification reads IV bytes from `$ARCHIVE` at `$iv_toc_pos` (the absolute archive position). After TOC encryption, IV is stored in the TOC entries (which are now in the decrypted file). The HMAC input is still IV || ciphertext from the archive data block. So for HMAC computation:
+- IV comes from the TOC entry (already parsed as `$iv_hex`).
+- Ciphertext comes from `$ARCHIVE` at `$data_offset`.
+- The HMAC input must be constructed from the parsed iv_hex and the raw ciphertext from the archive.
+
+Change the HMAC verification to construct IV from the parsed hex variable instead of reading from the archive at the TOC position:
+```sh
+computed_hmac=$( {
+    printf '%s' "$iv_hex" | xxd -r -p
+    cat "$TMPDIR/ct.bin"
+} | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${KEY_HEX}" -hex 2>/dev/null | awk '{print $NF}' )
+```
+With od fallback for `xxd -r -p` if needed.
+
+**5. No changes needed for decoy padding:** The decoder uses `data_offset` from TOC entries (absolute offsets), so padding between blocks is naturally skipped.
+
+**Re-run cross-validation tests** (shell/test_decoder.sh). No changes should be needed to the test script since it already tests Rust pack -> Shell decode -> SHA-256 comparison.
+  </action>
+  <verify>
+    <automated>cd /home/nick/Projects/Rust/encrypted_archive && sh shell/test_decoder.sh 2>&1 | tail -10</automated>
+    <manual>Check that decode.sh has XOR_KEY_HEX variable, XOR loop, and TOC decryption section</manual>
+  </verify>
+  <done>Shell decoder handles XOR-obfuscated headers, encrypted TOC, and archives with decoy padding. All 6 cross-validation tests pass (Rust pack -> Shell decode -> SHA-256 match). HMAC verification works with IV from parsed TOC entry.</done>
+</task>
+
+</tasks>
+
+<verification>
+1. `bash kotlin/test_decoder.sh` -- all 6 Kotlin cross-validation tests pass
+2. `sh shell/test_decoder.sh` -- all 6 Shell cross-validation tests pass
+3. Kotlin decoder correctly applies XOR bootstrapping + TOC decryption
+4. Shell decoder correctly applies XOR bootstrapping + TOC decryption from temp file
+5. Both decoders produce byte-identical output to Rust unpack on the same obfuscated archive
+6. `strings obfuscated_archive.bin | grep -i "hello\|test\|file"` returns nothing (no plaintext metadata leaks)
+</verification>
+
+<success_criteria>
+- All three decoders (Rust, Kotlin, Shell) produce byte-identical output from obfuscated archives
+- 12 cross-validation tests pass (6 Kotlin + 6 Shell)
+- Phase 6 success criteria from ROADMAP.md are fully met:
+  1. File table encrypted with its own IV -- hex dump reveals no plaintext metadata
+  2. Headers XOR-obfuscated -- no recognizable structure in first 256 bytes
+  3. Random decoy padding between blocks -- file boundaries not detectable
+  4. All three decoders still produce byte-identical output
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/06-obfuscation-hardening/06-02-SUMMARY.md`
+</output>