diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 083da3c..21d59f7 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -170,7 +170,10 @@ Plans: 2. Kotlin decoder creates the full directory hierarchy (nested directories) before extracting files into them 3. Kotlin decoder restores permissions on extracted files and directories using File.setReadable/setWritable/setExecutable 4. Kotlin decoder handles empty directory entries by creating the directory without attempting to decrypt data -**Plans**: TBD +**Plans**: 1 plan + +Plans: +- [ ] 09-01-PLAN.md -- Update ArchiveDecoder.kt for v1.1 TOC (entry_type, permissions, directory support) and test_decoder.sh with directory test cases ### Phase 10: Shell Decoder Update **Goal**: Shell decoder extracts directory archives, creating hierarchy with mkdir -p and restoring permissions with chmod @@ -210,6 +213,6 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 | 6. Obfuscation Hardening | v1.0 | 2/2 | Complete | 2026-02-25 | | 7. Format Spec Update | v1.1 | 1/1 | Complete | 2026-02-26 | | 8. Rust Directory Archiver | v1.1 | 1/1 | Complete | 2026-02-26 | -| 9. Kotlin Decoder Update | v1.1 | 0/TBD | Not started | - | +| 9. Kotlin Decoder Update | v1.1 | 0/1 | Not started | - | | 10. Shell Decoder Update | v1.1 | 0/TBD | Not started | - | | 11. Directory Cross-Validation | v1.1 | 0/TBD | Not started | - | diff --git a/.planning/phases/09-kotlin-decoder-update/09-01-PLAN.md b/.planning/phases/09-kotlin-decoder-update/09-01-PLAN.md new file mode 100644 index 0000000..6cdc6b4 --- /dev/null +++ b/.planning/phases/09-kotlin-decoder-update/09-01-PLAN.md @@ -0,0 +1,300 @@ +--- +phase: 09-kotlin-decoder-update +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - kotlin/ArchiveDecoder.kt + - kotlin/test_decoder.sh +autonomous: true +requirements: [KOT-05, KOT-06, KOT-07] + +must_haves: + truths: + - "Kotlin decoder parses v1.1 TOC entries with entry_type and permissions fields without errors" + - "Kotlin decoder creates full directory hierarchy (nested directories) before extracting files into them" + - "Kotlin decoder handles empty directory entries by creating the directory without attempting to decrypt data" + - "Kotlin decoder restores permissions on extracted files and directories" + - "Cross-validation test passes for directory archives (Rust pack -> Kotlin decode -> SHA-256 match)" + artifacts: + - path: "kotlin/ArchiveDecoder.kt" + provides: "v1.1-compatible Kotlin decoder with directory support and permission restoration" + contains: "entryType" + - path: "kotlin/test_decoder.sh" + provides: "Cross-validation test script with directory test cases" + contains: "directory" + key_links: + - from: "kotlin/ArchiveDecoder.kt" + to: "src/format.rs" + via: "v1.1 TOC binary layout (entry_type after name, permissions after entry_type)" + pattern: "entry_type.*permissions" + - from: "kotlin/test_decoder.sh" + to: "target/release/encrypted_archive" + via: "Rust pack with directories -> Kotlin decode -> SHA-256 verify" + pattern: "pack.*-o.*archive" +--- + + +Update the Kotlin archive decoder to handle v1.1 format with directory support: parse new TOC fields (entry_type, permissions), create directory hierarchies on extraction, handle empty directories without decryption, and restore Unix permissions. + +Purpose: Enable Kotlin/Android decoder to extract directory archives produced by the updated Rust archiver (Phase 8), completing KOT-05/KOT-06/KOT-07 requirements. +Output: Updated ArchiveDecoder.kt with v1.1 support + updated test_decoder.sh with directory test cases. + + + +@/home/nick/.claude/get-shit-done/workflows/execute-plan.md +@/home/nick/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/08-rust-directory-archiver/08-01-SUMMARY.md + + + + +Field order in write_toc_entry (FORMAT.md Section 5, v1.1): + name_length(2 LE) | name(N) | entry_type(1) | permissions(2 LE) | + original_size(4 LE) | compressed_size(4 LE) | + encrypted_size(4 LE) | data_offset(4 LE) | iv(16) | hmac(32) | sha256(32) | + compression_flag(1) | padding_after(2 LE) + +Entry size formula: 104 + name_length bytes (was 101 + name_length in v1.0) + +Entry types: + - 0x00 = file (has data block, normal crypto pipeline) + - 0x01 = directory (no data block, all sizes=0, crypto fields zeroed) + +Permissions: lower 12 bits of POSIX mode_t stored as u16 LE (e.g., 0o755 = 0x01ED) + +Version: FORMAT version is now 2 (was 1 in v1.0) + +Directory entries: entry_type=0x01, original_size=0, compressed_size=0, encrypted_size=0, + data_offset=0, iv=zeroed(16), hmac=zeroed(32), sha256=zeroed(32), compression_flag=0 + +Entry names: relative paths with `/` separator (e.g., "mydir/subdir/file.txt") + - No leading `/`, no `..`, no trailing `/` for directories + - Directories appear as TOC entries with their path (e.g., "mydir", "mydir/subdir") + +From existing kotlin/ArchiveDecoder.kt: +```kotlin +data class TocEntry( + val name: String, + // NEW: entry_type and permissions go here (after name, before originalSize) + val originalSize: Long, + val compressedSize: Long, + val encryptedSize: Int, + val dataOffset: Long, + val iv: ByteArray, + val hmac: ByteArray, + val sha256: ByteArray, + val compressionFlag: Int, + val paddingAfter: Int, +) +``` + +From existing kotlin/ArchiveDecoder.kt -- decode() function currently: +- Reads all entries as files +- Writes directly to `File(outputDir, entry.name)` +- Does not handle `/` in entry names (no parent directory creation) +- Version check: `require(version == 1)` + + + + + + + Task 1: Update ArchiveDecoder.kt for v1.1 format with directory support + kotlin/ArchiveDecoder.kt + +Update the Kotlin decoder to handle v1.1 format. All changes are in kotlin/ArchiveDecoder.kt: + +1. **Version check**: In `parseHeader()`, change `require(version == 1)` to `require(version == 2)`. Update the error message accordingly. + +2. **TocEntry data class**: Add two new fields AFTER `name` and BEFORE `originalSize`: + ```kotlin + data class TocEntry( + val name: String, + val entryType: Int, // 0x00=file, 0x01=directory + val permissions: Int, // Lower 12 bits of POSIX mode_t + val originalSize: Long, + // ... rest unchanged + ) + ``` + +3. **parseTocEntry()**: After reading `name` and BEFORE reading `originalSize`, read: + - `entry_type`: 1 byte (`data[pos].toInt() and 0xFF; pos += 1`) + - `permissions`: 2 bytes LE (`readLeU16(data, pos); pos += 2`) + + Include both new fields in the TocEntry constructor call. + +4. **Update entry size comment**: Change "101 + name_length" references to "104 + name_length" throughout. + +5. **decode() function -- directory hierarchy and permissions**: Replace the file extraction loop with logic that handles both files and directories: + + a. **Directory entries (entryType == 1)**: Create the directory with `File(outputDir, entry.name).mkdirs()`. Apply permissions. Print "Created dir: {name}". Do NOT attempt to read ciphertext, decrypt, or verify HMAC. Increment successCount. + + b. **File entries (entryType == 0)**: Before writing the file, ensure parent directories exist: `outFile.parentFile?.mkdirs()`. Then proceed with existing HMAC verify -> decrypt -> decompress -> SHA-256 verify -> write pipeline (unchanged). + + c. **Permissions restoration** (after writing file or creating directory): Apply permissions using Java File API: + ```kotlin + fun applyPermissions(file: File, permissions: Int) { + // Owner permissions (bits 8-6) + file.setReadable(permissions and 0b100_000_000 != 0, true) + file.setWritable(permissions and 0b010_000_000 != 0, true) + file.setExecutable(permissions and 0b001_000_000 != 0, true) + // Others permissions (bits 2-0) -- set non-owner-only flags + file.setReadable(permissions and 0b000_000_100 != 0, false) + file.setWritable(permissions and 0b000_000_010 != 0, false) + file.setExecutable(permissions and 0b000_000_001 != 0, false) + } + ``` + Note: Java's File.setReadable(readable, ownerOnly) -- when `ownerOnly=false`, it sets for everyone; when `ownerOnly=true`, it sets only for owner. The correct pattern is: + - First call with `ownerOnly=false` to set "everyone" bit (this also sets owner) + - The Java API is limited: it can only distinguish owner vs everyone, not owner/group/others separately. This is acceptable per KOT-07 requirement ("File.setReadable/setWritable/setExecutable"). + + Simplified approach (matching Java API limitations): + ```kotlin + fun applyPermissions(file: File, permissions: Int) { + val ownerRead = (permissions shr 8) and 1 != 0 // bit 8 + val ownerWrite = (permissions shr 7) and 1 != 0 // bit 7 + val ownerExec = (permissions shr 6) and 1 != 0 // bit 6 + val othersRead = (permissions shr 2) and 1 != 0 // bit 2 + val othersWrite = (permissions shr 1) and 1 != 0 // bit 1 + val othersExec = permissions and 1 != 0 // bit 0 + + // Set "everyone" permissions first (ownerOnly=false), then override owner-only + file.setReadable(othersRead, false) + file.setWritable(othersWrite, false) + file.setExecutable(othersExec, false) + // Owner-only overrides (ownerOnly=true) + file.setReadable(ownerRead, true) + file.setWritable(ownerWrite, true) + file.setExecutable(ownerExec, true) + } + ``` + +6. **Update decode summary**: Change "files extracted" to "entries extracted" in the final println. Count both files and directories. + +7. **parseToc assertion**: The assertion `require(pos == data.size)` remains correct since the binary layout changed consistently -- all entries now use 104+N instead of 101+N. + + + cd /home/nick/Projects/Rust/encrypted_archive && grep -c "entryType" kotlin/ArchiveDecoder.kt && grep -c "permissions" kotlin/ArchiveDecoder.kt && grep -c "version == 2" kotlin/ArchiveDecoder.kt && grep -c "mkdirs" kotlin/ArchiveDecoder.kt && grep -c "setReadable\|setWritable\|setExecutable" kotlin/ArchiveDecoder.kt + + + - TocEntry has entryType and permissions fields + - parseTocEntry reads entry_type (1 byte) and permissions (2 bytes LE) in correct position + - Version check accepts version 2 instead of version 1 + - Directory entries create directories without decryption + - File entries create parent directories before writing + - Permissions applied via setReadable/setWritable/setExecutable + + + + + Task 2: Update test_decoder.sh with directory test cases + kotlin/test_decoder.sh + +Add directory-specific test cases to the Kotlin cross-validation test script. Keep all existing 5 test cases intact. Add new test cases AFTER test 5: + +1. **Test 6: Directory with nested files** -- Tests KOT-06 (directory hierarchy creation): + ```bash + echo -e "${BOLD}Test 6: Directory with nested files${NC}" + + mkdir -p "$TMPDIR/testdir6/subdir1/deep" + mkdir -p "$TMPDIR/testdir6/subdir2" + echo "file in root" > "$TMPDIR/testdir6/root.txt" + echo "file in subdir1" > "$TMPDIR/testdir6/subdir1/sub1.txt" + echo "file in deep" > "$TMPDIR/testdir6/subdir1/deep/deep.txt" + echo "file in subdir2" > "$TMPDIR/testdir6/subdir2/sub2.txt" + + "$ARCHIVER" pack "$TMPDIR/testdir6" -o "$TMPDIR/test6.archive" + java -jar "$JAR" "$TMPDIR/test6.archive" "$TMPDIR/output6/" + + verify_file "$TMPDIR/testdir6/root.txt" "$TMPDIR/output6/testdir6/root.txt" "testdir6/root.txt" + verify_file "$TMPDIR/testdir6/subdir1/sub1.txt" "$TMPDIR/output6/testdir6/subdir1/sub1.txt" "testdir6/subdir1/sub1.txt" + verify_file "$TMPDIR/testdir6/subdir1/deep/deep.txt" "$TMPDIR/output6/testdir6/subdir1/deep/deep.txt" "testdir6/subdir1/deep/deep.txt" + verify_file "$TMPDIR/testdir6/subdir2/sub2.txt" "$TMPDIR/output6/testdir6/subdir2/sub2.txt" "testdir6/subdir2/sub2.txt" + ``` + +2. **Test 7: Empty directory** -- Tests that empty dirs are created without decryption errors: + ```bash + echo -e "${BOLD}Test 7: Directory with empty subdirectory${NC}" + + mkdir -p "$TMPDIR/testdir7/populated" + mkdir -p "$TMPDIR/testdir7/empty_subdir" + echo "content" > "$TMPDIR/testdir7/populated/file.txt" + + "$ARCHIVER" pack "$TMPDIR/testdir7" -o "$TMPDIR/test7.archive" + java -jar "$JAR" "$TMPDIR/test7.archive" "$TMPDIR/output7/" + + # Verify file content + verify_file "$TMPDIR/testdir7/populated/file.txt" "$TMPDIR/output7/testdir7/populated/file.txt" "testdir7/populated/file.txt" + + # Verify empty directory exists + if [ -d "$TMPDIR/output7/testdir7/empty_subdir" ]; then + pass "testdir7/empty_subdir (empty directory created)" + else + fail "testdir7/empty_subdir" "Empty directory not found in output" + fi + ``` + +3. **Test 8: Mixed files and directories** -- Tests mixed CLI args (standalone files + directory): + ```bash + echo -e "${BOLD}Test 8: Mixed standalone files and directory${NC}" + + ORIG8_FILE="$TMPDIR/standalone.txt" + echo "standalone content" > "$ORIG8_FILE" + mkdir -p "$TMPDIR/testdir8" + echo "dir content" > "$TMPDIR/testdir8/inner.txt" + + "$ARCHIVER" pack "$ORIG8_FILE" "$TMPDIR/testdir8" -o "$TMPDIR/test8.archive" + java -jar "$JAR" "$TMPDIR/test8.archive" "$TMPDIR/output8/" + + verify_file "$ORIG8_FILE" "$TMPDIR/output8/standalone.txt" "standalone.txt (standalone file)" + verify_file "$TMPDIR/testdir8/inner.txt" "$TMPDIR/output8/testdir8/inner.txt" "testdir8/inner.txt (from directory)" + ``` + +Update the summary section to reflect the correct total test count. + +Do NOT modify any of the existing 5 test cases -- they must continue to work unchanged (v1.1 format is not backward compatible, but the Rust archiver now always produces v1.1 archives, so existing test patterns still work). + + + cd /home/nick/Projects/Rust/encrypted_archive && bash -n kotlin/test_decoder.sh && grep -c "Test [0-9]" kotlin/test_decoder.sh + + + - test_decoder.sh has 8 test cases (5 original + 3 directory) + - Test 6 verifies nested directory extraction with 3+ levels + - Test 7 verifies empty directory creation + - Test 8 verifies mixed files + directory pack/unpack + - bash -n syntax check passes + + + + + + +1. `grep -c "entryType" kotlin/ArchiveDecoder.kt` returns >= 3 (data class + parsing + usage) +2. `grep -c "version == 2" kotlin/ArchiveDecoder.kt` returns 1 +3. `grep -c "mkdirs" kotlin/ArchiveDecoder.kt` returns >= 2 (directory creation + parent dir creation) +4. `grep -c "setReadable\|setWritable\|setExecutable" kotlin/ArchiveDecoder.kt` returns >= 6 +5. `bash -n kotlin/test_decoder.sh` passes (syntax check) +6. `grep -c "Test [0-9]" kotlin/test_decoder.sh` returns 8 +7. All existing v1.0 patterns preserved (XOR bootstrapping, encrypted TOC, HMAC-first) + + + +- ArchiveDecoder.kt accepts version 2 archives with entry_type and permissions fields +- Directory entries (entryType=1) create directories without decryption +- File entries with relative paths create parent directories first +- Permissions applied via Java File API (setReadable/setWritable/setExecutable) +- test_decoder.sh includes 3 new directory test cases (nested dirs, empty dir, mixed) +- All code follows established patterns: signed byte masking, contentEquals(), ByteBuffer LE + + + +After completion, create `.planning/phases/09-kotlin-decoder-update/09-01-SUMMARY.md` +