Rewrite test_unmapped_codes assertions with detailed message format:
- Expected: section clarifies the expected result
- Found: section shows actual result with placeholder
- Why this matters: section explains the significance
This improves diagnostic clarity when tests fail by explaining
why specific codes should be unmapped in StandardEncoding.
Refs: bf-5dopl
Updates the assertion in test_resolve_level2_unmapped_code to use the new
diagnostic message format designed in bf-w1o10.
- Changed bare assert!(result.is_failure()) to assert!(result.is_failure(), "...")
- Added structured message with Expected/Found/Why this matters sections
- Clarifies that codes above 0x7F in StandardEncoding are unmapped per build/unmapped-glyph-names.json
This completes the assertion message update task. All other unmapped glyph
assertions in unmapped.rs, encoding.rs, and test files were already updated
with the new format.
Closes bf-5dopl. All unmapped glyph tests pass (unmapped, encoding, resolver, cmap).
Enhanced assertion messages in unmapped_glyph_names_config.rs to include:
- Expected value description
- Actual value description
- Context about why each expectation exists
- References to build/unmapped-glyph-names.json configuration
All 4 tests in unmapped_glyph_names_config.rs pass.
All 7 tests in cmap_unmapped_glyphs.rs pass.
Closes bf-lpyhe
Enhanced all assertion messages in cmap_unmapped_glyphs.rs to follow a consistent pattern:
- What was expected
- What was found
- Why this matters
Modified tests:
- test_cmap_unmapped_glyph_skip: Added detailed messages for unmapped checks, map validation, and lookups
- test_cmap_multiple_mappings_with_unmapped_check: Enhanced mapping verification messages
- test_cmap_range_mapping_with_unmapped_awareness: Improved range expansion and boundary check messages
All 7 tests pass successfully. Assertion messages now provide maximally helpful diagnostics when failures occur.
Verification: notes/bf-18a0w.md
- Fix test_differences_overlay_filters_null_glyph expected value: /Z instead of Z
- Update verification note with latest test results showing all 7 tests pass
- Normal glyphs presence assertions working correctly (A, B, space, C verified present)
Closes bf-3ayf6
Documented attempt to execute pdftract CLI with OCR on degraded-200dpi.pdf.
Found that OCR feature requires leptonica system dependencies not currently
available in NixOS environment. CLI invocation and fixture file are valid,
but compilation fails due to missing lept.pc.
Acceptance criteria:
- Command executes: PARTIAL (valid syntax, build fails due to dependencies)
- Process starts: FAIL (blocked by missing leptonica headers)
- OCR process begins: FAIL (build prevents execution)
This is an environmental limitation, not a code defect.
References: bf-3n62c
Add assertions to test_cmap_unmapped_glyph_skip to verify that unmapped
glyphs (configured in build/unmapped-glyph-names.json as g001-g003,
.notdef, .null) are absent from the CMAP output structure.
Changes:
- Add count verification: Ensures CMAP contains exactly the expected
number of mappings, detecting spurious entries
- Add validity check: Iterates over all entries verifying valid Unicode
destinations, with clear failure messages
Assertion messages identify which mapping should be absent via byte
sequence inspection, meeting the bead's acceptance criteria.
Verification: notes/bf-5d2id.md
Closes: bf-5d2id
- Add standalone is_ssrf_blocked() function to mcp_helpers module
- Function accepts JsonRpcError reference and returns bool (stub returning false)
- Added comprehensive documentation with usage examples
- Located in TH-05-ssrf-block.rs after is_ssrf_blocked_error() function
- Verification: notes/bf-pxdn0.md
Closes bf-pxdn0
- Add iter() method to ToUnicodeMap for accessing CMAP structure
- Extend test_cmap_unmapped_glyph_skip to show CMAP contents
- Extend test_cmap_multiple_mappings_with_unmapped_check to show mappings
- Add debug output for inspection of source bytes and target chars
- Add verification note at notes/bf-okdnk.md
This enables inspection of CMAP output structure to verify unmapped glyphs
are properly handled before implementing actual filtering logic.
Closes bf-okdnk
- Identified two main diagnostic formats: JSON-Lines (compiler) and standard JSON (fixtures)
- Documented JSON-Lines format from .fingerprint directory output files
- Added schema details, field descriptions, and sample entries
- Noted encoding (UTF-8) and line endings (Unix \n)
- Updated verification note with comprehensive format findings
Closes bf-5g04q
Add comprehensive metric extraction logic for benchmark output parsing:
- Add RawTimingMetrics struct with fields for wall time, CPU time, file counts, and throughput
- Implement extract_raw_timing_metrics() to parse stdout/stderr for timing information
- Add extract_time_value() helper to parse CPU time from text formats
- Update run_benchmark() to use extraction logic instead of simple parsing
- Calculate throughput metrics (files/sec, MB/s) from extracted data
Closes bf-5b8mk. Verification: notes/bf-5b8mk.md, commit $(git rev-parse --short HEAD).
- Add READ_TIMEOUT (5 min) and MAX_OUTPUT_SIZE (100 MB) constants
- Implement read_pipe_with_timeout() with watchdog thread protection
- Update execute_grep_command() to capture and return both streams
- Prevents TH-03 hang scenario by bounding both time and memory
Acceptance criteria:
✓ Stdout captured completely
✓ Stderr captured completely
✓ Output buffered without truncation (100 MB limit)
✓ Read operations have timeout protection (5 min timeout)
✓ Both streams captured even if one is empty
Verification: notes/bf-3wkpz.md
Created crates/pdftract-core/tests/cmap_unmapped_glyphs.rs with:
- test_cmap_unmapped_glyph_skip: Basic test verifying unmapped glyph name detection and simple CMAP parsing
- test_cmap_multiple_mappings_with_unmapped_check: Tests multiple CMAP mappings with unmapped glyph awareness
- test_cmap_range_mapping_with_unmapped_awareness: Tests CMAP range mapping with unmapped glyph checks
All tests compile and pass successfully. This provides the foundation for comprehensive unmapped glyph filtering verification in CMAP generation.
Closes bf-4y66q
This commit cleans up the remaining working tree modifications as part of bead bf-56090:
1. test_truncated_fate_recovery.rs: Migrate test fixture path from
tests/fixtures/malformed/truncated-flate.pdf to
tests/error_recovery/fixtures/truncated_mid_stream.pdf and improve
test output formatting.
2. generate_form_fixtures.rs: Update lopdf API calls to use new StringFormat
enum instead of deprecated boolean parameter, and migrate from
save_to_bytes() to save_to() with Cursor.
Both changes are necessary for compilation with current lopdf version.
Fixtures verified to exist at both old and new paths.
Related to: bf-56090
- Add #[serde(default)] to unmapped_glyph_names field in UnmappedGlyphNamesConfig
- Field now defaults to empty Vec<String> when not specified in config files
- Add comprehensive integration tests covering all scenarios:
- Config without unmapped_glyph_names field (defaults to empty)
- Config with unmapped_glyph_names specified (parses correctly)
- Config with explicit empty array (works as expected)
- Minimal empty config (all fields default appropriately)
Acceptance criteria:
- ✅ unmapped_glyph_names defaults to empty list when not specified
- ✅ Default value handling integrates cleanly with existing config logic
- ✅ Code compiles without errors
- ✅ Both specified and unspecified cases work correctly (4/4 tests pass)
Closes bf-34xgg
- Use UnmappedGlyphNamesConfig struct for proper deserialization
- Replace serde_json::Value with typed parsing for better error messages
- Handle both empty and populated glyph name lists correctly
- Add verification note at notes/bf-66dwv.md
Closes bf-66dwv
Added typed config struct for unmapped glyph names configuration:
- unmapped_glyph_names: Vec<String> - list of glyph names to skip
- description: Option<String> - optional documentation
- version: Option<String> - format version identifier
The struct represents the JSON structure from build/unmapped-glyph-names.json
and is ready for parsing implementation in follow-up beads.
Closes bf-3vo80. Verification: notes/bf-3vo80.md
Fixed Debug formatting issue in test_truncated_flate_recovery.rs where XrefResolver
doesn't implement Debug. Changed assert to only format error case.
Verification:
- Basic fixture existence test passes
- Test compiles and runs without crashing
- Scaffold structure is in place for future implementation
Closes bf-3b41k
Add unmapped_glyph_names configuration field to DifferencesOverlay struct,
which serves as the CMAP generator for Type1 font encoding differences.
Changes:
- Added unmapped_glyph_names: HashSet<String> field to DifferencesOverlay
- Added with_unmapped_glyph_names() constructor for custom configuration
- Added is_unmapped_glyph_name() instance method for checking membership
- Added unmapped_glyph_names() getter and set_unmapped_glyph_names() setter
- Updated parse() to use instance method instead of global function
- Defaults to global UNMAPPED_GLYPH_NAMES set when not explicitly configured
Acceptance criteria:
- CMAP generator struct has unmapped_glyph_names field
- Field is HashSet<String> for O(1) lookup efficiency
- Field can be set via constructor or setter method
- Code compiles and all tests pass (24 encoding tests)
Closes bf-4zgpr. See notes/bf-4zgpr.md for verification details.
All acceptance criteria met:
- All 5 test modules present and properly structured
- Fixture validation tests run successfully (not ignored)
- Encrypted PDF tests have #[ignore] with clear reasons
- File compiles successfully with no errors
- 10 encrypted PDF tests ignored, 3 fixture validation tests pass
Verification note: notes/bf-1or48.md
- Verified forms_integration test runs successfully end-to-end
- All 6 tests pass without hanging, panicking, or infinite loops
- Test output is clear and readable with structured messages
- Fixtures directory scaffold ready for future PDF form fixtures
- Binary exists and is executable at target/debug/pdftract
Closes bf-2s1hq
Commented out unused internal module imports in encryption error tests
with comprehensive documentation explaining the subprocess testing
approach. Verified all modules exist and compile successfully.
Verified imports:
- pdftract_cli::password (crates/pdftract-cli/src/password.rs)
- pdftract_core::diagnostics types (crates/pdftract-core/src/diagnostics.rs)
Tests use std::process::Command to spawn CLI binary, not direct
module calls. Imports retained as comments for future reference.
Closes bf-31re9. Verification: notes/bf-31re9.md
- Add std::process::Stdio to imports in both test files
- Update code to use imported Stdio instead of fully qualified paths
- Verify compilation succeeds with cargo check
Closes bf-3x4rp. Verification: notes/bf-3x4rp.md
- Create fuzz/fuzz_targets/profile_yaml.rs to test YAML parsing
- Register profile_yaml target in fuzz/Cargo.toml
- Verify cargo-fuzz 0.13.1 is installed and working
- All 7 fuzz targets now operational per plan requirement (line 3236)
Closes bf-1b7od. Verification: notes/bf-1b7od.md
Add comprehensive test utility and error handling imports to encryption test files:
- Added std::error::Error and std::io for error handling
- Expanded pdftract_core::diagnostics imports to include Diagnostic, DiagnosticsCollector, and ObjRef
- Ensured both test_encryption_errors.rs and test_encryption_unsupported.rs have consistent imports
All imports verified against actual crate structure and compile successfully.
Closes bf-1h5og. Verification: notes/bf-1h5og.md.
Add inline code comments at ToUnicode entry creation points and update
comprehensive documentation notes.
Changes:
- Add MARKER comment at resolver.rs:398-402 for ToUnicode resolution entry creation
- Update notes/bf-2nob5-child-1.md to document all CMAP and ToUnicode creation points
- Verify existing MARKER comments in cmap.rs, codespace.rs, and encoding.rs
Acceptance criteria:
- Code comments added at ToUnicode creation points
- Comprehensive notes file updated with all findings
- Notes file includes file paths, function names, line numbers, and data flow summaries
Closes bf-65hvp
- Added docstring MARKER at CodespaceRange::new() (codespace.rs:56-60) noting
CMAP entry creation point for multi-byte CJK encodings
- Added inline comment MARKER at codespace range creation (codespace.rs:354)
where ranges are created from begincodespacerange blocks
- Created comprehensive notes file at notes/bf-2nob5-child-1.md with:
* All CMAP and ToUnicode entry creation points
* File paths, function names, and line numbers
* Complete data flow summaries and diagrams
* Related files and test locations
* Special cases and edge cases documentation
Verified existing MARKER comments at:
- ToUnicodeMap::add_mapping() (cmap.rs:86-88)
- DifferencesOverlay::parse() (encoding.rs:196-199)
Acceptance criteria met:
✅ Code comments added at CMAP creation points
✅ Code comments added at ToUnicode creation points
✅ Comprehensive notes file created with all findings
✅ Notes include file paths, function names, line numbers, data flows
Closes bf-65hvp
The cmap_tokenize benchmark uses CJK tokenization functions that are
behind the 'cjk' feature flag. Added required-features = ["cjk"] to
the benchmark configuration in Cargo.toml so cargo check --all-targets
passes cleanly.
This fixes compilation error: error[E0432]: unresolved import
pdftract_core::cmap::tokenize_cjk_bytes (item is gated behind cjk feature).
Verification: cargo check --all-targets now passes with no errors.
- Add warn! macro import to javascript.rs module
- Add warning log at detect_javascript() entry point stating execution is not supported
- Update verification note for bf-2pxsu
This complements the logging already present in detection.rs (from bf-2pyg1) by ensuring the JavaScript module itself has appropriate logging.
Per TH-04 threat model, pdftract NEVER executes embedded JavaScript; this logging explicitly confirms detection-only behavior.
- Add test_extract_all_discovered_pdfs() function to forms_integration.rs
- Use walkdir-based discover_pdf_fixtures() for recursive PDF discovery
- Spawn pdftract extract --json process for each PDF and capture output
- Print detailed status and output preview for debugging
- Fix pdftract_bin() path resolution (workspace target dir)
Closes bf-394lb. Verification: notes/bf-394lb.md. Tests: PASS (compiles, runs, no hangs).
- Add notes/bf-e4uvb-child-1.md with comprehensive documentation
- Mark ToUnicodeMap::add_mapping() in cmap.rs (line 86)
- Mark DifferencesOverlay entry creation in encoding.rs (line 195)
- Mark CodespaceRange creation in codespace.rs (line 256)
These are the critical insertion points for future unmapped glyph skip logic.
Closes bf-e4uvb-child-1.
