Commit graph

13 commits

Author SHA1 Message Date
jedarden
691ff10066 docs(bf-10vsm): document generator config structure location
- Identified build/unmapped-glyph-names.json as the primary config file
- Documented DifferencesOverlay struct in encoding.rs as the Rust config structure
- Described the build-time generation process via build.rs
- Listed all files involved in the config pipeline
Closes bf-10vsm
2026-07-06 14:38:34 -04:00
jedarden
8910e934da test(bf-cc7ru): create ground truth text for degraded 200 DPI PDF fixture
- Updated degraded-200dpi-ground-truth.txt with accurate transcription
- Matches first 2500 chars of source document (as created by PDF generation script)
- Contains Abraham Lincoln public domain text with metadata
- Verified with diff against source content
- Includes verification note at notes/bf-cc7ru.md

Acceptance criteria:
✓ Ground truth file exists in tests/fixtures/scanned/low-quality/
✓ Accurately reflects source document content (verified)
✓ Properly formatted and readable
✓ Special characters handled appropriately

Closes bf-cc7ru
2026-07-06 12:17:43 -04:00
jedarden
77eeaeccca test(bf-59ah8): document MCP server process spawning and JSON-RPC communication implementation
Completes bead bf-59ah8 - MCP server subprocess management and JSON-RPC
message handling for SSRF blocking tests.

Implementation includes:
- RAII process guard (McpServerGuard) with bounded cleanup waits
- JSON-RPC tools/call message construction
- LSP-style framed message I/O (Content-Length headers)
- Response parsing for error messages
- Seven test cases for SSRF blocking validation

Communication layer verified working - tests successfully spawn MCP server,
send JSON-RPC messages, receive and parse responses. No orphaned processes.

Test results: 1/7 tests pass (test_ipv4_loopback_blocked handles both error
and stub responses). Other 6 tests expect SSRF blocking implementation
(separate task).

Closes bf-59ah8
2026-07-06 12:03:16 -04:00
jedarden
7d4f1fa6d3 feat(bf-2ngt6): add public-domain source document for degraded OCR fixture 2026-07-06 11:36:14 -04:00
jedarden
bcab32d890 docs(bf-1puzp): verify CMAP skip logic for unmapped glyphs
All acceptance criteria verified:
- Glyph names checked against unmapped_glyph_names config
- Unmapped glyphs skipped during CMAP generation
- Code compiles without errors
- Tests demonstrate skip behavior

The implementation was already complete in encoding.rs:200.
See notes/bf-1puzp.md for detailed verification.
2026-07-06 10:03:10 -04:00
jedarden
bc9e6f8e57 test(bf-1b7od): add profile_yaml fuzz target and verify cargo-fuzz configuration
- Create fuzz/fuzz_targets/profile_yaml.rs to test YAML parsing
- Register profile_yaml target in fuzz/Cargo.toml
- Verify cargo-fuzz 0.13.1 is installed and working
- All 7 fuzz targets now operational per plan requirement (line 3236)

Closes bf-1b7od. Verification: notes/bf-1b7od.md
2026-07-06 09:38:59 -04:00
jedarden
dfc94f33d5 chore(bf-2yhak): relocate useful generators to tools/
Relocate convert_to_scanned.sh from tests/fixtures/scanned/ to tools/
with comprehensive documentation header. Delete incomplete
regenerate.sh stub from tests/fixtures/grep-corpus/. Create
tools/README.md cataloging all 18 generators with usage examples.

Changes:
- Move: tests/fixtures/scanned/convert_to_scanned.sh → tools/convert_pdf_to_scanned.sh
- Delete: tests/fixtures/grep-corpus/regenerate.sh (incomplete stub)
- Create: tools/README.md (comprehensive generator catalog)
- Enhance: convert_pdf_to_scanned.sh with detailed documentation

Closes bf-2yhak
2026-07-05 20:17:04 -04:00
jedarden
bbbb7640d4 chore(bf-xqib3): remove obsolete generators and compiled artifacts from tests/fixtures/
Remove 5 DELETE-category generator files identified in bf-1iefu:
- tests/fixtures/scanned/generate_scanned_fixtures.rs (Rust stub, use Python version)
- tests/fixtures/security/generate_sensitive_fixture.py (duplicate, Rust version preferred)
- tests/fixtures/encoding/generate_unmapped_glyphs.rs (superseded by Python version)
- tests/fixtures/malformed/gen-bomb-10k-2g.sh (superseded by Python version)
- tests/fixtures/forms/generate_form_fixtures.d (orphaned Makefile dependency)

Verification: No compiled object files (.o, .so, .a, .dylib, .dll) or executable binaries found. The .bin files in fixtures root are test data files, not compiled artifacts.
2026-07-05 20:14:39 -04:00
jedarden
a68d01c726 feat(bf-337i2): create 300 DPI invoice OCR fixture
Add invoice fixture with ground truth text for OCR testing.

Files:
- tests/fixtures/scanned/invoice/invoice-300dpi.pdf: 300 DPI scanned invoice
- tests/fixtures/scanned/invoice/invoice-300dpi-ground-truth.txt: complete text content
- tools/generate_invoice_pdf_fixtures.py: fixture generator script
- tests/fixtures/PROVENANCE.md: added provenance entries for new fixtures
- notes/bf-337i2.md: verification note

Verification:
- PDF is image-based (type=image, not text-embedded)
- Resolution: 300 DPI (x-ppi=300, y-ppi=300)
- Dimensions: 2550 x 3300 pixels (letter size at 300 DPI)
- Ground truth contains complete invoice text with headers, addresses, line items, totals

Closes bf-337i2
2026-07-05 18:20:09 -04:00
jedarden
8473941ebb docs(bf-e4uvb-child-1): document CMAP and ToUnicode entry creation points
- Add notes/bf-e4uvb-child-1.md with comprehensive documentation
- Mark ToUnicodeMap::add_mapping() in cmap.rs (line 86)
- Mark DifferencesOverlay entry creation in encoding.rs (line 195)
- Mark CodespaceRange creation in codespace.rs (line 256)

These are the critical insertion points for future unmapped glyph skip logic.
Closes bf-e4uvb-child-1.
2026-07-05 14:48:53 -04:00
jedarden
a0c3ebb237 feat(bf-5for4): design manifest schema for grep-corpus
- Add manifest.csv with header row: filename,source_url,page_count,file_size,checksum,license
- Update README.md with comprehensive schema documentation and field descriptions
- Schema supports provenance tracking, integrity verification, and license compliance

Closes bf-5for4
2026-07-05 12:21:25 -04:00
jedarden
8fe61a1ba5 docs(pdftract-25k4x): add verification note for figure/caption detection 2026-06-01 09:35:02 -04:00
jedarden
3d795a2d11 feat(bf-2he4t): assemble scanned fixtures corpus with ground-truth transcripts
Created tests/fixtures/scanned/ directory structure for WER gate testing:

- README.md: Corpus overview and WER targets (<3% on clean 300-DPI scans)
- GEN_MANIFEST.md: Fixture specifications and generation checklist
- receipt/receipt-300dpi.txt: Ground truth for AS-02 test scenario (37 lines)
- documents/invoice-300dpi.txt: Business invoice ground truth (55 lines)
- documents/form-300dpi.txt: Employment application form (78 lines)
- multi-page/doc-10page-300dpi.txt: Performance fixture (255 lines, 10 pages)

Generation tools:
- generate_scanned_fixtures.py: Python script for PDF generation
- generate_scanned_fixtures.rs: Rust alternative for fixture metadata
- calculate_wer.py: WER/CER calculation utility for OCR validation

Test stub:
- wer_gate_stub.rs: Placeholder for WER gate tests (marked #[ignore])

Total ground-truth content: 425 lines across 4 fixtures

Next steps:
1. Generate PDFs from ground truth using generation script
2. Verify WER < 3% on generated fixtures
3. Enable WER gate tests

Closes bf-2he4t
2026-06-01 09:25:53 -04:00