- Add test_extract_all_discovered_pdfs() function to forms_integration.rs
- Use walkdir-based discover_pdf_fixtures() for recursive PDF discovery
- Spawn pdftract extract --json process for each PDF and capture output
- Print detailed status and output preview for debugging
- Fix pdftract_bin() path resolution (workspace target dir)
Closes bf-394lb. Verification: notes/bf-394lb.md. Tests: PASS (compiles, runs, no hangs).
Add discover_pdf_fixtures function to recursively find all PDF files in
tests/fixtures/forms/ directory using walkdir. Includes test that prints
discovered fixture names to stdout.
Acceptance criteria:
- walkdir is available in crates/pdftract-cli/Cargo.toml
- discover_pdf_fixtures function returns Vec<PathBuf> of discovered PDFs
- test_discover_pdf_fixtures runs and prints fixture names
- No compilation errors
Closes bf-1pxdm.
- Created tests/forms_integration.rs with basic test module structure
- Created tests/mod.rs to aggregate test modules
- Added module documentation for forms integration tests
- File compiles successfully with cargo test --test forms_integration
Acceptance criteria:
- tests/forms_integration.rs exists ✓
- tests/mod.rs includes the module declaration ✓
- cargo test --test forms_integration compiles ✓
- File has basic test module skeleton ✓
Closes bf-2y7uh
- Add notes/bf-e4uvb-child-1.md with comprehensive documentation
- Mark ToUnicodeMap::add_mapping() in cmap.rs (line 86)
- Mark DifferencesOverlay entry creation in encoding.rs (line 195)
- Mark CodespaceRange creation in codespace.rs (line 256)
These are the critical insertion points for future unmapped glyph skip logic.
Closes bf-e4uvb-child-1.
Create scripts/measure-wer.sh that computes Word Error Rate between OCR output and ground truth text.
Features:
- Takes two arguments: <ocr_output_file> and <ground_truth_file>
- Computes WER using formula: (substitutions + insertions + deletions) / total_words
- Exit code 0 if WER ≤ 3%, otherwise exit code 1
- Includes --help for usage information
- Supports VERBOSE=1 for detailed output
Acceptance criteria verified:
- ✓ Script exists and is executable
- ✓ Identical inputs: WER = 0.00%, exit code 0
- ✓ 10% substitutions: WER = 11.11%, exit code 1
- ✓ --help shows usage
Closes bf-3gewx
- Add content target to fuzz/Cargo.toml
- Fix font fingerprint build script to use hex string keys instead of byte arrays
- Update fingerprint.rs to use hex string lookups
- Build verification: 54MB binary produced successfully
Acceptance criteria:
- ✅ cargo fuzz build content completes successfully
- ✅ No compilation errors or warnings
- ✅ Harness binary built and available
Closes bf-1c4f6
- Create profiles/community/ directory structure with CONTRIBUTING.md
- CONTRIBUTING.md covers all required contribution areas:
* File naming convention and directory structure
* Required profile fields and schema requirements
* Local testing workflow with validation steps
* PR submission process and review guidelines
* License requirements for fixture PDFs
* Review checklist and quality standards
- Update extraction_loader.rs to add Community profile origin
- Update profiles_cmd.rs to list community profiles separately
- Document community profiles in profile-authoring.md search path
- Add test-profile example to verify implementation
Acceptance criteria met:
✅ profiles/community/CONTRIBUTING.md exists with comprehensive guide
✅ Code supports loading and listing community profiles
✅ Path documented in profile-authoring.md
Closes bf-4cbfb
- corpus-licensing.md: OQ-01 resolution - all fixtures are synthetic with no external licensing
- font-fingerprinting.md: OQ-02 resolution - Level 3 fingerprint database methodology and curation pipeline
- ocr-accuracy.md: PB-3 fallback plan - Tesseract WER targets (3% primary, 5% fallback) with methodology
- pdf-2-coverage.md: PB-10/R10 analysis - PDF 2.0 feature compatibility matrix
All four files are required phase sign-off artifacts referenced in the plan.
Resolves OQ-01, OQ-02, PB-3, PB-10.
Fixed critical bug where validation script exited prematurely under
set -euo pipefail due to arithmetic operations returning non-zero exit
codes when evaluating to 0.
Changes:
- Replaced ((var++)) with var=$((var + 1))
- Replaced ((var += value)) with var=$((var + value))
- Restructured progress indicator to avoid modulo 0 result
Verification:
- Script now successfully validates all 1,260 corpus files
- Correctly reports 603 valid files with complete license info
- Detects 657 files with missing licenses (expected - PDFium test files)
- Exits with code 1 when validation fails (as designed)
Closes bf-632yt
Implement scripts/validate-corpus.sh to verify corpus integrity against
manifest.csv. Validates:
- File existence
- File size matching manifest
- SHA256 checksums
- License information presence
- Total counts (files, pages, size)
Exit 0 on success, non-zero on validation failure. Outputs summary with
total files, pages, size, and validation status.
Closes bf-632yt
- download-grep-corpus.sh now generates manifest.csv entries during download
- Implements SHA256 checksum computation via sha256sum
- Extracts page counts using pdfinfo
- Captures file sizes in bytes via stat
- Records source URLs for each downloaded file
- Appends entries to manifest.csv with all 6 required fields
- Handles incremental downloads (skips existing files, appends to manifest)
- Added grep-corpus-generate-manifest.sh helper for bootstrapping existing corpus
- Generated manifest entries for all 1,260 existing PDFs
- Verified checksum accuracy against actual files
Acceptance criteria:
- ✅ download-grep-corpus.sh writes manifest.csv entries for each downloaded PDF
- ✅ All 6 fields populated: filename, source_url, page_count, file_size, checksum, license
- ✅ Script handles incremental downloads (appends to existing manifest)
Closes bf-20fot
Verification: notes/bf-20fot.md
- Add pdftract-dotnet-publish.yaml for NuGet package publishing
- Add pdftract-libpdftract-build.yaml for C FFI library builds
- Both workflows model existing PHP/Ruby publish patterns
- NuGet publish with proper API key and symbol packaging
- C library build using cbindgen and cross-compilation
- Supports all 5 target triples (Linux x64/ARM, macOS x64/ARM, Windows)
- Homebrew formula and vcpkg port generation included
Closes bf-20v7h
- Confirm ArgoCD sync status - templates exist in declarative-config
- Document commit hash (cb7b4f28) for traceability
- All acceptance criteria verified and met
- Fix /Differences array construction in generate_unmapped_glyphs.py
- Changed from '[code glyph code glyph ...]' to '[starting_code glyph1 glyph2 ...]' format
- All 7 unmapped glyphs from design spec now properly encoded without ToUnicode CMap
- Fixture validation passes: no ToUnicode, correct encoding format, all glyphs present
Closes bf-84xr8. Verification: notes/bf-84xr8.md
- Implement parameterized generator for unmapped glyph PDF fixtures
- Support custom glyph names via --glyphs JSON parameter
- Support custom font titles via --title parameter
- Include built-in test set matching design specification (notes/bf-68f9i-design.md)
- Generate valid PDF 1.4 with Type1 custom encoding
- Passes all acceptance criteria:
* Script exists in tests/fixtures/encoding/
* Accepts parameters for glyph names and encoding
* Generates valid PDF with specified structure
* Tested with dry runs (default and custom glyphs)
* Committed to git
Closes bf-3cwge
- Added PDF Structure section based on bf-4ozvw findings
- Updated SHA256 in profiles/PROVENANCE.md for encoding fixtures
- Section includes file characteristics, font analysis, and technical significance
Examine the no-mapping.pdf test fixture using PDF analysis tools:
- pdfinfo: extracted metadata (1 page, 660 bytes, PDF 1.4, letter)
- pdffonts: identified CustomNoMap Type 1 font with custom encoding
- Documented structural findings for later reference
Key finding: fixture uses custom encoding with no embedding, designed to
test pdftract's handling of glyphs without standard Unicode mappings.
Closes bf-4ozvw.
Verified that existing documentation at tests/fixtures/no-mapping.md
comprehensively catalogs the 3 unmapped glyphs (g001, g002, g003)
with CID numbers and explanations for why they're unmapped.
All acceptance criteria met - documentation already complete from
previous bead bf-68f9i.
Closes bf-644ci
Successfully executed generate_unicode_recovery_fixtures_bin to create
no-mapping.pdf fixture (650 bytes). Validated PDF structure includes
custom /CustomNoMap font with /Differences encoding, non-AGL glyph
names, and no ToUnicode entry. Ground truth contains U+FFFD × 3.
Closes bf-ttbb5.
Add comprehensive research documentation for creating a PDF fixture
that tests the ENCODING_NO_MAPPING failure mode.
Covers:
- PDF specification for fonts without ToUnicode CMap
- Custom encoding approach using /Differences array
- Tooling and library options (manual construction, ReportLab, lopdf)
- Test text content specification
- Fixture structure validation
- Recovery level exercise confirmation
- Complete implementation examples in Rust and validation script
References:
- PDF 1.7 Specification (ISO 32000-1:2008)
- Adobe Glyph List (AGL)
- Internal docs: pdf-fonts-and-encoding.md, cmap-format-and-cid-encoding.md
Acceptance criteria met:
- [x] Documented approach exists in notes/bf-f0xqd-research.md
- [x] Approach specifies how to create a font with no ToUnicode CMap
- [x] Approach includes tooling/library requirements
- [x] Approach specifies test text content for the fixture
Closes bf-2g412
