- Confirm ArgoCD sync status - templates exist in declarative-config
- Document commit hash (cb7b4f28) for traceability
- All acceptance criteria verified and met
- Fix /Differences array construction in generate_unmapped_glyphs.py
- Changed from '[code glyph code glyph ...]' to '[starting_code glyph1 glyph2 ...]' format
- All 7 unmapped glyphs from design spec now properly encoded without ToUnicode CMap
- Fixture validation passes: no ToUnicode, correct encoding format, all glyphs present
Closes bf-84xr8. Verification: notes/bf-84xr8.md
- Implement parameterized generator for unmapped glyph PDF fixtures
- Support custom glyph names via --glyphs JSON parameter
- Support custom font titles via --title parameter
- Include built-in test set matching design specification (notes/bf-68f9i-design.md)
- Generate valid PDF 1.4 with Type1 custom encoding
- Passes all acceptance criteria:
* Script exists in tests/fixtures/encoding/
* Accepts parameters for glyph names and encoding
* Generates valid PDF with specified structure
* Tested with dry runs (default and custom glyphs)
* Committed to git
Closes bf-3cwge
- Added PDF Structure section based on bf-4ozvw findings
- Updated SHA256 in profiles/PROVENANCE.md for encoding fixtures
- Section includes file characteristics, font analysis, and technical significance
Examine the no-mapping.pdf test fixture using PDF analysis tools:
- pdfinfo: extracted metadata (1 page, 660 bytes, PDF 1.4, letter)
- pdffonts: identified CustomNoMap Type 1 font with custom encoding
- Documented structural findings for later reference
Key finding: fixture uses custom encoding with no embedding, designed to
test pdftract's handling of glyphs without standard Unicode mappings.
Closes bf-4ozvw.
Verified that existing documentation at tests/fixtures/no-mapping.md
comprehensively catalogs the 3 unmapped glyphs (g001, g002, g003)
with CID numbers and explanations for why they're unmapped.
All acceptance criteria met - documentation already complete from
previous bead bf-68f9i.
Closes bf-644ci
Successfully executed generate_unicode_recovery_fixtures_bin to create
no-mapping.pdf fixture (650 bytes). Validated PDF structure includes
custom /CustomNoMap font with /Differences encoding, non-AGL glyph
names, and no ToUnicode entry. Ground truth contains U+FFFD × 3.
Closes bf-ttbb5.
Add comprehensive research documentation for creating a PDF fixture
that tests the ENCODING_NO_MAPPING failure mode.
Covers:
- PDF specification for fonts without ToUnicode CMap
- Custom encoding approach using /Differences array
- Tooling and library options (manual construction, ReportLab, lopdf)
- Test text content specification
- Fixture structure validation
- Recovery level exercise confirmation
- Complete implementation examples in Rust and validation script
References:
- PDF 1.7 Specification (ISO 32000-1:2008)
- Adobe Glyph List (AGL)
- Internal docs: pdf-fonts-and-encoding.md, cmap-format-and-cid-encoding.md
Acceptance criteria met:
- [x] Documented approach exists in notes/bf-f0xqd-research.md
- [x] Approach specifies how to create a font with no ToUnicode CMap
- [x] Approach includes tooling/library requirements
- [x] Approach specifies test text content for the fixture
Closes bf-2g412
The no-mapping.pdf fixture uses custom glyph names (/g001, /g002, /g003)
that are not in the Adobe Glyph List and cannot be recovered through any
Unicode recovery mechanism (no ToUnicode CMap, no standard encoding,
no AGL match).
The ground truth has been corrected from 'ABC' to three U+FFFD replacement
characters (���), which is the expected output when all encoding recovery
methods fail per the Failure Mode Taxonomy.
Also updated PROVENANCE.md with new SHA256 hashes for encoding fixtures
that were regenerated on 2026-07-02.
This fixture correctly exercises the ENCODING_NO_MAPPING failure mode as
specified in the plan (line 733).
Closes bf-f0xqd.
CJK fixtures and tests already exist from previous work:
- tests/fixtures/cjk/ contains all 4 required PDFs
- Ground truth files for each encoding (GB18030, Shift-JIS, EUC-KR, Big5)
- Tests in crates/pdftract-core/tests/cjk_encoding.rs and tests/test_encoding.rs
- Tests fail due to unimplemented CJK encoding (expected for Phase 2.3)
- Updated PROVENANCE.md with CJK fixture entries
Fixtures are ready for CJK encoding implementation.
Closes bf-3ourh
- no-mapping.txt: fix garbled unicode to correct 'ABC' output
- shape-match.txt: fix from 'Shape' to 'S' (actual PDF content)
- Add PROVENANCE.md entries for all 4 encoding fixtures
- PDFs remain unchanged (already valid)
Fixes ground truth for Level 2-4 Unicode recovery fixtures:
- no-mapping.pdf: PDF with no ToUnicode, no standard encoding
- agl-only.pdf: PDF with AGL glyph names only
- fingerprint-match.pdf: PDF with embedded font for fingerprint matching
- shape-match.pdf: PDF with subset font for shape recognition
Closes bf-512z1
- Add MSRV 1.78 to installation section
- Enhance documentation section with descriptive link text
- Ensure all required links are present (user-docs, extraction-output-schema, sdk-architecture, manual-platform-smoke)
Closes pdftract-5gld
Verified all acceptance criteria:
- All documentation pages exist and build successfully with mdbook
- CLI reference is up-to-date (auto-generated from clap)
- JSON schema reference links to correct source file
- SDK quickstarts match tested API patterns
- Troubleshooting covers 28+ diagnostic codes from Phases 1-7
- FAQ covers 24 questions including all planned topics
No gaps identified - documentation is complete and comprehensive.
Regenerated cli-reference.md with updated autogen comment pointing to
the correct xtask manifest path. The clap-markdown generator already
adds the '# CLI Reference' header, so removed the duplicate.
Closes pdftract-1j0f8.
Verified CLI reference documentation is complete and working:
- cli-reference.md exists (646 lines, 28 commands)
- Auto-gen compiles and runs via cargo run --bin gen-cli-reference
- CI gate cli-ref-gen fails on stale content
- mdBook builds successfully
All acceptance criteria PASS.
The conformance subcommand had duplicate short options (-s) for both
--suite and --sdk, causing the CLI reference generator to panic with
"Short option names must be unique".
Changed --sdk short option from -s to -k (matching the CI workflow
convention). This allows the gen-cli-reference binary to run and the
CI cli-ref-gen gate to function correctly.
Also regenerated mdBook build output including the new cli-reference.html.
Closes pdftract-1j0f8. Verification: notes/pdftract-1j0f8.md.
The gen-cli-reference binary was accumulating extra blank lines after
the <!-- AUTOGEN END --> marker on each regeneration because it
preserved all content after the marker (including leading whitespace)
and then added its own newlines.
Fix: Trim leading whitespace from hand-curated content before appending.
Also regenerated cli-reference.md to remove accumulated blank lines.
Closes pdftract-1j0f8
- Regenerated CLI reference using the CLI crate binary (gen-cli-reference)
- Updated all subcommands to use clap-markdown auto-generation format
- Preserved hand-curated content after AUTOGEN END marker
- CI gate verifies docs stay in sync with CLI changes
Acceptance criteria verified:
- cli-reference.md covers all subcommands (extract, classify, profiles, serve, mcp, inspect, grep, cache, doctor, verify-receipt, hash, validate, conformance)
- Auto-gen compiles and runs: cargo run --bin gen-cli-reference
- CI gate in pdftract-ci.yaml checks for stale docs
- mdBook builds without errors
- Add Ligature::Ff to the skip_next pattern in repair_split_ligatures
- Update mojibake test patterns to use readable Unicode escape sequences
- Fix NBSP test to use correct UTF-8 byte sequences
- Simplify multiple mojibake test to focus on accented character repair
- Update ligature test with more realistic scenario and complete glyph sequence
This fixes the handling of 'ff' ligatures that appear as f<U+FFFD>f in
split ligature scenarios, ensuring the second 'f' is properly skipped
during reconstruction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All 7 sub-phases (4.1-4.7) are now fully implemented:
- 4.1 Glyph to Span Merging: grouping consecutive glyphs into spans
- 4.2 Line Formation: baseline clustering and direction detection
- 4.3 Column Detection: histogram-based gap analysis
- 4.4 Block Formation: paragraph/heading/list/table/caption/figure/code classification
- 4.5 Reading Order: XY-cut algorithm with Docstrum fallback
- 4.6 Output Serialization: plain text projection with configurable filters
- 4.7 Text Readability: composite scoring and correction pipeline
Closes pdftract-4k1x4. Verification: notes/pdftract-4k1x4.md.
Changes:
- extract.rs: integrate Phase 4 modules into main pipeline
- layout/correction.rs: expand correction pipeline with 2048 lines of tests
- layout/readability.rs: five-signal scoring with char-weighted median
- text.rs: plain text serialization with page breaks and filters
- span/mod.rs: Span struct with flags and confidence tracking
- layout/columns.rs: column assignment to lines and spans
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
