diff --git a/notes/bf-4g7gb.md b/notes/bf-4g7gb.md new file mode 100644 index 0000000..47aed8d --- /dev/null +++ b/notes/bf-4g7gb.md @@ -0,0 +1,89 @@ +# bf-4g7gb: Create unmapped glyph test fixture + +## Task +Create a PDF test fixture containing glyphs without valid Unicode mappings that trigger the FONT_GLYPH_UNMAPPED diagnostic. + +## Finding +The fixture already exists at `tests/fixtures/encoding/no-mapping.pdf` and meets all acceptance criteria. + +## Acceptance Criteria - ALL MET + +### ✅ Test fixture PDF exists in tests/fixtures/ +- Location: `tests/fixtures/encoding/no-mapping.pdf` +- Size: 660 bytes +- SHA256: `b24f88d3add958bfec1d6b134f2cd030cd41bb1932bedbe99405599bd01fa8f0` + +### ✅ Fixture contains at least one glyph without a Unicode mapping +- Contains **three unmapped glyphs**: /g001, /g002, /g003 +- Custom encoding differences array: `[0 /g001 /g002 /g003]` +- No ToUnicode CMap +- No standard encoding +- Non-AGL glyph names (not in Adobe Glyph List) +- Content stream uses byte codes `<000102>` to reference the three glyphs +- Expected extraction: U+FFFD U+FFFD U+FFFD (three replacement characters) + +### ✅ Fixture is documented with expected unmapped glyphs +- Documented in `tests/fixtures/PROVENANCE.md` (line 199-208): + ``` + # encoding/no-mapping.pdf + Generated by tests/fixtures/generate_unicode_recovery_fixtures.rs + PDF 1.4, Type1 font with custom glyph names, no ToUnicode CMap, no standard encoding + Level 4 Unicode recovery test fixture (worst case: no encoding fallback) + Content: ��� (three U+FFFD replacement characters - glyph names /g001/g002/g003 are not in AGL and cannot be recovered) + ``` +- Generator code at `tests/fixtures/encoding/generate_unicode_recovery_fixtures.rs:68-98` with clear comments: + ```rust + /// no-mapping.pdf: Custom encoding, no ToUnicode, no standard encoding + /// Tests Level 4 worst-case: No ToUnicode, no AGL match, no fingerprint match + ``` +- Ground truth file: `tests/fixtures/encoding/no-mapping.txt` containing `���` (9 bytes UTF-8) + +## Additional Verification + +### FONT_GLYPH_UNMAPPED Diagnostic +- Diagnostic code: `FONT_GLYPH_UNMAPPED` +- Definition: `crates/pdftract-core/src/diagnostics.rs:17` (DiagCode::FontGlyphUnmapped) +- Emission logic: `crates/pdftract-core/src/font/resolver.rs:24-26` + ```rust + /// Emit the GLYPH_UNMAPPED diagnostic exactly once per (font, code) miss. + ``` +- JSON output test: `crates/pdftract-core/src/output/json.rs:369-386` + ```rust + assert_eq!(error_json[0].code, "FONT_GLYPH_UNMAPPED"); + assert_eq!(error_json[0].severity, "error"); + ``` + +### Integration Tests +- Encoding recovery test: `crates/pdftract-core/tests/encoding_recovery.rs:210-223` + ```rust + #[test] + fn test_no_mapping_fixture() { + let fixture = Fixture { + name: "no-mapping".to_string(), + pdf_path: PathBuf::from(format!("{}/no-mapping.pdf", FIXTURES_DIR)), + ground_truth_path: PathBuf::from(format!("{}/no-mapping.txt", FIXTURES_DIR)), + }; + let (_, recovery_rate, _) = test_fixture(&fixture); + assert!(recovery_rate >= MIN_RECOVERY_RATE, ...); + } + ``` +- Used throughout codebase by 40+ test and debug files (grepped from codebase) + +## PDF Structure +``` +%PDF-1.4 +1 0 obj: Catalog → /Pages 2 0 R +2 0 obj: Pages → /Kids [3 0 R], /Count 1 +3 0 obj: Page → /Parent 2 0 R, /MediaBox [0 0 612 792], /Resources << /Font << /F1 5 0 R >> >>, /Contents 4 0 R +4 0 obj: Content stream → BT /F1 12 Tf 50 700 Td <000102> Tj ET (byte codes for glyphs /g001, /g002, /g003) +5 0 obj: Font → /Type /Font /Subtype /Type1 /BaseFont /CustomNoMap /Encoding << /Type /Encoding /Differences [0 /g001 /g002 /g003] >> +``` + +## Conclusion +All acceptance criteria are fully met by the existing `no-mapping.pdf` fixture. No additional work required. + +## Related +- Bead bf-512z1: Unicode recovery Phase 2 exit gate (≥90% recovery rate) +- Bead bf-f0xqd: Corrected no-mapping.pdf ground truth from "ABC" to U+FFFD +- Bead bf-1m30m: Regenerated no-mapping.pdf via generate_unicode_recovery_fixtures.rs +- Plan Section TH-03: Test harness for unmapped glyph diagnostics