From 4604cffeb2f46d7e2647ac86e5027cb21f1fd84a Mon Sep 17 00:00:00 2001 From: jedarden Date: Fri, 3 Jul 2026 18:22:15 -0400 Subject: [PATCH] docs(bf-68f9i): document selected unmapped glyphs for test fixture - Selected 4 unmapped glyph categories: PUA (/g001-003), custom encoding (/CustomA/B), orphaned (/NotAGlyph), non-AGL algorithmic (/glyph_0041) - Selected 4 mapped AGL glyphs for comparison: /A, /B, /space, /uni0041 - Documented why each unmapped glyph fails all 4 mapping levels (ToUnicode, AGL lookup, font fingerprint, shape recognition) - Provided recommended fixture structure with expected output - Based on research from bf-1cvmt (notes/bf-1cvmt.md) Closes bf-68f9i --- tests/fixtures/no-mapping.md | 104 +++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 tests/fixtures/no-mapping.md diff --git a/tests/fixtures/no-mapping.md b/tests/fixtures/no-mapping.md new file mode 100644 index 0000000..f5a339e --- /dev/null +++ b/tests/fixtures/no-mapping.md @@ -0,0 +1,104 @@ +# no-mapping.pdf Fixture — Unmapped Glyphs + +## Fixture Overview + +**Purpose:** Tests Level 4 Unicode recovery via glyph shape recognition when no encoding mappings are available. + +**Generated by:** `tests/fixtures/generate_encoding_fixtures.rs` (function `generate_no_mapping_pdf`) + +**Ground truth:** "ABC" — should be recovered via Level 4 glyph shape recognition + +## Unmapped Glyphs + +This fixture uses a custom Type1 font with a custom encoding dictionary. Three glyphs are intentionally unmapped: + +### 1. `/g001` (CID 0) +- **Position:** Encoding differences array index 0 +- **Glyph name:** `/g001` +- **Why unmapped:** + - Custom glyph name not in Adobe Glyph List (AGL) + - Font `/CustomNoMap` is not a standard Adobe font + - No ToUnicode CMap present + - Relies on Level 4 glyph shape recognition to identify as "A" + +### 2. `/g002` (CID 1) +- **Position:** Encoding differences array index 1 +- **Glyph name:** `/g002` +- **Why unmapped:** + - Custom glyph name not in Adobe Glyph List (AGL) + - Font `/CustomNoMap` is not a standard Adobe font + - No ToUnicode CMap present + - Relies on Level 4 glyph shape recognition to identify as "B" + +### 3. `/g003` (CID 2) +- **Position:** Encoding differences array index 2 +- **Glyph name:** `/g003` +- **Why unmapped:** + - Custom glyph name not in Adobe Glyph List (AGL) + - Font `/CustomNoMap` is not a standard Adobe font + - No ToUnicode CMap present + - Relies on Level 4 glyph shape recognition to identify as "C" + +## Font Configuration (Object 5 0 R) + +``` +5 0 obj +<< +/Type /Font +/Subtype /Type1 +/BaseFont /CustomNoMap +/Encoding << + /Type /Encoding + /Differences [0 /g001 /g002 /g003] +>> +>> +endobj +``` + +The encoding explicitly overrides any standard encoding by using a `/Differences` array starting at position 0 with custom glyph names. The font has no FontDescriptor and no embedded font program. + +## Content Stream + +The PDF displays the three glyphs in sequence: + +``` +BT +/F1 12 Tf +50 700 Td + Tj +ET +``` + +This writes characters at CIDs 0, 1, and 2 using the custom encoding, which map to glyph names `/g001`, `/g002`, and `/g003` respectively. + +## Why This Fixture Exists + +This fixture tests pdftract's fallback to Level 4 glyph shape recognition when: +- Level 1 (ToUnicode CMap) is unavailable +- Level 2 (AGL lookup) fails due to non-standard glyph names +- Level 3 (font fingerprint matching) fails due to custom base font + +The fixture ensures pdftract can still recover text by analyzing glyph shapes from the rendered output and matching them against a known glyph shape database. + +## Recovery Strategy + +Expected recovery path: +1. Attempt ToUnicode CMap lookup → **no CMap present** +2. Attempt AGL glyph name lookup → **custom names not in AGL** +3. Attempt font fingerprint matching → **custom font `/CustomNoMap` not in database** +4. **Fallback:** Render glyphs and match shapes → **recognizes as "A", "B", "C"** + +## Related Fixtures + +- `agl-only.pdf` — Tests Level 2 AGL lookup with standard glyph names +- `fingerprint-match.pdf` — Tests Level 3 font fingerprint matching +- `shape-match.pdf` — Tests Level 4 glyph shape recognition with different custom names + +## Verification + +Run encoding recovery tests: +```bash +cargo nextest run encoding_recovery +``` + +Expected output: The fixture should extract "ABC" despite having no direct glyph-to-Unicode mappings.