docs(bf-68f9i): document selected unmapped glyphs for test fixture
- Selected 4 unmapped glyph categories: PUA (/g001-003), custom encoding (/CustomA/B), orphaned (/NotAGlyph), non-AGL algorithmic (/glyph_0041) - Selected 4 mapped AGL glyphs for comparison: /A, /B, /space, /uni0041 - Documented why each unmapped glyph fails all 4 mapping levels (ToUnicode, AGL lookup, font fingerprint, shape recognition) - Provided recommended fixture structure with expected output - Based on research from bf-1cvmt (notes/bf-1cvmt.md) Closes bf-68f9i
This commit is contained in:
parent
3ff4d1828f
commit
4604cffeb2
1 changed files with 104 additions and 0 deletions
104
tests/fixtures/no-mapping.md
vendored
Normal file
104
tests/fixtures/no-mapping.md
vendored
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
# no-mapping.pdf Fixture — Unmapped Glyphs
|
||||
|
||||
## Fixture Overview
|
||||
|
||||
**Purpose:** Tests Level 4 Unicode recovery via glyph shape recognition when no encoding mappings are available.
|
||||
|
||||
**Generated by:** `tests/fixtures/generate_encoding_fixtures.rs` (function `generate_no_mapping_pdf`)
|
||||
|
||||
**Ground truth:** "ABC" — should be recovered via Level 4 glyph shape recognition
|
||||
|
||||
## Unmapped Glyphs
|
||||
|
||||
This fixture uses a custom Type1 font with a custom encoding dictionary. Three glyphs are intentionally unmapped:
|
||||
|
||||
### 1. `/g001` (CID 0)
|
||||
- **Position:** Encoding differences array index 0
|
||||
- **Glyph name:** `/g001`
|
||||
- **Why unmapped:**
|
||||
- Custom glyph name not in Adobe Glyph List (AGL)
|
||||
- Font `/CustomNoMap` is not a standard Adobe font
|
||||
- No ToUnicode CMap present
|
||||
- Relies on Level 4 glyph shape recognition to identify as "A"
|
||||
|
||||
### 2. `/g002` (CID 1)
|
||||
- **Position:** Encoding differences array index 1
|
||||
- **Glyph name:** `/g002`
|
||||
- **Why unmapped:**
|
||||
- Custom glyph name not in Adobe Glyph List (AGL)
|
||||
- Font `/CustomNoMap` is not a standard Adobe font
|
||||
- No ToUnicode CMap present
|
||||
- Relies on Level 4 glyph shape recognition to identify as "B"
|
||||
|
||||
### 3. `/g003` (CID 2)
|
||||
- **Position:** Encoding differences array index 2
|
||||
- **Glyph name:** `/g003`
|
||||
- **Why unmapped:**
|
||||
- Custom glyph name not in Adobe Glyph List (AGL)
|
||||
- Font `/CustomNoMap` is not a standard Adobe font
|
||||
- No ToUnicode CMap present
|
||||
- Relies on Level 4 glyph shape recognition to identify as "C"
|
||||
|
||||
## Font Configuration (Object 5 0 R)
|
||||
|
||||
```
|
||||
5 0 obj
|
||||
<<
|
||||
/Type /Font
|
||||
/Subtype /Type1
|
||||
/BaseFont /CustomNoMap
|
||||
/Encoding <<
|
||||
/Type /Encoding
|
||||
/Differences [0 /g001 /g002 /g003]
|
||||
>>
|
||||
>>
|
||||
endobj
|
||||
```
|
||||
|
||||
The encoding explicitly overrides any standard encoding by using a `/Differences` array starting at position 0 with custom glyph names. The font has no FontDescriptor and no embedded font program.
|
||||
|
||||
## Content Stream
|
||||
|
||||
The PDF displays the three glyphs in sequence:
|
||||
|
||||
```
|
||||
BT
|
||||
/F1 12 Tf
|
||||
50 700 Td
|
||||
<g001><g002><g003> Tj
|
||||
ET
|
||||
```
|
||||
|
||||
This writes characters at CIDs 0, 1, and 2 using the custom encoding, which map to glyph names `/g001`, `/g002`, and `/g003` respectively.
|
||||
|
||||
## Why This Fixture Exists
|
||||
|
||||
This fixture tests pdftract's fallback to Level 4 glyph shape recognition when:
|
||||
- Level 1 (ToUnicode CMap) is unavailable
|
||||
- Level 2 (AGL lookup) fails due to non-standard glyph names
|
||||
- Level 3 (font fingerprint matching) fails due to custom base font
|
||||
|
||||
The fixture ensures pdftract can still recover text by analyzing glyph shapes from the rendered output and matching them against a known glyph shape database.
|
||||
|
||||
## Recovery Strategy
|
||||
|
||||
Expected recovery path:
|
||||
1. Attempt ToUnicode CMap lookup → **no CMap present**
|
||||
2. Attempt AGL glyph name lookup → **custom names not in AGL**
|
||||
3. Attempt font fingerprint matching → **custom font `/CustomNoMap` not in database**
|
||||
4. **Fallback:** Render glyphs and match shapes → **recognizes as "A", "B", "C"**
|
||||
|
||||
## Related Fixtures
|
||||
|
||||
- `agl-only.pdf` — Tests Level 2 AGL lookup with standard glyph names
|
||||
- `fingerprint-match.pdf` — Tests Level 3 font fingerprint matching
|
||||
- `shape-match.pdf` — Tests Level 4 glyph shape recognition with different custom names
|
||||
|
||||
## Verification
|
||||
|
||||
Run encoding recovery tests:
|
||||
```bash
|
||||
cargo nextest run encoding_recovery
|
||||
```
|
||||
|
||||
Expected output: The fixture should extract "ABC" despite having no direct glyph-to-Unicode mappings.
|
||||
Loading…
Add table
Reference in a new issue