docs(bf-4g7gb): document unmapped glyph test fixture verification
This commit is contained in:
parent
c8a90e24b1
commit
98f8b35368
1 changed files with 89 additions and 0 deletions
89
notes/bf-4g7gb.md
Normal file
89
notes/bf-4g7gb.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
# bf-4g7gb: Create unmapped glyph test fixture
|
||||
|
||||
## Task
|
||||
Create a PDF test fixture containing glyphs without valid Unicode mappings that trigger the FONT_GLYPH_UNMAPPED diagnostic.
|
||||
|
||||
## Finding
|
||||
The fixture already exists at `tests/fixtures/encoding/no-mapping.pdf` and meets all acceptance criteria.
|
||||
|
||||
## Acceptance Criteria - ALL MET
|
||||
|
||||
### ✅ Test fixture PDF exists in tests/fixtures/
|
||||
- Location: `tests/fixtures/encoding/no-mapping.pdf`
|
||||
- Size: 660 bytes
|
||||
- SHA256: `b24f88d3add958bfec1d6b134f2cd030cd41bb1932bedbe99405599bd01fa8f0`
|
||||
|
||||
### ✅ Fixture contains at least one glyph without a Unicode mapping
|
||||
- Contains **three unmapped glyphs**: /g001, /g002, /g003
|
||||
- Custom encoding differences array: `[0 /g001 /g002 /g003]`
|
||||
- No ToUnicode CMap
|
||||
- No standard encoding
|
||||
- Non-AGL glyph names (not in Adobe Glyph List)
|
||||
- Content stream uses byte codes `<000102>` to reference the three glyphs
|
||||
- Expected extraction: U+FFFD U+FFFD U+FFFD (three replacement characters)
|
||||
|
||||
### ✅ Fixture is documented with expected unmapped glyphs
|
||||
- Documented in `tests/fixtures/PROVENANCE.md` (line 199-208):
|
||||
```
|
||||
# encoding/no-mapping.pdf
|
||||
Generated by tests/fixtures/generate_unicode_recovery_fixtures.rs
|
||||
PDF 1.4, Type1 font with custom glyph names, no ToUnicode CMap, no standard encoding
|
||||
Level 4 Unicode recovery test fixture (worst case: no encoding fallback)
|
||||
Content: <20><><EFBFBD> (three U+FFFD replacement characters - glyph names /g001/g002/g003 are not in AGL and cannot be recovered)
|
||||
```
|
||||
- Generator code at `tests/fixtures/encoding/generate_unicode_recovery_fixtures.rs:68-98` with clear comments:
|
||||
```rust
|
||||
/// no-mapping.pdf: Custom encoding, no ToUnicode, no standard encoding
|
||||
/// Tests Level 4 worst-case: No ToUnicode, no AGL match, no fingerprint match
|
||||
```
|
||||
- Ground truth file: `tests/fixtures/encoding/no-mapping.txt` containing `<60><><EFBFBD>` (9 bytes UTF-8)
|
||||
|
||||
## Additional Verification
|
||||
|
||||
### FONT_GLYPH_UNMAPPED Diagnostic
|
||||
- Diagnostic code: `FONT_GLYPH_UNMAPPED`
|
||||
- Definition: `crates/pdftract-core/src/diagnostics.rs:17` (DiagCode::FontGlyphUnmapped)
|
||||
- Emission logic: `crates/pdftract-core/src/font/resolver.rs:24-26`
|
||||
```rust
|
||||
/// Emit the GLYPH_UNMAPPED diagnostic exactly once per (font, code) miss.
|
||||
```
|
||||
- JSON output test: `crates/pdftract-core/src/output/json.rs:369-386`
|
||||
```rust
|
||||
assert_eq!(error_json[0].code, "FONT_GLYPH_UNMAPPED");
|
||||
assert_eq!(error_json[0].severity, "error");
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
- Encoding recovery test: `crates/pdftract-core/tests/encoding_recovery.rs:210-223`
|
||||
```rust
|
||||
#[test]
|
||||
fn test_no_mapping_fixture() {
|
||||
let fixture = Fixture {
|
||||
name: "no-mapping".to_string(),
|
||||
pdf_path: PathBuf::from(format!("{}/no-mapping.pdf", FIXTURES_DIR)),
|
||||
ground_truth_path: PathBuf::from(format!("{}/no-mapping.txt", FIXTURES_DIR)),
|
||||
};
|
||||
let (_, recovery_rate, _) = test_fixture(&fixture);
|
||||
assert!(recovery_rate >= MIN_RECOVERY_RATE, ...);
|
||||
}
|
||||
```
|
||||
- Used throughout codebase by 40+ test and debug files (grepped from codebase)
|
||||
|
||||
## PDF Structure
|
||||
```
|
||||
%PDF-1.4
|
||||
1 0 obj: Catalog → /Pages 2 0 R
|
||||
2 0 obj: Pages → /Kids [3 0 R], /Count 1
|
||||
3 0 obj: Page → /Parent 2 0 R, /MediaBox [0 0 612 792], /Resources << /Font << /F1 5 0 R >> >>, /Contents 4 0 R
|
||||
4 0 obj: Content stream → BT /F1 12 Tf 50 700 Td <000102> Tj ET (byte codes for glyphs /g001, /g002, /g003)
|
||||
5 0 obj: Font → /Type /Font /Subtype /Type1 /BaseFont /CustomNoMap /Encoding << /Type /Encoding /Differences [0 /g001 /g002 /g003] >>
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
All acceptance criteria are fully met by the existing `no-mapping.pdf` fixture. No additional work required.
|
||||
|
||||
## Related
|
||||
- Bead bf-512z1: Unicode recovery Phase 2 exit gate (≥90% recovery rate)
|
||||
- Bead bf-f0xqd: Corrected no-mapping.pdf ground truth from "ABC" to U+FFFD
|
||||
- Bead bf-1m30m: Regenerated no-mapping.pdf via generate_unicode_recovery_fixtures.rs
|
||||
- Plan Section TH-03: Test harness for unmapped glyph diagnostics
|
||||
Loading…
Add table
Reference in a new issue