From 30822aff02ab92df01580dc22d9556fefc5bed92 Mon Sep 17 00:00:00 2001 From: jedarden Date: Mon, 6 Jul 2026 17:46:59 -0400 Subject: [PATCH] docs(bf-5n4dp): document GLYPH_UNMAPPED diagnostic message patterns --- notes/bf-3ye57.md | 43 +++++++++ notes/bf-5n4dp.md | 216 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 259 insertions(+) create mode 100644 notes/bf-3ye57.md create mode 100644 notes/bf-5n4dp.md diff --git a/notes/bf-3ye57.md b/notes/bf-3ye57.md new file mode 100644 index 0000000..05823db --- /dev/null +++ b/notes/bf-3ye57.md @@ -0,0 +1,43 @@ +# Verification Note for bf-3ye57 + +## Task: Check content fuzz harness exists + +### Scope +Verify that the 'content' fuzz target harness exists in the project. + +### Implementation +Verified the fuzz harness exists and is properly configured. + +### Acceptance Criteria Results + +#### ✅ PASS: `cargo fuzz list` succeeds +Command executed successfully and returned the list of available fuzz targets: +- cmap_parser +- content +- lexer +- object_parser +- profile_yaml +- stream_decoder +- xref + +#### ✅ PASS: 'content' target appears in the output list +The 'content' fuzz target is present in the available targets list. + +#### ✅ PASS: Harness file exists at fuzz/fuzz_targets/content.rs +File verified: +- Path: `fuzz/fuzz_targets/content.rs` +- Size: 822 bytes +- Permissions: -rw-r--r-- (644) +- Last modified: 2026-07-05 18:05 + +### Harness Details + +The content fuzz harness is properly structured: +- Uses `libfuzzer_sys::fuzz_target!` macro +- Tests the content stream interpreter (`process_with_mode`) +- Validates INV-8 (no panic at public boundary) +- Tests both `ProcessingMode::Normal` and `ProcessingMode::PositionHint` +- Uses minimal empty `ResourceDict` for testing + +### Conclusion +All acceptance criteria PASS. The content fuzz harness exists and is correctly implemented. diff --git a/notes/bf-5n4dp.md b/notes/bf-5n4dp.md new file mode 100644 index 0000000..2fc49c5 --- /dev/null +++ b/notes/bf-5n4dp.md @@ -0,0 +1,216 @@ +# bf-5n4dp: GLYPH_UNMAPPED Diagnostic Message Patterns + +## Task +Identify and document GLYPH_UNMAPPED diagnostic message patterns in the pdftract codebase. + +## Analysis Summary + +The GLYPH_UNMAPPED diagnostic (`DiagCode::FontGlyphUnmapped`) is emitted when a character code cannot be resolved to Unicode through the 4-level fallback chain. The messages follow distinct patterns depending on the resolution context. + +## Pattern Structure + +### JSON Diagnostic Schema +All GLYPH_UNMAPPED diagnostics follow this JSON structure (from `crates/pdftract-core/src/schema/mod.rs:813-834`): + +```json +{ + "code": "FONT_GLYPH_UNMAPPED", + "message": "...", + "severity": "error", + "page_index": 0, + "location": null, + "hint": null +} +``` + +## Message Patterns + +### Pattern 1: Standard Font Miss (Most Common) +**Location**: `crates/pdftract-core/src/font/resolver.rs:693-699` + +**Format**: `Character code could not be resolved to Unicode (font ID: )` + +**Examples**: +``` +Character code 41 could not be resolved to Unicode (font ID: FontId(123456)) +Character code 0041 could not be resolved to Unicode (font ID: FontId(789012)) +Character code 01 could not be resolved to Unicode (font ID: FontId(345678)) +``` + +**Characteristics**: +- Character code is formatted as uppercase hex string (1 or more bytes) +- Font ID is a `FontId` struct displayed in debug format +- Emitted exactly once per (font_id, char_code) pair +- Emitted when all 4 levels fail (ToUnicode, AGL, Fingerprint, Shape) + +--- + +### Pattern 2: Type3 Font - Generic Failure +**Location**: `crates/pdftract-core/src/font/resolver.rs:567-571` + +**Format**: `Type3 font: character code 0x could not be resolved to Unicode` + +**Examples**: +``` +Type3 font: character code 0x41 could not be resolved to Unicode +Type3 font: character code 0x01 could not be resolved to Unicode +Type3 font: character code 0xFF could not be resolved to Unicode +``` + +**Characteristics**: +- Always single-byte (0x00-0xFF) for Type3 fonts +- Emitted when Type3-specific resolution chain fails +- Message is specific to Type3 font context + +--- + +### Pattern 3: Type3 Font - Shape Recognition Disabled +**Location**: `crates/pdftract-core/src/font/resolver.rs:578-583` + +**Format**: `Type3 font: character code 0x could not be resolved (shape recognition disabled)` + +**Examples**: +``` +Type3 font: character code 0x41 could not be resolved (shape recognition disabled) +Type3 font: character code 0x7F could not be resolved (shape recognition disabled) +``` + +**Characteristics**: +- Only emitted when `shape-db` feature is disabled +- Indicates Level 4 fallback is unavailable +- Suggests recompiling with shape recognition enabled + +--- + +### Pattern 4: Type3 Font - No Glyph Name +**Location**: `crates/pdftract-core/src/font/resolver.rs:618-624` + +**Format**: `Type3 font: character code 0x has no glyph name in encoding` + +**Examples**: +``` +Type3 font: character code 0x20 has no glyph name in encoding +Type3 font: character code 0x41 has no glyph name in encoding +``` + +**Characteristics**: +- Indicates the encoding dictionary has no entry for this code +- Prevents shape recognition (no glyph name to look up in /CharProcs) +- Resolution fails immediately with this message + +--- + +### Pattern 5: Type3 Font - Glyph Not in CharProcs +**Location**: `crates/pdftract-core/src/font/resolver.rs:632-638` + +**Format**: `Type3 font: glyph '' not found in /CharProcs for code 0x` + +**Examples**: +``` +Type3 font: glyph 'A' not found in /CharProcs for code 0x41 +Type3 font: glyph 'custom' not found in /CharProcs for code 0x01 +Type3 font: glyph 'g003' not found in /CharProcs for code 0xFF +``` + +**Characteristics**: +- Glyph name was present in encoding but missing from /CharProcs +- Indicates malformed Type3 font (orphan encoding entry) +- Includes both glyph name and character code + +--- + +### Pattern 6: Type3 Font - Rasterization Failed +**Location**: `crates/pdftract-core/src/font/resolver.rs:646-652` + +**Format**: `Type3 font: failed to rasterize glyph '' for code 0x` + +**Examples**: +``` +Type3 font: failed to rasterize glyph 'A' for code 0x41 +Type3 font: failed to rasterize glyph 'complex' for code 0x02 +``` + +**Characteristics**: +- Glyph exists in /CharProcs but content stream rendering failed +- Indicates corrupt or malformed content stream +- Rendering error occurred during bitmap generation + +--- + +### Pattern 7: Type3 Font - Shape Match Below Threshold +**Location**: `crates/pdftract-core/src/font/resolver.rs:668-674` + +**Format**: `Type3 font: shape match for '' (code 0x) found but distance exceeds threshold` + +**Examples**: +``` +Type3 font: shape match for 'A' (code 0x41) found but distance 12 exceeds threshold +Type3 font: shape match for 'glyph7' (code 0x07) found but distance 15 exceeds threshold +``` + +**Characteristics**: +- Shape match WAS found in database +- Hamming distance exceeded acceptable threshold (default: 8) +- Indicates "close but not confident enough" match +- Distance value provides quantitative closeness measure + +--- + +## Pattern Variations by Context + +### Regular Fonts (Type1, TrueType, CIDFont) +- **Only Pattern 1** applies +- Character codes can be multi-byte (e.g., `0041` for 2-byte CID) +- Font ID is always included + +### Type3 Fonts +- **Patterns 2-7** apply exclusively +- Character codes are always single-byte (0x00-0xFF) +- Font ID is NOT included (font is implicit from context) +- More detailed failure reasons (encoding, CharProcs, rasterization, shape match) + +## Edge Cases + +1. **Deduplication**: Standard font miss (Pattern 1) is emitted only once per (font_id, char_code) pair via `ResolverCache::emitted_misses` tracking + +2. **Multi-byte codes**: Only Pattern 1 supports multi-byte character codes (e.g., `0041`, `000101`). All Type3 patterns are single-byte only. + +3. **Conditional compilation**: Pattern 3 only appears when `shape-db` feature is disabled. Patterns 5-7 only appear when `shape-db` feature is enabled. + +4. **Hex formatting**: Pattern 1 uses concatenated hex without prefix (`41`, `0041`). Type3 patterns use `0x` prefix (`0x41`). + +## Emission Frequency + +| Pattern | Frequency | Deduplication | +|---------|-----------|----------------| +| Pattern 1 | Once per (font, code) | Yes (via cache) | +| Pattern 2 | Once per Type3 code failure | Yes (via cache) | +| Pattern 3 | Once per Type3 code failure | Yes (via cache) | +| Pattern 4 | Once per Type3 code failure | Yes (via cache) | +| Pattern 5 | Once per Type3 code failure | Yes (via cache) | +| Pattern 6 | Once per Type3 code failure | Yes (via cache) | +| Pattern 7 | Once per Type3 code failure | Yes (via cache) | + +## Code References + +- Emission logic: `crates/pdftract-core/src/font/resolver.rs` + - Regular fonts: `emit_miss_diagnostic()` (lines 681-708) + - Type3 fonts: `resolve_type3_level4()` (lines 588-679) +- Diagnostic code: `crates/pdftract-core/src/diagnostics.rs:17` +- JSON schema: `crates/pdftract-core/src/schema/mod.rs:813-834` +- Example usage: `crates/pdftract-core/src/output/json.rs:371` (test) + +## Verification + +Pattern identification based on: +1. Source code analysis of `resolver.rs` (all emission points) +2. JSON schema definition in `schema/mod.rs` +3. Test fixture at `tests/fixtures/encoding/no-mapping.pdf` (triggers Pattern 1) + +## Conclusion + +GLYPH_UNMAPPED diagnostics follow **7 distinct patterns**: +- **1 pattern** for regular fonts (character code + font ID) +- **6 patterns** for Type3 fonts (single-byte code, specific failure reason) + +All patterns share the same JSON diagnostic structure with `code: "FONT_GLYPH_UNMAPPED"` and `severity: "error"`, but messages vary by context to provide actionable debugging information.