# no-mapping.pdf Fixture

## Purpose

Level 4 Unicode recovery test fixture - worst case scenario where glyph names cannot be recovered via any standard method. This PDF uses custom glyph names (`/g001`, `/g002`, `/g003`) that are not in the Adobe Glyph List (AGL), has no ToUnicode CMap, and uses a custom encoding with no standard fallback.

Expected output: `���` (three U+FFFD replacement characters) since the glyph names have no mapping.

## Structure

### PDF Properties
- **PDF Version:** 1.4
- **Pages:** 1
- **Page Size:** 612 x 792 pts (Letter)
- **File Size:** 660 bytes
- **Encrypted:** No
- **Tagged:** No

### Font Details
```
Name: CustomNoMap
Type: Type 1
Encoding: Custom (Differences array)
Embedded: No
ToUnicode CMap: No
```

**Font Object (4 0 obj):**
```
<<
/Type /Font
/Subtype /Type1
/BaseFont /CustomNoMap
/Encoding <<
/Type /Encoding
/Differences [0 /g001 /g002 /g003]
>>
>>
```

### Content Stream

**Location:** Object 5 0 obj

**Raw Content:**
```
BT
/F1 12 Tf
50 700 Td
<g001><g002><g003> Tj
ET
```

**Explanation:**
- Sets font F1 (CustomNoMap) at 12pt
- Moves to position (50, 700)
- Shows three glyphs by name: g001, g002, g003
- These glyph names are NOT in the AGL and cannot be mapped to Unicode

### Glyph Names

The content uses three custom glyph names:
- `/g001` - No standard mapping (not in AGL)
- `/g002` - No standard mapping (not in AGL)
- `/g003` - No standard mapping (not in AGL)

Since these glyphs have no ToUnicode mapping and their names are not in any standard encoding or the AGL, they should render as U+FFFD (replacement character) during extraction.

## Inspection Commands

### Basic PDF info:
```bash
pdfinfo tests/fixtures/encoding/no-mapping.pdf
```

**Expected Output:**
```
Pages:           1
Page size:       612 x 792 pts (letter)
File size:       660 bytes
PDF version:     1.4
```

### Font details:
```bash
pdffonts tests/fixtures/encoding/no-mapping.pdf
```

**Expected Output:**
```
name                                 type              encoding         emb sub uni object ID
------------------------------------ ----------------- ---------------- --- --- --- ---------
CustomNoMap                          Type 1            Custom           no  no  no       5  0
```

### Content stream inspection:
```bash
# Extract raw content stream
pdftk tests/fixtures/encoding/no-mapping.pdf unpack_files output /tmp/no-mapping-unpack/
cat /tmp/no-mapping-unpack/doc_5_0.txt

# Or examine the hex dump
hexdump -C tests/fixtures/encoding/no-mapping.pdf | grep -A 5 "g001"
```

**Expected Content:**
```
<g001><g002><g003> Tj
```

### Test extraction:
```bash
# Build pdftract first if needed
cargo build --release

# Extract to see the U+FFFD replacement characters
./target/release/pdftract extract tests/fixtures/encoding/no-mapping.pdf \
  --format markdown -o /tmp/no-mapping-output
cat /tmp/no-mapping-output.md
```

## Regeneration Instructions

### Method 1: Using the Rust generator (Recommended)

The fixture is generated by `tests/fixtures/generate_encoding_fixtures.rs`.

**Steps:**
```bash
# Build the generator binary
cargo build --bin generate_encoding_fixtures

# Run the generator (creates all 4 encoding fixtures)
./target/debug/generate_encoding_fixtures

# Or run directly via cargo
cargo run --bin generate_encoding_fixtures
```

**Output location:** `tests/fixtures/encoding/no-mapping.pdf`

**Ground truth:** `tests/fixtures/encoding/no-mapping.txt` (should contain `���`)

### Method 2: Manual reconstruction (Not recommended)

To manually reconstruct this PDF:

1. Create the 5 PDF objects as specified in `generate_encoding_fixtures.rs`
2. Build the xref table with correct byte offsets
3. Add the trailer with `/Size 6` and `/Root 1 0 R`
4. Write the complete PDF to `tests/fixtures/encoding/no-mapping.pdf`

**Use the Rust generator instead** - it computes all offsets automatically and ensures the xref table is correct.

## Verification

After regeneration, verify:

```bash
# Check SHA256 matches expected value
sha256sum tests/fixtures/encoding/no-mapping.pdf
# Expected: b24f88d3add958bfec1d6b134f2cd030cd41bb1932bedbe99405599bd01fa8f0

# Verify extraction produces U+FFFD characters
./target/release/pdftract extract tests/fixtures/encoding/no-mapping.pdf \
  --format text -o /tmp/no-mapping-output.txt
cat /tmp/no-mapping-output.txt | od -c
# Should see: 0000000   f f f d   f f f d   f f f d
```

## Test References

This fixture is used in the following tests:

- `tests/debug_encoding_pdf.rs` - Encoding recovery debug tests
- `tests/debug_encoding_fixtures.rs` - Fixture validation tests
- `tests/encoding_recovery.rs` - Unicode recovery integration tests

## Related Fixtures

- `agl-only.pdf` - Uses standard Type1 font with AGL glyph names (Level 2)
- `fingerprint-match.pdf` - Uses embedded Type1 font subset (Level 3)
- `shape-match.pdf` - Uses custom glyph names with shape recognition (Level 4)

## History

- **Generated:** 2026-06-09
- **Regenerated:** 2026-07-02 (bf-512z1)
- **Regenerated:** 2026-07-03 (bf-f0xqd) - Corrected ground truth from "ABC" to U+FFFD
- **Regenerated:** 2026-07-03 (bf-1m30m) - Via `generate_encoding_fixtures.rs`
- **SHA256:** `b24f88d3add958bfec1d6b134f2cd030cd41bb1932bedbe99405599bd01fa8f0`

## Notes

The key insight of this fixture is that it represents the **worst-case scenario** for Unicode recovery:
1. No ToUnicode CMap
2. Custom encoding (no StandardEncoding/WinAnsiEncoding/MacRomanEncoding fallback)
3. Custom glyph names not in AGL (`/g001`, `/g002`, `/g003`)
4. No embedded font program for fingerprinting

This forces the extractor to either:
- Emit U+FFFD replacement characters (correct behavior)
- Attempt glyph shape recognition (if implemented)
- Fail gracefully

The fixture validates that pdftract handles unmapped glyphs correctly rather than producing garbage text or crashing.