# PDF Fonts and Encoding: Technical Reference for Text Extraction

This document describes every font type found in PDF files, how character codes are decoded to Unicode, and the data structures a Rust extraction engine must interpret. References are to the PDF 1.7 specification (ISO 32000-1:2008) and Adobe technical notes where applicable.

---

## 1. Font Types

### 1.1 Type 1 (Simple Font)

Type 1 fonts originate from the Adobe Type 1 format, stored as PFB (binary) or PFA (ASCII) font programs. In a PDF the font dictionary has `/Subtype /Type1`.

**Glyph storage.** The font program is a PostScript charstring program. When embedded, it appears under `/FontDescriptor` as the stream value of `/FontFile` (Type 1 binary). The charstrings are keyed by glyph name, not by a numeric glyph ID.

**Character code interpretation.** A one-byte character code from the content stream is mapped through the font's `/Encoding` to a glyph name, then the glyph name is looked up in the charstring dictionary. See §3 for encoding details.

**Widths.** The `/Widths` array (required) contains `LastChar - FirstChar + 1` entries, each giving the horizontal advance width in text-space units (1/1000 em). `/FirstChar` and `/LastChar` define the range. Codes outside this range use `/MissingWidth` from the font descriptor.

**Standard 14 fonts.** PDF readers must implement the 14 standard Type 1 fonts (Helvetica, Times-Roman, Courier, Symbol, ZapfDingbats, and their variants) without an embedded font program. These are never embedded; the reader synthesizes metrics.

### 1.2 Type 3 (Simple Font)

`/Subtype /Type3`. Glyphs are defined as PDF content streams directly in the font dictionary under `/CharProcs`, a dictionary from glyph name to stream. There is no external font program.

**Character code interpretation.** One-byte code → glyph name via `/Encoding` → content stream in `/CharProcs`. Because glyph names are arbitrary (user-defined), there is often no reliable path to Unicode without a `/ToUnicode` CMap. If `/ToUnicode` is absent, extraction must fall back to glyph name heuristics or report the text as unresolvable.

**Widths.** `/Widths`, `/FirstChar`, `/LastChar` as in Type 1. Additionally, `/FontMatrix` transforms glyph-space coordinates; the default for Type 1 is `[0.001 0 0 0.001 0 0]`, but Type 3 fonts frequently use `[1 0 0 1 0 0]` with glyph streams drawn at full size.

### 1.3 TrueType (Simple Font)

`/Subtype /TrueType`. The embedded program is a TrueType font binary under `/FontFile2` in the font descriptor.

**Glyph storage.** Glyphs are stored by integer glyph ID (GID) inside the `glyf` table. The `cmap` table maps Unicode codepoints (or platform-specific codes) to GIDs.

**Character code interpretation.** One-byte code → glyph name via `/Encoding` → GID via the font's `cmap`. When the encoding is a standard PDF encoding (WinAnsiEncoding, MacRomanEncoding, etc.), the implementation maps code → Unicode codepoint → GID using `cmap` platform/encoding subtable (platform 3, encoding 1: Windows Unicode BMP). If the font's `cmap` contains only platform 1 (Macintosh), platform-specific code mappings apply. This is a common source of extraction errors.

**Widths.** Same `/Widths` array mechanism as Type 1. The `hmtx` TrueType table provides the authoritative advance widths; the PDF `/Widths` array should match but may differ in broken documents.

### 1.4 Type 0 (Composite Font)

`/Subtype /Type0`. This is the container for multi-byte (CJK and other large character set) text. The font dictionary has:

- `/Encoding` — a CMap name (e.g., `Identity-H`) or a stream containing a CMap program.
- `/DescendantFonts` — a one-element array holding a CIDFont dictionary.

**Character code interpretation.** The multi-byte content stream codes are fed through the CMap named in `/Encoding`, which maps character codes to CIDs. The CIDFont then maps CIDs to GIDs. See §4.

**Widths.** Widths are specified in the CIDFont descendant, not in the Type 0 dictionary itself.

### 1.5 CIDFont Type 0 (CFF-Based)

`/Subtype /CIDFontType0` inside a `/DescendantFonts` array. The font program is a CFF (Compact Font Format, also called Type 2 charstrings) font embedded under `/FontFile3` with `/Subtype /CIDFontType0C` or `/Subtype /OpenType`.

**Glyph storage.** CFF stores charstrings keyed by GID (integer index). GIDs map directly to charstrings; glyph names may or may not be present depending on the CFF variant.

**Widths.** The CIDFont dictionary uses `/DW` (default width, default 1000) and `/W` (array of per-CID widths). The `/W` syntax is: an array whose elements alternate between `c [w1 w2 ...]` (individual CIDs) and `c1 c2 w` (range with uniform width).

### 1.6 CIDFont Type 2 (TrueType-Based)

`/Subtype /CIDFontType2`. The embedded program is a TrueType or OpenType/TT font under `/FontFile2` (TrueType) or `/FontFile3` with `/Subtype /OpenType`.

**CID-to-GID mapping.** The `/CIDToGIDMap` entry in the CIDFont dictionary is critical:
- If the value is the name `/Identity`, CID equals GID directly (CID = GID).
- Otherwise it is a stream of 2×65536 bytes: the GID for CID `n` is the 16-bit big-endian value at byte offset `2n`.

**Widths.** Same `/DW` and `/W` mechanism as CIDFont Type 0.

### 1.7 OpenType in PDF

OpenType fonts are embedded as `/FontFile3` streams with `/Subtype /OpenType`. An OpenType font may contain either CFF outlines (`CFF` table present → CIDFont Type 0) or TrueType outlines (`glyf` table present → CIDFont Type 2). The handling follows the respective CIDFont rules. The PDF spec does not treat OpenType as a separate subtype; it is identified by the stream subtype.

---

## 2. Encoding Mechanisms

### 2.1 Predefined Encodings

The PDF spec defines four named encodings for simple fonts (§D.1–D.4, PDF 1.7):

| Name | Character set | Typical use |
|------|--------------|-------------|
| `StandardEncoding` | 229 glyphs from the Adobe standard | Default for Type 1 fonts that omit `/Encoding` |
| `MacRomanEncoding` | Mac OS Roman 256 code points | Older Mac-generated PDFs |
| `WinAnsiEncoding` | Windows-1252 (cp1252) | Windows-generated PDFs; most common |
| `MacExpertEncoding` | Expert font character set (fractions, small caps) | Rare; expert-set fonts |

`PDFDocEncoding` is a PDF-internal encoding used for text strings in the document catalog (info dictionary, annotations) but **not** for font encoding; it must not be confused with font encodings. It extends Latin-1 by filling 0x18–0x1F and 0x80–0x9F with additional characters.

`Symbol` and `ZapfDingbats` fonts use built-in symbol encodings defined in the respective AFM files. They do **not** use the standard named encodings; their code-to-glyph mapping is private and must be looked up against the font-specific tables provided in PDF Annex D.

### 2.2 The `/Encoding` Dictionary and `/Differences` Array

When a font's `/Encoding` value is a dictionary rather than a name, the dictionary may contain:

- `/Type /Encoding` (optional)
- `/BaseEncoding` — a name (`StandardEncoding`, `MacRomanEncoding`, `WinAnsiEncoding`) designating the starting table. If absent, the base depends on font type (Type 1 defaults to built-in; others to StandardEncoding).
- `/Differences` — an array of the form `[code name code name ...]` or `[code name name name ...]`. Starting from the numeric code, each following name overrides successive slots. Example: `[32 /space /exclam /quotedbl]` overrides slots 32, 33, 34.

Encoding resolution algorithm for simple fonts:
1. Start from the BaseEncoding table.
2. Apply each `/Differences` entry, replacing the glyph name at the given code position.
3. Resolve each resulting glyph name to Unicode via the Adobe Glyph List (§5).

### 2.3 Symbol and ZapfDingbats

These two standard fonts carry the `Symbolic` flag (bit 3 of `/Flags` in the font descriptor). Their encoding is defined entirely by the glyph names in the font program; the predefined named encodings do not apply. Extraction must use the AGL or the font's own encoding vector. ZapfDingbats glyph names are documented in the PDF spec Annex D.6.

---

## 3. ToUnicode CMaps

### 3.1 CMap Stream Format

A ToUnicode CMap is a PostScript-inspired stream embedded directly in the PDF. The structure (PDF §9.10.3):

```
/CIDInit /ProcSet findresource begin
12 dict begin
begincmap
/CIDSystemInfo 3 dict dup begin
  /Registry (Adobe) def
  /Ordering (UCS) def
  /Supplement 0 def
end def
/CMapName /Adobe-Identity-UCS def
/CMapType 2 def
4 beginbfchar
<0041> <0041>          % code 0x41 → U+0041 (A)
<00A0> <00A0>
<F001> <FB01>          % code 0xF001 → U+FB01 (fi ligature)
<F002> <FB02>          % code 0xF002 → U+FB02 (fl ligature)
endbfchar
1 beginbfrange
<0061> <007A> <0061>   % codes 0x61–0x7A → U+0061–U+007A (a–z)
endbfrange
endcmap
CMapName currentdict /CMap defineresource pop
end
end
```

**`beginbfchar` / `endbfchar`:** Each entry is a pair `<src-code> <dst-unicode>`. The destination is UTF-16BE hex bytes; a surrogate pair encodes a codepoint above U+FFFF.

**`beginbfrange` / `endbfrange`:** Range `<lo> <hi> <start-unicode>` maps a contiguous code range to a contiguous Unicode range. Alternatively, `<lo> <hi> [<u1> <u2> ...]` maps each code in the range to the corresponding Unicode string in the array.

**`begincidrange` / `endcidrange`:** Used in Type 0 CMaps (not ToUnicode) to map codes to CID ranges; see §4.

### 3.2 Embedding in PDF

The ToUnicode CMap appears as the value of the `/ToUnicode` key in the font dictionary (both simple and composite fonts). It is a stream object, usually with `/Filter /FlateDecode`.

### 3.3 When ToUnicode is Absent or Wrong

**Absent:** Extraction must fall back to encoding → glyph name → AGL lookup (simple fonts) or CID-to-Unicode tables derived from the predefined CMap ordering (composite fonts). Many PDFs produced by older tools (TeX-based pipelines, some CAD exporters) omit `/ToUnicode`; the AGL fallback is the only reliable option.

**Wrong or incomplete:** Some generators emit a `/ToUnicode` CMap with missing entries or incorrect mappings. A bfchar entry with destination `<0000>` or `<FFFD>` signals an intentionally unmapped glyph. An implementation should not blindly trust all mappings; NUL and replacement-character destinations should be treated as absent.

**Implications for extraction:** Without a `/ToUnicode` map, ligature glyphs (`fi`, `fl`, `ffi`, etc.) will be decoded as their AGL expansions (multi-character strings), which is usually correct. Private Use Area (PUA) codepoints require a `/ToUnicode` map to resolve; without one the extracted text should preserve the PUA codepoint but flag it as unresolved.

---

## 4. CID-to-GID Mapping (Composite Fonts)

### 4.1 Decoding Path

For a Type 0 composite font, the decoding pipeline is:

```
content-stream bytes
    → CMap (named in /Encoding)
    → CID
    → GID (via CIDToGIDMap or CFF index)
    → glyph outline
```

The `/Encoding` CMap converts multi-byte character codes (1–4 bytes) to CIDs. The CMap may be:
- A name referring to a predefined CMap (see §4.2).
- A stream object containing a CMap program.

### 4.2 Predefined CMaps

Adobe distributes predefined CMaps for CJK encodings (PDF Annex M). Key examples:

| Name | Script | Code space | Notes |
|------|--------|-----------|-------|
| `Identity-H` | any (horizontal) | 2-byte | CID = code (identity) |
| `Identity-V` | any (vertical) | 2-byte | CID = code, vertical writing |
| `90ms-RKSJ-H` | Japanese | Shift-JIS | Maps SJIS codes → Adobe-Japan1 CIDs |
| `GBK-EUC-H` | Simplified Chinese | GBK/EUC | Maps GBK → Adobe-GB1 CIDs |
| `UniGB-UTF16-H` | Simplified Chinese | UTF-16BE | Unicode input → Adobe-GB1 CIDs |
| `UniJIS-UTF16-H` | Japanese | UTF-16BE | Unicode input → Adobe-Japan1 CIDs |

For `Identity-H`/`Identity-V`, the CID equals the raw 2-byte code value, and if `/CIDToGIDMap /Identity`, the GID equals the CID. These are the simplest cases for TrueType-based CIDFonts.

### 4.3 CIDSystemInfo

Every CIDFont and its associated CMap must declare `/CIDSystemInfo`, a dictionary with `/Registry` (string), `/Ordering` (string), and `/Supplement` (integer). This identifies the CID character collection, e.g., Adobe-Japan1-6. The CIDFont and its CMap must share the same Registry and Ordering. Implementations should use this to select fallback Unicode tables when `/ToUnicode` is absent (Adobe publishes CID→Unicode mappings for its standard collections).

---

## 5. Glyph Name to Unicode (Adobe Glyph List)

### 5.1 The AGL

The Adobe Glyph List (AGL, `aglfn.txt`, version 1.7) maps glyph names to Unicode scalar values. An implementation should embed the AGL as a static hash table (approximately 4,000 entries).

**Algorithmic fallback** (AGL specification §2): If a glyph name is not in the AGL table:
1. Strip any trailing `.<suffix>` (e.g., `A.sc` → `A`).
2. If the name starts with `uni`, parse the following hex digits as UTF-16BE codepoint(s): `uni0041` → U+0041.
3. If the name starts with `u`, parse the following hex as a Unicode scalar: `u1F600` → U+1F600.
4. If none of the above, the glyph is unmapped.

**Ligatures.** `fi` → U+FB01, `fl` → U+FB02, `ffi` → U+FB03, `ffl` → U+FB04. These are single AGL entries mapping to single Unicode codepoints. Many extraction engines prefer to expand ligatures to their component characters (fi → "fi") for searchability; this is a policy choice, not a spec requirement.

**`.notdef`.** The glyph named `.notdef` is the fallback glyph for unmapped codes. It has no Unicode mapping. Extractors should silently skip or emit U+FFFD for `.notdef`.

**`afii` names.** Legacy glyph names starting with `afii` (e.g., `afii57506`) appear in older Arabic and Hebrew fonts. The AGL maps these to their correct Unicode codepoints; no special handling beyond AGL lookup is needed.

---

## 6. Font Descriptors

The `/FontDescriptor` dictionary (§9.8, PDF 1.7) is referenced by the font dictionary via `/FontDescriptor`. It provides metrics and the embedded font binary.

### 6.1 Key Entries

| Key | Type | Description |
|-----|------|-------------|
| `/FontName` | name | PostScript name of the font |
| `/FontBBox` | rectangle | Glyph bounding box in glyph-space units |
| `/Flags` | integer | Bitfield describing font characteristics |
| `/ItalicAngle` | number | Dominant italic angle in degrees |
| `/Ascent` | number | Maximum ascent above baseline |
| `/Descent` | number | Maximum descent below baseline (negative) |
| `/CapHeight` | number | Height of capital letters |
| `/XHeight` | number | Height of lowercase letters |
| `/StemV` | number | Dominant vertical stem width |
| `/FontFile` | stream | Type 1 PFB data |
| `/FontFile2` | stream | TrueType binary |
| `/FontFile3` | stream | CFF, OpenType, or CIDFontType0C binary (identified by stream `/Subtype`) |

### 6.2 Flags Bitfield

The `/Flags` integer is a 32-bit field; bits are numbered from 1 (LSB). Key bits:

| Bit | Mask | Meaning |
|-----|------|---------|
| 1 | 0x0001 | FixedPitch |
| 2 | 0x0002 | Serif |
| 3 | 0x0004 | Symbolic — font uses a private encoding; standard encodings do not apply |
| 4 | 0x0008 | Script (cursive) |
| 6 | 0x0020 | Nonsymbolic — font uses a standard Latin encoding |
| 7 | 0x0040 | Italic |
| 17 | 0x10000 | AllCap |
| 18 | 0x20000 | SmallCap |
| 19 | 0x40000 | ForceBold |

The `Symbolic` (bit 3) and `Nonsymbolic` (bit 6) flags are mutually exclusive and affect encoding resolution: a symbolic font's encoding is its own built-in table; a nonsymbolic font follows the standard named encoding fallback rules.

### 6.3 Inferring Unicode When CMap Data Is Absent

When both `/ToUnicode` and a useful `/Encoding` are missing, the following heuristics apply, in order:
1. If the embedded font is TrueType (`/FontFile2`) and the `/Flags` `Nonsymbolic` bit is set, use the font's `cmap` table with the `WinAnsiEncoding` assumption (platform 3, encoding 1).
2. If the font is CFF (`/FontFile3` with `/Subtype /CIDFontType0C`), the CFF `charset` table may supply glyph names; apply AGL.
3. If `/FontName` identifies a known standard font (e.g., `Symbol`, `ZapfDingbats`), apply the font-specific encoding table from PDF Annex D.
4. Otherwise, emit PUA codepoints or U+FFFD and flag the text as requiring post-processing.

The font descriptor `/FontBBox` and `/Flags` provide no path to Unicode; they are useful only for layout heuristics (detecting whitespace, line boundaries) when Unicode resolution fails.

---

## Appendix: Key Dictionary Locations

```
/Font dictionary
  /Subtype               → Type1 | Type3 | TrueType | Type0 | CIDFontType0 | CIDFontType2
  /Encoding              → name or dictionary (simple); CMap name or stream (Type0)
  /ToUnicode             → stream (CMap program)
  /FontDescriptor        → dictionary
    /Flags               → integer (bitfield)
    /FontFile            → stream (Type 1)
    /FontFile2           → stream (TrueType)
    /FontFile3           → stream (CFF/OpenType; /Subtype in stream dict)
  /Widths                → array (simple fonts)
  /FirstChar             → integer
  /LastChar              → integer
  /DescendantFonts       → array [ CIDFont dict ] (Type0 only)

CIDFont dictionary (inside /DescendantFonts)
  /Subtype               → CIDFontType0 | CIDFontType2
  /CIDSystemInfo         → dict (/Registry /Ordering /Supplement)
  /DW                    → integer (default advance width)
  /W                     → array (per-CID widths)
  /CIDToGIDMap           → /Identity or stream (CIDFontType2 only)
  /FontDescriptor        → dictionary (as above)
```

---

*Spec references: ISO 32000-1:2008 §9 (Fonts), §D (Character Sets), §M (Predefined CMaps); Adobe Glyph List Specification v1.7; Adobe Type 1 Font Format (Black Book); Adobe CMap and CIDFont Files Specification v1.0.*