pdftract/docs/notes/ocr-accuracy.md

# OCR Accuracy Fallback Plan

**Proof Obligation PB-3:** Accept WER 5% on clean 300-DPI scans with a methodology footnote tying the number to the Tesseract version pinned in Dockerfile (per OQ-03).

**Tied to:** Risk R3 — Tesseract WER > 3% on clean 300-DPI scans

## Overview

This document outlines the fallback plan if pdftract fails to achieve the primary objective's **Word Error Rate (WER) < 3%** on clean 300-DPI scanned documents using Tesseract 5.x.

## Primary Target

### Claim: WER < 3% on clean 300-DPI scans

**What Must Be True:**
- The `tests/fixtures/scanned/` corpus produces a measured WER < 3% on extractions using Tesseract 5.x with default language pack
- Test fixtures are synthesized at 300 DPI with minimal noise
- Ground-truth text is known from source vector PDFs

**Invalidation Signal:**
- Phase 5.4 integration test reports WER ≥ 3%
- Consistent failure across multiple fixture types (receipts, invoices, forms)

## Fallback Plan: WER 5% Target

### Trigger Conditions

Activate PB-3 fallback if **ALL** of the following are true:

1. WER ≥ 3% on the baseline `tests/fixtures/scanned/` corpus after Phase 5.3 preprocessing tuning
2. WER failure is **consistent** (not a single flaky fixture)
3. Root cause analysis identifies **Tesseract limitations** (not preprocessing bugs)

### Mitigation Steps

#### Step 1: Verify Test Conditions

Before degrading the target, confirm:

```bash
# Verify fixture DPI
identify -verbose tests/fixtures/scanned/receipt/receipt-300dpi-scanned.pdf | grep Resolution

# Verify Tesseract version
tesseract --version  # Should be 5.x

# Verify language pack installation
tesseract --list-langs | grep eng
```

#### Step 2: Preprocessing Pipeline Retuning

Phase 5.3 preprocessing can be further tuned:

1. **Deskew threshold**: Currently 0.5°; try 0.3° or 0.7°
2. **Sauvola window size**: Currently 15×15 pixels; try 11×11 or 21×21
3. **Noise reduction**: Add median blur (3×3 kernel) before binarization

```rust
// Example: Phase 5.3 preprocessing tuning
pub fn preprocess_ocr(image: &DynamicImage) -> Result<GrayImage> {
    let mut img = image.to_luma8();
    
    // Tune: Adjust deskew threshold
    let angle = detect_skew(&img, 0.3);  // Default: 0.5
    
    // Tune: Adjust Sauvola window
    let binarized = sauvola_binarize(&img, 11);  // Default: 15
    
    // Tune: Add noise reduction
    let denoised = median_blur(&binarized, 3);
    
    Ok(denoised)
}
```

#### Step 3: Per-Fixture WER Analysis

If WER remains ≥ 3% after retuning, analyze **per-fixture WER**:

```bash
# Run WER test with detailed output
cargo nextest run --feature ocr ocr_wer_detailed -- --nocapture

# Expected output:
# receipt-300dpi-scanned.pdf: WER 2.8% (PASS)
# invoice-300dpi-scanned.pdf: WER 3.2% (FAIL)
# form-300dpi-scanned.pdf: WER 4.1% (FAIL)
```

If **specific fixtures fail** while others pass:
- Document fixture-specific limitations
- Exclude failing fixtures from the WER benchmark
- Add new fixtures that better represent real-world scans

If **all fixtures fail uniformly**:
- Proceed to Step 4 (target revision)

#### Step 4: Revise Target to 5%

If uniform failure persists after preprocessing retuning:

1. **Update Proof Obligation Ledger:**
   - Change claim from "WER < 3%" to "WER < 5%"
   - Add Revision History entry documenting the change

2. **Document Methodology Footnote:**
   ```markdown
   ## Revision History
   
   ### 2026-XX-XX: Revised OCR WER target from 3% to 5%
   
   **Rationale:** Tesseract 5.3.1 (pinned in Dockerfile) achieves 4.2% WER on the
   `tests/fixtures/scanned/` corpus after Phase 5.3 preprocessing retuning.
   
   **Per-fixture breakdown:**
   - receipt-300dpi-scanned.pdf: 4.1% WER
   - invoice-300dpi-scanned.pdf: 4.3% WER
   - form-300dpi-scanned.pdf: 4.5% WER
   
   **Root cause:** Tesseract's handling of low-contrast regions and multi-column
   layouts introduces consistent errors that cannot be eliminated via preprocessing
   alone without degrading performance on simpler fixtures.
   
   **Mitigation:**
   - Preprocessing pipeline tuned for deskew (0.3° threshold) and Sauvola
     binarization (11×11 window)
   - Per-span confidence scoring enables downstream filtering of low-confidence OCR
   - Future v1.1+ may integrate PaddleOCR or doctr as opt-in `--alt-ocr` feature
   ```

3. **Update Primary Objectives:**
   - Change "OCR WER < 3%" to "OCR WER < 5%" with footnote reference

4. **No Binary Changes Required:**
   - The OCR pipeline itself does not change
   - Only the documented target changes

## Per-Fixture WER Table

The following table should be maintained in `benches/results/ocr-wer/<commit-sha>.json`:

```json
{
  "commit": "abc123",
  "tesseract_version": "5.3.1",
  "timestamp": "2026-07-05T12:00:00Z",
  "overall_wer": 0.042,
  "fixtures": [
    {
      "fixture": "tests/fixtures/scanned/receipt/receipt-300dpi-scanned.pdf",
      "wer": 0.041,
      "word_count": 42,
      "errors": 2,
      "ground_truth": "tests/fixtures/scanned/receipt/receipt-300dpi.txt"
    },
    {
      "fixture": "tests/fixtures/scanned/documents/invoice-300dpi-scanned.pdf",
      "wer": 0.043,
      "word_count": 85,
      "errors": 4,
      "ground_truth": "tests/fixtures/scanned/documents/invoice-300dpi.txt"
    }
  ]
}
```

## Alternative OCR Engines (v1.1+)

If WER target cannot be met even at 5%, **PB-7** activates:

### PB-7: Bundle PaddleOCR or doctr as opt-in `--alt-ocr` feature

**Activation:**
- WER consistently > 5% after Tesseract tuning
- User demand for higher OCR accuracy on low-quality scans

**Implementation:**
- Add feature flag `alt-ocr` gated behind `--features alt-ocr`
- Bundle PaddleOCR models (~80 MB) or doctr (~50 MB) in Docker image
- Exclude from default-binary Weight Target (feature is opt-in)

**Trade-offs:**
- PaddleOCR: Higher accuracy (~2% WER), larger models (~80 MB), CPU-only inference
- doctr: Lower accuracy (~3.5% WER), smaller models (~50 MB), GPU/CPU inference

## Tesseract Version Policy (OQ-03)

The OCR accuracy is tied to the **Tesseract version pin**:

### Current Pin: Tesseract 5.3.1

```dockerfile
# Dockerfile (ocr / full variants)
RUN apt-get update && apt-get install -y \
    tesseract-ocr=5.3.1-1 \
    tesseract-ocr-eng=5.3.1-1 \
    && rm -rf /var/lib/apt/lists/*
```

### Version Pinning Rationale

**Why pin to 5.3.1:**
- Reproducibility: WER measurements are tied to this specific version
- Stability: Avoids regressions from upstream changes
- CI consistency: All CI runs use the same version

**When to update:**
- Critical security vulnerability (CVE in Tesseract)
- Measurable WER improvement (> 0.5% absolute gain) in a new patch release
- Language pack expansion that supports required scripts

**Update process:**
1. Test new version in `iad-ci` CI against `tests/fixtures/scanned/`
2. Measure WER delta; if improved, update Dockerfile pin
3. Update this document's methodology footnote with new version
4. Tag release with changelog entry

## Verification

To verify OCR accuracy:

```bash
# Run OCR WER benchmark
cargo nextest run --features ocr ocr_wer_benchmark

# Check against 3% (or 5% after fallback) target
cargo nextest run --features ocr ocr_wer_benchmark -- --fail-on-wer-exceeds 0.05

# View per-fixture breakdown
cat benches/results/ocr-wer/latest.json | jq '.fixtures[] | select(.wer > 0.05)'
```

## References

- Plan Proof Obligation PB-3 (line ~580)
- Plan Risk R3 (line ~557)
- Plan Open Question OQ-03 (line ~514)
- `tests/fixtures/scanned/` — OCR test fixtures
- Phase 5.3 implementation: Image preprocessing pipeline
- Phase 5.4 implementation: OCR accuracy validation