- Fixed duplicate Luma import: `use image::{GrayImage, ImageBuffer, Luma, Luma}` → `use image::{GrayImage, ImageBuffer, Luma}`
- Added re-exports in lib.rs for all preprocessing functions
- Updated verification note
The border padding, pipeline orchestration, and fixtures were already
implemented from previous work. This commit cleans up a minor duplicate
import issue.
Related: pdftract-27n3
Implement step 5 (white-border padding: 10 px on all sides), wire all
preprocessing steps into the final preprocess(input, ImageSource) ->
GrayImage entry point, and curate fixtures for the three image-source
paths (PhysicalScan / DigitalOrigin / Jbig2).
Changes:
- Add add_border_padding() function: creates (width+20) x (height+20)
image with 10px white border on all sides
- Add preprocess() pipeline orchestrator: applies deskew, contrast
normalization, binarization, denoising, and padding in correct order
- Skip contrast, binarization, and denoising for JBIG2 images
- Generate test fixtures for skewed_2deg, uneven_lighting, clean_digital,
and jbig2_scan scenarios
- Add integration tests for all critical test scenarios
- Add A4-page benchmarks targeting < 500ms for physical/digital, < 200ms
for JBIG2
Refs:
- Plan section: Phase 5.3 step 5 (line 1878) + critical tests (lines 1882-1885)
- Bead: pdftract-27n3
- Note: notes/pdftract-27n3.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implement Level 3 of the encoding fallback chain. Hash the raw decoded
font program bytes (/FontFile, /FontFile2, /FontFile3) with SHA-256
and look up the 32-byte digest in a compile-time phf::Map.
- build.rs: generate_font_fingerprints() reads JSON, builds phf::Map
- src/font/fingerprint.rs: FontFingerprint, CachedFingerprint, lookup API
- build/font-fingerprints.json: empty database (placeholder)
Acceptance criteria:
- Empty JSON produces valid phf::Map
- Hash is stable across runs
- Lookup of unknown digest returns None
- Binary footprint < 500KB for 200-font DB (empty = negligible)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add [package.metadata.binstall] to crates/pdftract-cli/Cargo.toml to enable
cargo binstall to download pre-built binaries from GitHub Releases instead
of compiling from source. Also add comprehensive Installation section to
README.md documenting cargo binstall as the recommended install method.
Bead: pdftract-1u80
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Added three new tests to verify the deskew acceptance criteria:
- test_deskew_2_degree_skew: Verifies 2-degree skew is deskewed within 0.1 deg
- test_deskew_0_2_degree_skew_skipped: Verifies 0.2-degree skew is skipped
- test_deskew_20_degree_skew_out_of_range: Verifies out-of-range diagnostic
Helper function create_skewed_text_lines() creates synthetic test images
with known skew angles using small-angle trigonometric approximations.
Note: Tests compile but cannot run without leptonica library (NixOS limitation).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add two comprehensive integration tests to validate the ParentTree resolver:
1. test_parent_tree_annotation_with_struct_parent:
- Creates a body paragraph StructElem
- Creates ParentTree with page array (MCID 0 -> body, MCID 1 -> orphan/null)
- Creates ParentTree with annotation entry (key 100 -> body)
- Verifies MCID resolution returns correct map and orphans
- Verifies annotation /StructParent resolution returns the body ref
- Verifies the referenced StructElem is in the tree
2. test_parent_tree_off_by_one_missing_entries:
- Creates ParentTree with sparse array (only 3 entries for potentially more MCIDs)
- Verifies non-null entries are correctly mapped
- Verifies null entries are recorded as orphans
- Documents that MCIDs beyond array length would be detected in Phase 7.1.4
Also export ParentTreeResolver and ParentTreeEntry from parser module
for use by the block builder in Phase 7.1.4.
All 67 struct_tree tests pass (18 ParentTree-specific tests).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Fix 8 tests that incorrectly passed ParentTree dict directly instead of
wrapping it in a StructTreeRoot-like structure with /ParentTree key
- Fix process_nums_array() to preserve null entries as ObjRef { object: 0 }
instead of filtering them out, ensuring orphan MCIDs are correctly reported
- Add verification note for ParentTree-based MCID-to-StructElem resolver
References: pdftract-57o4, plan 7.1 line 2550 (MCID-to-StructElem mapping)
Implements the 6 named-encoding character-code-to-glyph-name lookup
tables required by Level 2 of the encoding fallback chain:
- WinAnsiEncoding (Windows-1252 superset of StandardEncoding)
- MacRomanEncoding (Mac OS Roman encoding)
- MacExpertEncoding (Mac OS Expert character set)
- StandardEncoding (Adobe Standard encoding)
- SymbolEncoding (Symbol font encoding)
- ZapfDingbatsEncoding (Zapf Dingbats font encoding)
These tables map character codes (0-255) to glyph names, which are then
mapped to Unicode via the Adobe Glyph List (AGL).
Acceptance criteria:
- All 6 tables compile into static arrays with binary footprint < 30 KB
- WIN_ANSI[0x92] == Some("quoteright") (canonical WinAnsi test)
- MAC_ROMAN[0xD2] == Some("quotedblleft") and MAC_ROMAN[0xD3] == Some("quotedblright")
- STANDARD[0x20] == Some("space")
- NamedEncoding::from_name("WinAnsiEncoding") == Some(NamedEncoding::WinAnsi)
Files:
- crates/pdftract-core/build/named-encodings.json - Source data from ISO 32000-1 Annex D
- crates/pdftract-core/src/font/encoding.rs - Public API with NamedEncoding enum
- crates/pdftract-core/build.rs - Build script updates for encoding generation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements Phase 7.1.2: StandardType -> BlockKind mapping for converting
walked StructElem nodes into the BlockKind taxonomy used by Phase 4 output.
Changes:
- Add BlockKind enum with all output block kinds (paragraph, heading with
level, table, list, list_item, figure, caption, code, block_quote, toc,
formula, reference, note, form_field_struct, inline, structural_container,
artifact, unknown)
- Add MappingResult struct bundling block_kind, is_emitted flag, and optional
diagnostic
- Add structure_type_to_block_kind() function for pure type mapping
- Add map_element_to_block() function as primary mapping API
- Add is_artifact() placeholder for Phase 3.4 marked-content integration
- Add 32 comprehensive unit tests covering all mapping paths
Key features:
- Complete type mapping for all 40+ PDF standard structure types
- Heading level extraction: H->level 1, H1..H6->level 1..6
- Inline elements (Span, Quote) map to Inline (not emitted as blocks)
- Structural containers (Document, Part, Sect, Div, etc.) map to
StructuralContainer (descend without emitting)
- Unknown types emit diagnostic and fall back to paragraph
Acceptance criteria:
- Every Standard structure type has a mapping decision
- Critical test: H1/H2 -> heading level 1/2
- Unit tests: list nesting, table grouping, span passthrough
- Unknown-type fallback path emits a diagnostic line
Refs: Plan section 7.1 lines 2552-2553
The ToUnicode CMap parser (Level 1) implementation was already complete
in crates/pdftract-core/src/font/cmap.rs. This commit fixes test assertion
type mismatches where arrays were compared to slices.
Changes:
- Fixed array-to-slice conversions in test assertions (e.g., &['A'] -> &['A'][..])
- Fixed test_odd_length_utf16_emits_diagnostic to use correct hex string input
- All 18 CMap parser tests now pass
Acceptance criteria verified:
- beginbfchar with single-codepoint (U+FB01 fi ligature)
- beginbfchar with multi-codepoint expansion (<00660069> -> 'f' 'i')
- beginbfrange contiguous range (A..=Z mapping)
- beginbfrange explicit array form
- Comment stripping (%)
- Variable-width source codes
- Multi-codepoint destinations in contiguous ranges
Closes: pdftract-udz
This commit completes Phase 5.2.2 by integrating the pdfium-render path
into serve mode with runtime validation and feature propagation.
Changes:
- Propagate ocr and full-render features from CLI to pdftract-core
- Add full_render parameter to serve mode ExtractParams
- Implement runtime validation in build_options():
* Returns BadRequest if full_render requested but PDFium unavailable
* Falls back to direct compositing if feature not compiled
- Update all three serve handlers to handle Result from build_options()
Acceptance Criteria:
✅ cargo build --features ocr,serve,full-render succeeds
✅ cargo build --features ocr,serve (no full-render) succeeds
✅ Runtime fallback: full_render=true with feature absent uses direct path
Notes:
- Binary size CI gate (140 MB) requires separate CI infrastructure
- Soft-mask regression tests require separate fixture work
Refs: pdftract-4my
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Complete verification of direct image compositing path implementation.
All 23 unit tests pass covering CTM tracking, image placement, rotation,
and soft mask handling.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
All 5 child beads completed:
- pdftract-3uq: Font subtype classifier and BaseFont prefix stripper
- pdftract-juc: Standard 14 font registry with hardcoded metrics
- pdftract-6ah: Embedded font program loader (ttf-parser/owned_ttf_parser)
- pdftract-cv4: Type 0 composite font + descendant CIDFont loader
- pdftract-5sh: CIDToGIDMap resolver (Identity and stream forms)
77 font module tests pass. Acceptance criteria:
- PASS: All children closed
- PASS: Classifier returns all 8 FontKind variants
- PASS: Subset prefix stripping works correctly
- PASS: CIDToGIDMap Identity and stream forms verified
- PASS: No unwrap/expect on resource dict access
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Updated notes/pdftract-2zw.md to reflect that the page classification
fixture integration test suite now has 5 tests (added
test_reproducibility_gate_with_perturbation).
Co-Authored-By: Claude Code <noreply@anthropic.com>
All acceptance criteria PASS:
- TrueType font from fixture: glyph_id_for('A') matches Face cmap
- OpenType CFF support: handled via OpenTypeMetrics
- Type1 limited capability: graceful without CharStrings parser
- Corrupt font handling: FONT_PARSE_FAILED diagnostic emitted
15/15 embedded font tests passing.
Update notes/pdftract-33g.md to reflect:
- Micro-benchmark test now PASS (p99 < 5 ms)
- Test count updated from 53 to 54
- Future work section updated (benchmark item removed)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implement FontKind enum and classify_font() function for Phase 2.1
font type detection. Includes strip_subset_prefix() for handling
font subset names (e.g., ABCDEF+Times-Roman).
FontKind variants:
- Type1, Type1Std14 (Standard 14)
- TrueType, OpenTypeCFF
- Type0, CIDFontType0, CIDFontType2
- Type3
Classifier reads /Subtype, /BaseFont, and for Type0 fonts, descendant
CIDFont subtype. OpenTypeCFF detected via /FontDescriptor /FontFile3
with /Subtype /OpenType.
All 27 font tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add verification note documenting memory ceiling implementation
for fuzz and proptest harnesses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add 4 new tests to verify PNG and TIFF predictor functions use row-by-row
processing with bounded peak memory (2x stride), never pre-allocating full
output buffers inside tests.
- test_png_predictor_budget_enforcement_small_fixture: 200-byte fixture,
100-byte budget, verifies truncation at row boundary
- test_tiff_predictor_2_budget_enforcement_small_fixture: 160-byte fixture,
80-byte budget, verifies row-by-row processing for grayscale
- test_png_predictor_multiple_selectors_budget_per_row: 25-byte fixture
with all PNG selector types, verifies per-row budget checking
- test_tiff_predictor_2_rgb_budget_enforcement: 45-byte RGB fixture,
verifies multi-byte pixel handling with budget enforcement
All fixtures are under 250 bytes, no full-buffer pre-allocation, tests
mirror the row-by-row discipline from bf-49wmw production fix.
Closes bf-21hw8
- Add decode_page_content_streams() function for per-page lazy decode
- Update extract_page_from_dict() to support lazy stream decoding
- Modify extract_pdf() and extract_pdf_ndjson() to enable lazy decoding
- Fix borrow checker issue in LazyPageIter::next()
This ensures content streams are decoded lazily per page and dropped
immediately after processing, keeping peak RSS flat across page count.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Documents the bug fixes made to enable the semaphore-based parallel
page extraction implementation to compile and work correctly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verification note for the completion of Phase 0: CI Infrastructure epic.
All 10 sub-phase coordinator beads are closed:
- pdftract-1wqec: WorkflowTemplate scaffolding
- pdftract-1bn: Cross-compilation build matrix (5 targets)
- pdftract-30n: Test execution (musl + glibc)
- pdftract-2rf: Static analysis and quality gates
- pdftract-33v: Property tests and nightly fuzz
- pdftract-2t9: Regression corpus runner (500 PDFs)
- pdftract-60h: Competitive benchmarks (Tier 4)
- pdftract-23k1: pdftract-py-ci stub
- pdftract-4b0z: Release publishing
- pdftract-3i1o: CI observability
This epic adds the final missing piece: the CI sensor that triggers
pdftract-ci workflow on push and PR events.
See also: ci(pdftract-4nj7y) in declarative-config
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Document the implementation and verification of the test-matrix DAG
branch with musl and glibc test legs.
Summary:
- Created pdftract-test-image-build WorkflowTemplate
- Verified test-matrix DAG implementation (test-glibc, test-musl)
- Both legs emit JUnit XML for test reporting
- Acceptance criteria: PASS (with notes on setup step and Docker image)
Known dependencies:
- Setup step still a placeholder (handled by separate Phase 0 bead)
- Docker image needs to be built via pdftract-test-image-build workflow
Relates to pdftract-30n: Phase 0.3 Test execution — cargo test on musl + glibc
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bead-Id: pdftract-30n
Document acceptance criteria PASS status for:
- Custom Docker image with OCR support
- nextest configuration with ci/ci-proptest profiles
- Updated test-glibc template in CI
All criteria PASS. Ready to close bead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Convert test-matrix from single container to DAG with two parallel branches:
- test-glibc: Full test suite including OCR (tesseract available on Debian)
- test-musl: Production binary feature set (no OCR, unavailable on Alpine)
Musl leg configuration:
- Image: ghcr.io/cross-rs/x86_64-unknown-linux-musl:main
- Test: cross test --release --target x86_64-unknown-linux-musl --features default,serve,decrypt
- Output: JUnit XML artifact (test-results-musl.xml)
- Test threads: 4 (parallel execution)
Also updates:
- .nextest.toml: Add JUnit XML output settings to profile.ci
- Cross.toml: Add cross configuration for musl target
Bead: pdftract-5gtcj
Plan section: Phase 0.3
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The MSRV check gate (rust:1.78-slim build) was already fully
implemented in the initial CI workflow. This verification note
documents the existing implementation and confirms all acceptance
criteria are met.
Acceptance criteria:
- Gate runs in pdftract-ci on every PR: PASS
- Failure blocks PR merge: PASS
- Successful run reports artifact: PASS
- Failure mode produces actionable error: PASS
No changes to the workflow were required.
Related: pdftract-2rf (quality gates coordinator)
Document the implementation of the cargo-audit quality gate with
severity gating and audit.toml allow-list.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Documents the implementation of the clippy quality gate with INV-8
enforcement via clippy::unwrap_used and clippy::expect_used lints.
Bead: pdftract-3cp3a
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add Licensing section to CONTRIBUTING.md explaining:
- Dual MIT OR Apache-2.0 licensing model
- Apache NOTICE file policy (optional for upstream, redistributors MAY add)
- Attribution guidelines for downstream redistributors
Also add verification note confirming all acceptance criteria PASS:
- LICENSE-MIT and LICENSE-APACHE files present at repo root
- All workspace crates declare "MIT OR Apache-2.0" license
- cargo deny check licenses passes (implicit deny-by-default via allow list)
- Binary and wheel distributions configured to include both license files
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
