Commit graph

5 commits

Author SHA1 Message Date
jedarden
e11b487b19 feat(pdftract-2w3r): implement StructTree coverage check and XY-cut fallback
Implements Phase 7.1.4: coverage-based fallback for Suspects-tagged PDFs.

## Changes

### New files
- crates/pdftract-core/src/parser/marked_content.rs: MCID tracking and CoverageResult
- crates/pdftract-core/tests/struct_tree_coverage.rs: Integration tests

### Modified files
- crates/pdftract-core/src/parser/catalog.rs: MarkInfo::requires_coverage_check(), ReadingOrderAlgorithm enum
- crates/pdftract-core/src/parser/struct_tree.rs: check_coverage_for_pages(), ParentTreeResolver::compute_coverage()
- crates/pdftract-core/src/extract.rs: MCID tracking per page, coverage check integration

## Implementation

Coverage calculation:
- claimed_mcids = MCIDs resolving to non-Artifact StructElem via ParentTree
- total_mcids = All MCIDs from marked-content sequences on the page
- coverage = claimed_mcids / total_mcids

Fallback rule (per plan §7.1 line 2572):
- If /MarkInfo /Suspects is true AND coverage < 0.80 → use XY-cut
- Otherwise → use StructTree

## Tests

Unit tests (20):  All passing
- Suspects false + 50% coverage → no fallback
- Suspects true + 95% coverage → no fallback
- Suspects true + 60% coverage → fallback
- Edge cases: no MCIDs, 80% threshold, multi-page

Integration tests: ⚠️ Skipped (malformed fixture PDFs)
- tagged-suspects-*.pdf have invalid xref tables
- Core functionality verified by unit tests
- Fixtures need regeneration or real-world tagged PDFs

## Acceptance Criteria (from pdftract-2w3r)

- [x] Unit tests: Suspects false + 50% coverage → no fallback
- [x] Unit tests: Suspects true + 95% coverage → no fallback
- [x] Unit tests: Suspects true + 60% coverage → fallback
- [x] Per-page diagnostic appears in receipts when fallback triggers
- [x] reading_order_algorithm field set to "struct_tree" or "xy_cut"
- [ ] Integration test: tagged-suspects-true.pdf (fixture malformed)

Refs: pdftract-2w3r, plan §7.1 line 2554, INV-8

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 20:53:25 -04:00
jedarden
b72d8312ce test(pdftract-57o4): add ParentTree integration tests for annotation and sparse arrays
Add two comprehensive integration tests to validate the ParentTree resolver:

1. test_parent_tree_annotation_with_struct_parent:
   - Creates a body paragraph StructElem
   - Creates ParentTree with page array (MCID 0 -> body, MCID 1 -> orphan/null)
   - Creates ParentTree with annotation entry (key 100 -> body)
   - Verifies MCID resolution returns correct map and orphans
   - Verifies annotation /StructParent resolution returns the body ref
   - Verifies the referenced StructElem is in the tree

2. test_parent_tree_off_by_one_missing_entries:
   - Creates ParentTree with sparse array (only 3 entries for potentially more MCIDs)
   - Verifies non-null entries are correctly mapped
   - Verifies null entries are recorded as orphans
   - Documents that MCIDs beyond array length would be detected in Phase 7.1.4

Also export ParentTreeResolver and ParentTreeEntry from parser module
for use by the block builder in Phase 7.1.4.

All 67 struct_tree tests pass (18 ParentTree-specific tests).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:36:09 -04:00
jedarden
ecf78671b5 feat(pdftract-57o4): fix ParentTree resolver tests and null entry handling
- Fix 8 tests that incorrectly passed ParentTree dict directly instead of
  wrapping it in a StructTreeRoot-like structure with /ParentTree key
- Fix process_nums_array() to preserve null entries as ObjRef { object: 0 }
  instead of filtering them out, ensuring orphan MCIDs are correctly reported
- Add verification note for ParentTree-based MCID-to-StructElem resolver

References: pdftract-57o4, plan 7.1 line 2550 (MCID-to-StructElem mapping)
2026-05-23 18:32:56 -04:00
jedarden
0882962861 feat(pdftract-2ork): implement element-type to block-kind mapping table
Implements Phase 7.1.2: StandardType -> BlockKind mapping for converting
walked StructElem nodes into the BlockKind taxonomy used by Phase 4 output.

Changes:
- Add BlockKind enum with all output block kinds (paragraph, heading with
  level, table, list, list_item, figure, caption, code, block_quote, toc,
  formula, reference, note, form_field_struct, inline, structural_container,
  artifact, unknown)
- Add MappingResult struct bundling block_kind, is_emitted flag, and optional
  diagnostic
- Add structure_type_to_block_kind() function for pure type mapping
- Add map_element_to_block() function as primary mapping API
- Add is_artifact() placeholder for Phase 3.4 marked-content integration
- Add 32 comprehensive unit tests covering all mapping paths

Key features:
- Complete type mapping for all 40+ PDF standard structure types
- Heading level extraction: H->level 1, H1..H6->level 1..6
- Inline elements (Span, Quote) map to Inline (not emitted as blocks)
- Structural containers (Document, Part, Sect, Div, etc.) map to
  StructuralContainer (descend without emitting)
- Unknown types emit diagnostic and fall back to paragraph

Acceptance criteria:
- Every Standard structure type has a mapping decision
- Critical test: H1/H2 -> heading level 1/2
- Unit tests: list nesting, table grouping, span passthrough
- Unknown-type fallback path emits a diagnostic line

Refs: Plan section 7.1 lines 2552-2553
2026-05-23 17:24:00 -04:00
jedarden
d41d47de66 feat(pdftract-1x2): implement StructTree depth-first walker with RoleMap resolution
Implements the StructTree parser (Phase 7.1.1) with:
- Depth-first walker over /StructTreeRoot via /K array
- Support for all four /K entry types: StructElem, MCID, MCR, OBJR
- /RoleMap resolution with chain handling and cycle detection
- /Lang inheritance through the structure tree
- /ActualText inheritance (applies to all descendant content)
- Public API: StructureType, StructElemNode, StructTreeRoot, RoleMap, Kid

Acceptance criteria:
- PASS: All four /K element kinds handled without crashing
- PASS: /RoleMap chains resolve to standard type or NonStruct
- PASS: /Lang and /ActualText inherit correctly down tree
- PASS: Unit tests for Word RoleMap (Heading1 -> H1)
- PASS: Unit tests for nested /Lang and /ActualText scope
- PASS: Public type StructElemNode documented in core crate

References:
- Plan section 7.1 StructTree Exploitation (lines 2547-2549, 2552-2553)
- PDF 1.7 spec 14.7.4 (Structure Tree) and 14.8.4 (Standard Structure Types)

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-05-23 16:43:22 -04:00