Documents that the extract_text PyO3 entry point was already
implemented in extract_text.rs and exposed in lib.rs. This bead
only fixed a minor compilation bug where extract_markdown was calling
the wrong function name.
Acceptance criteria:
- Returns PyString (PASS)
- Matches CLI output (PASS)
- Supports pages kwarg (PASS)
- GIL release during extraction (PASS)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
All acceptance criteria PASS. The extract() function was already
implemented in crates/pdftract-py/src/extract.rs with:
- Strict kwarg validation (ALLOWED_KWARGS list)
- GIL release via py.allow_threads during extraction
- Python dict conversion via pythonize::pythonize
- Error mapping to PdftractError hierarchy
See notes/pdftract-41lbg.md for detailed verification.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verified non-Range server fallback implementation:
- download_to_temp_and_mmap function (http_range.rs)
- TempMmapSource wrapper (source/mod.rs)
- Fallback integration in open_source and open_remote
- Diagnostic emission for REMOTE_NO_RANGE_SUPPORT and REMOTE_INSUFFICIENT_DISK
- Disk space checking with 10% buffer
- RAII cleanup via NamedTempFile
All acceptance criteria verified PASS.
Add verification note documenting that all acceptance criteria for
the --audit-log flag and audit logging infrastructure are already
implemented in the codebase.
Acceptance criteria verified:
- --audit-log FILE flag on serve, mcp, and inspect subcommands
- Per-request NDJSON line with all documented fields
- Stdio MCP omits client_ip field
- Log-policy enforcement (compile-time CI gate + runtime redaction)
- TH-08 test for log policy verification
- Rotation policy documented in --help
- Fingerprint logged instead of path/URL
- AuditLogWriter is crash-safe
All audit module tests pass (6/6).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Integrates log-policy enforcement as a Tier-1 quality gate in CI and
installs the panic hook for SecretString redaction in backtraces.
Changes:
- Add log-policy-check to quality-matrix in pdftract-ci.yaml
- Install panic_hook in main.rs for crash dump redaction
- Create verification note at notes/pdftract-3990k.md
Existing implementations verified:
- secrecy crate (v0.10) in workspace dependencies
- SecretString used consistently for credentials
- redact_headers_for_log() in mcp/http.rs strips auth headers
- check-log-policy.sh CI gate scans for forbidden patterns
- CONTRIBUTING.md documents NEVER-log secrets policy
- Fuzz test (tests/log_secret_fuzz.rs) with 10,000 case coverage
Acceptance criteria:
- secrecy crate added ✅ PASS (already in workspace)
- SecretString used for credentials ✅ PASS
- CI gate runs on every PR ✅ PASS
- Fuzz-test confirms no credential leaks ✅ PASS
- Auth headers stripped from logging ✅ PASS
- Panic hook redacts SecretString ✅ PASS
- CONTRIBUTING.md section ✅ PASS
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The emit! macro expects diagnostic codes without the DiagCode:: prefix.
Changed three occurrences in codespace.rs:
- Line 281: DiagCode::CmapInvalidCodespace → CmapInvalidCodespace
- Line 290: DiagCode::CmapInvalidCodespace → CmapInvalidCodespace
- Line 412: DiagCode::CmapInvalidCodespace → CmapInvalidCodespace
This fixes compilation errors that prevented the codebase from building.
The --pages, --header, and URL credential parsing features are fully
implemented in pages.rs, header.rs, and url.rs modules with comprehensive
tests and integration in main.rs, grep/mod.rs, and hash.rs.
References: pdftract-25igv, notes/pdftract-25igv.md
Add call-site diagnostic emission for DCTDecode SOI/EOI marker validation.
Previously, DCTDecoder.validate_markers() created diagnostics but they were
dropped because StreamDecoder trait doesn't support returning them. Now
diagnostics are emitted in decode_stream_impl() like JBIG2/JPX/CCITT.
Also include source module refactoring:
- Add PdfSource adapter trait for source::PdfSource compatibility
- Feature-gate http_range module with `remote` feature
- Update document.rs to use new source traits
Acceptance criteria:
- DCTDecode emits STREAM_INVALID_JPEG for missing SOI/EOI markers
- JBIG2Decode emits OCR_JBIG2_UNSUPPORTED when full-render disabled
- JPXDecode emits OCR_JPX_UNSUPPORTED and validates JP2 magic
- CCITTFaxDecode emits OCR_CCITT_UNSUPPORTED when no libtiff
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bead-Id: pdftract-4xmp6
Bead-Id: pdftract-57np8
Bead-Id: pdftract-3954u
The --header CLI flag implementation was already complete in the codebase.
This note documents the implementation and verifies all acceptance criteria.
Acceptance criteria verified:
- Single header with URL: PASS
- Multiple headers: PASS
- Managed header rejection: PASS
- CRLF injection protection: PASS
- No colon error: PASS
- Local file silent ignore: PASS
No new code was required - the feature was already fully implemented
in main.rs, header.rs, source/mod.rs, and http_range.rs.
Documents the implementation, acceptance criteria status, and design
decisions for the CMap codespace range parser.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Remove unused jpx::JpxDecoder import from stream.rs (code uses fully qualified paths)
- Add notes/pdftract-36glh.md with acceptance criteria verification
The JPXDecode passthrough implementation was already complete in commit 4ba4687.
This change is minor cleanup only.
References: pdftract-36glh
This commit fixes a compilation error in the javascript tests that were
using PageDict::default(). The JBIG2 decoder module was already fully
implemented; this change only enables the tests to compile and run.
Changes:
- Add Default impl for PageDict in parser/pages.rs
- Verify all 11 JBIG2-related tests pass
The JBIG2Decode passthrough filter implementation is complete:
- Passthrough of raw JBIG2 bytes
- /JBIG2Globals reference recording for downstream consumers
- OCR_JBIG2_UNSUPPORTED diagnostic emission when full-render disabled
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Made map_error_to_exit_code() function public in hash.rs so it can be
called from main.rs
- Added test file test_hash_exit_codes.rs to verify exit code behavior
- Updated verification note with current implementation status
The hash subcommand was already implemented but map_error_to_exit_code
was private, causing a compilation error. This fix resolves the issue.
Related: pdftract-3954u
Implement O'Gorman 1993 Docstrum algorithm for reading order detection
on irregular layouts (magazines with sidebars) where XY-cut produces
fragmented regions.
Implementation:
- k=5 nearest neighbors per block (Docstrum standard)
- Euclidean center-to-center distance in PDF user space
- Angle constraints: ±30° from horizontal (within-line) and vertical (between-line)
- Root detection: nodes with no incoming edges from blocks above
- Root sorting by (column ASC, y DESC)
- DFS traversal per component in y-then-x order
Acceptance criteria PASS:
- Magazine main+sidebar: 2 components; main first, sidebar second
- Pathological scattered: each a root, visited (column, y desc)
- All-one-line horizontal: 1 component, left-to-right
- All-one-column vertical: 1 component, top-to-bottom
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The detect_xfa function was already implemented in the codebase at the
time of bead assignment. This note documents the verification of the
existing implementation against the bead's acceptance criteria.
All 6 tests pass, covering all acceptance criteria:
- XFA stream presence → true
- XFA array packet form → true
- No XFA key → false
- XFA null → false
- No AcroForm → false
- XFA as indirect reference → true
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verification confirms the CLI parsing and validation for multi-format
output flags is already fully implemented in crates/pdftract-cli/src/output.rs.
All acceptance criteria verified:
- Duplicate format rejection ✓
- NDJSON exclusivity ✓
- At most one stdout ✓
- Auto-naming with --format + -o ✓
No code changes required.
Update the verification note for pdftract-2qw5j to clarify that the
bead's "Critical considerations" enum values differ from the actual
implementation:
- confidence_source: bead lists ["vector", "ocr", ...] but plan/Rust
code uses ["native", "heuristic", "ocr"] (per plan line 363)
- severity: bead omits "fatal" but Rust code includes it for
extraction-aborting conditions
The schema generation system is complete and correct per the plan
specification. The bead requirements appear to be from an earlier
spec version and are superseded by the plan.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add std::sync::Arc import for thread sharing
- Fix lifetime issue in test_sync_multiple_threads using Arc
- Add mut to source in test_empty_file for Read trait
All FileSource tests pass (12/12).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The --pages RANGE CLI flag implementation was already complete in the
codebase. All required functionality was present including:
- Range parser in pages.rs with comprehensive tests
- CLI integration in main.rs
- HTTP serve support in serve.rs
- MCP tools integration
- PyO3 bindings in pdftract-py
All acceptance criteria verified PASS.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implement FileSource as a PdfSource fallback for when memory-mapping
is not available or desired. Uses parking_lot::Mutex<File> for
thread-safe concurrent access across rayon workers.
Changes:
- Add parking_lot = "0.12" dependency to pdftract-core/Cargo.toml
- Rewrite FileSource to use Mutex<File> for Send + Sync support
- Implement PdfSource, Read, and Seek traits
- Add 12 comprehensive tests including concurrent read tests
All tests pass. Thread-safe concurrent access verified via
test_sync_multiple_threads and test_concurrent_read_range.
Co-Authored-By: Claude Code (claude-opus-4.7) <noreply@anthropic.com>
Bead-Id: pdftract-5ik66
- Implemented aes_128_decrypt with CBC mode + PKCS#7 padding
- Implemented derive_aes_128_object_key with 'sAlT' suffix
- Implemented is_identity_filter for crypt filter handling
- All 11 unit tests passing
- Integration work deferred to coordinator bead pdftract-1z0qt
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Define the PdfSource trait abstraction over PDF byte sources. This trait
provides a uniform API for reading PDF data from different sources:
local files (MmapSource, FileSource), and eventually remote HTTPS PDFs.
Trait features:
- Read + Seek + Send + Sync supertrait bounds for rayon page-parallelism
- len() returns total source length
- read_range() returns Bytes for zero-copy slicing
- prefetch() with no-op default (MmapSource overrides for MADV_SEQUENTIAL)
MmapSource:
- Memory-mapped file access via memmap2
- Applies MADV_SEQUENTIAL advice via prefetch()
- Zero-copy read_range() using Bytes::copy_from_slice()
- Fallback for platforms/filesystems where mmap fails
FileSource:
- Standard I/O implementation using std::fs::File
- Read+Seek delegation to underlying File
- read_range() uses try_clone() for thread-safe concurrent access
Re-exports from pdftract-core::source::PdfSource.
Verification note: notes/pdftract-1mmq9.md documents completion status.
Parser module migration to use new PdfSource is deferred to follow-up.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Blocker identified:
- Extraction pipeline (extract.rs) doesn't use Phase 4 layout pipeline
- Column detection functions never called in production
- SpanJson.column hardcoded to None (lines 1059, 1916)
- No end-to-end tests for acceptance criteria
Span struct HAS column field (line 179) but extraction doesn't use it.
Coordinator CANNOT CLOSE - sub-phase not end-to-end functional.
The encrypt_padded_mut API requires the buffer to be large enough to
hold the padded ciphertext. The tests were using plaintext.to_vec() which
only allocated plaintext.len() bytes, insufficient for padding.
Changed pattern:
- Before: plaintext.to_vec() (insufficient space)
- After: vec![0u8; plaintext.len() + 16] with copy_from_slice
Also fixed incorrect usage: encrypt_padded_mut returns Result<(), Error>,
not a length. Use data_copy.len() directly for ciphertext length.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This bead asked for implementation of BMC/BDC/EMC marked-content
operators and MarkedContentStack, but these were already fully
implemented in the codebase with comprehensive test coverage.
Verification note documents:
- MarkedContentStack in marked_content_stack.rs
- BMC/BDC/EMC parsers in marked_content_operators.rs
- Integration into execute_with_do in content_stream.rs
- All 6 acceptance criteria covered by passing tests
- 57 marked-content tests all passing
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Fixed compilation errors in Span constructors by adding missing `column: None` field.
Verified that the existing multi-output CLI parsing implementation meets all
acceptance criteria for bead pdftract-37qim.
Changes:
- crates/pdftract-core/src/span/mod.rs: Add column field to new() and empty() constructors
Verification:
- All 23 output::tests pass
- CLI parsing validated for duplicate format detection, ndjson exclusivity, stdout uniqueness
- Format auto-naming (--format with -o) works correctly
- Default behavior (no flags -> JSON to stdout) confirmed
See notes/pdftract-37qim.md for detailed verification results.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
All 4 children beads closed with verification:
- Line struct + baseline computation (pdftract-sdx9z)
- Baseline clustering algorithm (pdftract-6bwq4)
- Within-line span sorting (pdftract-1jkme)
- RTL direction detection (pdftract-1ofnz)
Acceptance criteria:
- ✅ All 4 children closed
- ✅ Two-column layout: columns NOT merged into one line (test_two_column_separate_blocks)
- ✅ Superscript span at higher y: clustered with baseline text
- ✅ Arabic text: bidi R characters detected, spans sorted right-to-left
- ✅ Mixed Latin+Arabic line: detected as "mixed" direction
44/44 tests pass in layout::line module.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add map_confidence_source to confidence module re-exports in lib.rs
- Remove duplicate map_confidence_source function from span/mod.rs
- Add Ocr case to map_unicode_source_to_confidence helper
- Add comprehensive tests for map_confidence_source in span module
The ConfidenceSource enum and map_confidence_source function were already
implemented in the confidence module from bead pdftract-2etcd. This change
completes the public API exposure and removes the duplicate implementation.
Acceptance criteria (all PASS):
- Single-glyph to_unicode span: confidence_source == Native
- Single-glyph shape_match span: confidence_source == Heuristic
- Mixed-glyph span (agl + shape_match): confidence_source == Heuristic
- 4.7 correction applied: Native -> Heuristic override
- OCR span: confidence_source == Ocr
- JSON serialization: lowercase strings
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
