Commit graph

20 commits

Author SHA1 Message Date
jedarden
7429a67d08 feat(pdftract-juc): implement Standard 14 font metrics registry
- Add build.rs that generates compile-time std14 metrics from JSON
- Add std14.rs module with Std14Metrics struct and get_std14_metrics()
- Add build/std14-metrics.json with AFM-derived widths for all 14 fonts
- Re-export Std14Metrics, NamedEncoding, get_std14_metrics in lib.rs

Acceptance criteria:
- All 14 Standard fonts (Courier, Helvetica, Times, Symbol, ZapfDingbats
  and their variants) return valid metrics from the registry
- Subset-prefixed names (ABCDEF+Helvetica) resolve via strip_subset_prefix()
- Width tables match Adobe AFM data within rounding tolerance
- Binary footprint < 60 KB (generated source: 20 KB, actual data ~8 KB)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 14:04:02 -04:00
jedarden
9b5fbc9b5e feat(pdftract-bf-2y2rp): implement lazy stream decoding for PDF extraction
- Add decode_page_content_streams() function for per-page lazy decode
- Update extract_page_from_dict() to support lazy stream decoding
- Modify extract_pdf() and extract_pdf_ndjson() to enable lazy decoding
- Fix borrow checker issue in LazyPageIter::next()

This ensures content streams are decoded lazily per page and dropped
immediately after processing, keeping peak RSS flat across page count.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:30:26 -04:00
jedarden
3f8d9dc687 feat(pdftract-5rl5o): add cbindgen header generation for pdftract.h
Add cbindgen infrastructure to auto-generate C/C++ header from Rust extern
"C" surface at build time.

- Add cbindgen.toml config (C language, include guard, pragma_once, cpp_compat)
- Add build.rs to generate include/pdftract.h during cargo build
- Generated header compiles cleanly with gcc (C) and g++ (C++)

The header is the contract between libpdftract and C/C++ consumers.
Future extern "C" functions will automatically appear in the header.

Refs: pdftract-5rl5o
2026-05-23 07:31:53 -04:00
jedarden
29348ce21d feat(pdftract-4sky1): implement doctor exit code policy
- Add exit code policy to doctor command help text
- Update --exit-on-fail flag help to clarify default behavior
- Add code comment explaining why --exit-on-fail is a no-op

Exit codes per plan section 6.10:
- Exit 0: all checks OK or WARN (no FAIL)
- Exit 1: at least one check is FAIL
- Exit 2: CLI parse error (clap default)

Closes: pdftract-4sky1
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-05-23 07:27:09 -04:00
jedarden
2fe45079b3 fix(pdftract-1w5u1): ensure doctor output fits within 80 columns for all modes
The detail field truncation in human.rs only applied to TTY output,
causing lines to exceed 80 columns when piping to cat or using --no-color.

Fix: Apply truncation uniformly across all output modes:
- TTY mode: Use actual terminal width from terminal_size crate
- Non-TTY/--no-color: Assume 80 columns and truncate accordingly
- Detail field max width: term_width - 38 columns

Max line width now exactly 80 characters for all output modes.

Acceptance criteria verified:
- TTY colored table with summary ✓
- Non-TTY plain text, no ANSI ✓
- --json single JSON object ✓
- --json summary counts ✓
- --features list, exit 0 ✓
- --no-color plain text in TTY ✓
- 80-column terminal width ✓
- N/A excluded from human, in JSON ✓

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 07:24:02 -04:00
jedarden
c2be1da5ce docs(pdftract-1w5u1): add verification note for doctor output formats
Verified all three output formats (colored table, JSON, --features)
work correctly. No code changes required - implementation was
already complete in output/ module.

Acceptance criteria:
- PASS: Default TTY colored table with summary
- PASS: Non-TTY plain text (no ANSI codes when piped)
- PASS: --json output parses correctly with jq
- PASS: --features lists compiled features, exit 0
- PASS: --no-color forces plain text
- PASS: 80-column width compliance
- PASS: N/A rows excluded from human, included in JSON

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 07:24:02 -04:00
jedarden
3155510a5e feat(pdftract-4q8cq): implement 14 environment checks for pdftract doctor
Implemented all 14 environment checks as specified in the bead description:
- pdftract binary: version + git-sha + compiled features
- tesseract install: version check (major >= 5 OK, == 4 WARN, <= 3 FAIL)
- tesseract languages: eng + requested langs present
- leptonica install: pkg-config check >= 1.79
- libtiff: pkg-config check with ldconfig fallback
- libopenjp2: pkg-config check with ldconfig fallback
- pdfium native lib: runtime detection >= 6555
- network reachability: HEAD example.com 5s timeout
- cache directory: writable + 1 GiB free + layout version
- profile search path: YAML parse + PROFILE_SECRETS_FORBIDDEN
- ulimit -n: getrlimit check >= 1024
- available RAM: /proc/meminfo or sysctl
- system locale: UTF-8 check
- temp dir writable: TMPDIR + 100 MiB free

All checks feature-gated appropriately. Panic-safe via run_check_safe().
CLI output layer integrated with --json and --features flags.

Acceptance criteria:
-  Unit tests for OK/WARN/FAIL paths in each check
-  Runtime < 6s (network: 5s, others: <100ms)
-  Panic catching via catch_unwind
-  Feature-gated checks return NotApplicable
-  pkg-config fallback to ldconfig
-  Profile secret detection with PROFILE_SECRETS_FORBIDDEN

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-05-23 07:05:49 -04:00
jedarden
e2c1e2817b feat(pdftract-2i6rt): implement cache CLI subcommand and HTTP integration
This commit implements Phase 6.9.6: surfacing the cache as user-visible
CLI and HTTP affordances.

## Changes

- Add `pdftract cache` subcommand with stats/clear/purge actions
  - `stats DIR`: show entry count, size, hit ratio, age distribution
  - `stats DIR --json`: emit JSON with same fields
  - `clear DIR`: delete all entries (preserves index.json/sentinel)
  - `purge DIR --older-than 30d`: delete entries older than duration
  - `purge DIR --version '<1.0.0'`: version constraint purge (stub)

- Add global flags to extract-style subcommands
  - `--cache-dir DIR`: enable cache at directory
  - `--cache-size SIZE`: set LRU size limit (default 1 GiB)
  - `--no-cache`: disable cache for this call

- Add `X-Pdftract-Cache: hit|miss|skipped` HTTP header on /extract endpoints
  - Set in response headers before body streaming

- Add JSON metadata fields
  - `metadata.cache_status`: "hit" | "miss" | "skipped"
  - `metadata.cache_age_seconds`: integer seconds (present only on hit)

## Acceptance Criteria

-  pdftract cache stats on empty dir: "Entries: 0"
-  pdftract cache stats on populated dir: correct counts and ratios
-  pdftract cache clear -y: deletes entries, preserves index/sentinel
-  pdftract cache purge --older-than: deletes old entries
-  extract --cache-dir: metadata.cache_status populated
-  extract second run: cache_status "hit" with age
-  extract --no-cache: cache_status "skipped"
-  HTTP serve: X-Pdftract-Cache header present
-  --cache-size parsing: 4GiB → 4 * 1024^3 bytes

## Modules

- crates/pdftract-cli/src/cache_cmd.rs: subcommand implementation
- crates/pdftract-cli/src/serve.rs: HTTP handler integration
- crates/pdftract-cli/src/main.rs: CLI flag definitions
- crates/pdftract-core/src/cache/mod.rs: extract_with_cache() integration
- crates/pdftract-core/src/extract.rs: cache_status metadata fields

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 06:33:43 -04:00
jedarden
8c9a940159 feat(pdftract-15pz8): implement multi-process safe cache operations
Implements Phase 6.9.5: atomic file writes and concurrent access safety
for multiple pdftract processes sharing the same cache directory.

## Changes

- Add `multi_process.rs` module with atomic write/read primitives
- Atomic write protocol: temp file + fsync + rename
- Reader protocol with corruption handling (deletes corrupt entries)
- Startup cleanup of stale temp files (> 1 hour old)
- fsync control via PDFTRACT_CACHE_NO_FSYNC env var
- No distributed locks - tolerates duplicated work on first-miss races

## Module structure

- `Writer`: Atomic cache entry writes via temp + rename
- `Reader`: Safe reads with decompression and corruption detection
- `cleanup_stale_temp_files()`: Startup cleanup for crash-recovered temp files

## Acceptance criteria met

- [x] Concurrent extractors on same fingerprint: both succeed; no deadlock
- [x] Reader sees fully-decompressable entry always (never torn write)
- [x] 8 concurrent writers writing 8 different keys: all materialize correctly
- [x] Corrupt entry on disk: treated as miss; entry deleted
- [x] Stale temp file > 1 hour old: cleaned up at startup
- [x] Stress test: 4 processes × 100 iterations → no errors

## Tests

- 18 tests in `multi_process.rs`
- 92 total cache module tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 05:31:11 -04:00
jedarden
7ea539f8aa feat(pdftract-39g4j): implement --receipts CLI flag + ExtractionOptions.receipts threading
- Add value_parser = ["off", "lite", "svg"] to --receipts CLI flag for clap validation
- Add receipts field to ExtractTextArgs and ExtractMarkdownArgs in MCP tools args
- Add ExtractionOptions and ReceiptsMode to pdftract-core (options.rs module)
- Expose options module in pdftract-core/lib.rs

The CLI now validates receipts mode at parse time with helpful error messages.
MCP tools accept receipts argument matching the schema defined in sibling 6.7.5.
ExtractionOptions struct provides the threading mechanism for the extraction pipeline.

Acceptance criteria:
- PASS: CLI validates --receipts values (off/lite/svg only)
- PASS: CLI shows proper help text with possible values
- PASS: ExtractionOptions serializes for HTTP/MCP transport
- PASS: MCP tools args have receipts field
- WARN: Full extraction implementation pending (deferred to extraction beads)

Closes pdftract-39g4j

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 04:07:23 -04:00
jedarden
7566ab0f0f feat(pdftract-36wlt): implement verify-receipt subcommand + verifier protocol
Implement the pdftract verify-receipt subcommand and the underlying verifier
protocol. The verifier validates receipts against original PDFs by checking:
(1) PDF fingerprint matches, (2) at least one span has bbox overlap >= 90% IoU,
(3) that span's NFC-normalized SHA-256 equals the receipt's content_hash.

Modules:
- crates/pdftract-core/src/receipts/verifier.rs: verifier protocol logic
- crates/pdftract-cli/src/verify_receipt.rs: CLI integration
- crates/pdftract-core/src/document.rs: PDF parsing helpers

Exit codes:
- 0: success
- 10: fingerprint mismatch
- 11: bbox mismatch (no span meets 90% IoU threshold)
- 12: content hash mismatch
- 1: extraction failed

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 04:00:15 -04:00
jedarden
210c40de8c feat(pdftract-mcp): add MCP server implementation changes
Changes from Phase 6.7 child beads that were not committed earlier:

- Add subtle dependency for constant-time token comparison
- Add root directory for path-traversal protection in HTTP+SSE transport
- Update MCP server state to support --root flag
- Minor fixes and improvements across MCP modules

These changes support the 7 closed child beads:
- pdftract-5xq16: JSON-RPC 2.0 framing layer
- pdftract-67tm8: stdio transport
- pdftract-g0ro2: HTTP+SSE transport
- pdftract-24kut: transport mutual exclusion enforcement
- pdftract-1rami: tool catalog (10 tools)
- pdftract-6696g: path-traversal protection
- pdftract-zltqd: bearer-token auth

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 03:09:56 -04:00
jedarden
539627795b feat(pdftract-g0ro2): implement MCP HTTP+SSE transport with integration tests
Implements the HTTP+SSE transport for the MCP server per bead pdftract-g0ro2.
All acceptance criteria PASS.

Routes:
- POST /: JSON-RPC requests (single or batch)
- GET /sse: Server-Sent Events for notifications
- GET /health: Health check (auth-exempt)

Key features:
- Reuses axum/tokio/tower-http from Phase 6.4 (no new deps)
- Bearer token auth (from sibling bead 6.7.7)
- Request body limit (256 MB default, configurable via --max-upload-mb)
- SSE keepalive every 30 seconds
- Broadcast channel for fan-out notifications
- Backpressure handling (drops lagged clients with WARN log)
- 100-client SSE limit (MAX_SSE_CLIENTS)
- Custom 413 Payload Too Large JSON response
- Batch request support per JSON-RPC 2.0 spec

All 10 integration tests pass:
- test_post_tools_list: POST / returns tool catalog
- test_get_sse_stream: GET /sse opens SSE stream with keepalive
- test_50_concurrent_clients: 50 concurrent clients succeed
- test_health_during_load: GET /health returns 200 under load
- test_post_batch_request: Batch requests return batch responses
- test_post_payload_too_large: POST / over limit returns 413 with JSON body
- test_auth_required_for_non_loopback: Bearer auth returns 401 with WWW-Authenticate
- test_post_single_request_returns_single_response: Single request returns single response
- test_unknown_method: Unknown method returns method_not_found error
- test_get_health: GET /health returns 200 with version info

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 00:35:14 -04:00
jedarden
c4ff5194dd feat(pdftract-67tm8): implement MCP stdio transport with integration tests
Implements the stdio transport for the MCP server, enabling communication
with local agents (Claude Desktop, Claude Code, Continue, Cursor) over
standard input/output with Content-Length framing.

Core features:
- LSP-style Content-Length framing with \r\n terminators
- JSON-RPC 2.0 message parsing and serialization
- INV-9 compliance: stdout contains only JSON-RPC frames
- Panic hook redirects panics to stderr
- SIGTERM handler for graceful shutdown
- Parse errors return -32700 with id: null, then continue

Acceptance criteria:
-  Piping tools/list with framing produces expected response < 50ms
-  EOF on stdin → clean exit within 100ms
-  Malformed JSON → -32700 error, subsequent requests work
-  No println!/log output to stdout (INV-9 enforced)
-  Panics go to stderr, no partial JSON on stdout
-  SIGTERM → exit 0, SIGINT → immediate non-zero exit

Tests added:
- crates/pdftract-cli/tests/mcp-stdio.rs (8 integration tests, all pass)
- All 49 existing unit tests continue to pass

Refs: pdftract-67tm8, plan Phase 6.7.2
2026-05-23 00:16:42 -04:00
jedarden
f0919e67d8 feat(pdftract-3gk5): implement SLSA Level 3 provenance generation
- Wire generate-provenance and verify-provenance steps into workflow DAG
- Update publish-if-tag to upload multiple.intoto.jsonl to GitHub Release
- Fix provenance reproducibility by using SOURCE_DATE_EPOCH from git commit
- Docker images already have cosign attest --type slsaprovenance

Acceptance criteria:
- PASS: generate-provenance step wired into DAG
- PASS: provenance uploaded to GitHub Release
- PASS: Docker image cosign attest already implemented
- WARN: Full slsa-verifier verification requires OIDC issuer registration
- PASS: Provenance is reproducible using git commit timestamp
- PASS: Automated smoke test validates JSON structure

Refs: pdftract-3gk5, plan line 3415 (Signing and Provenance)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 23:27:41 -04:00
jedarden
6a35bdd869 feat(pdftract-29z7b): implement unified diagnostic system + CLI commands
- Added `cmd_explain_diagnostic` function to CLI for detailed diagnostic code explanation
- Added `--list-diagnostics` and `--explain-diagnostic <code>` CLI commands
- Verified all Phase 1.1-1.5 modules use unified DiagCode (lexer, parser, xref, stream, catalog, outline, pages)
- DIAGNOSTIC_CATALOG provides metadata for all 61 diagnostic codes
- Diagnostic struct size: 56 bytes (within 48-64 target range)
- emit! macro provides ergonomic diagnostic emission
- INV-8 maintained: no panics in error paths

All diagnostic codes follow naming convention:
- STRUCT_*: PDF structure errors
- STREAM_*: Stream decoder errors
- XREF_*: Cross-reference table errors
- ENCRYPTION_*: Encryption-related errors
- OCR_*: OCR pipeline errors
- REMOTE_*: Remote source errors
- PAGE_*: Page-level errors
- FONT_*: Font pipeline errors
- GSTATE_*: Graphics state errors
- LAYOUT_*: Layout and reading order errors
- MCP_*: MCP server errors
- CACHE_*: Cache errors

References: Phase 1.6 (error recovery), INV-8, Phase 0.4 (clippy enforces doc comments)
2026-05-22 22:38:31 -04:00
jedarden
e0b293c3d6 fix(pdftract-2a6rk): fix xref.rs u64 literal overflow in proptest
Fixed compilation error in xref.rs where u64 literal 0x5DEECE66D was used
with u32 state, causing overflow. Changed state to u64 for proper Java
Random algorithm behavior.

The OCG /OCProperties parsing implementation was already complete and
all tests pass. See notes/pdftract-2a6rk.md for verification.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:26:27 -04:00
jedarden
9aa26a449e docs(pdftract-49f8): establish Cargo.lock policy and documentation
This commit implements the Cargo.lock policy for reproducible builds
across all workspace members (pdftract-core, pdftract-cli, pdftract-py).

Changes:
- Add CONTRIBUTING.md with lockfile-update workflow documentation
- Add .renovaterc.json for weekly lockfile-only PRs (human-gated)
- Add crates/pdftract-core/README.md with rationale for checked-in lockfiles
- Add notes/pdftract-49f8.md with verification note

The Argo workflow updates (pdftract-ci.yaml) are committed separately
in the declarative-config repo.

Acceptance criteria:
- PASS: Cargo.lock tracked by git, not in .gitignore
- PASS: Argo workflow templates document --locked/--frozen requirements
- WARN: Enforcement to be completed when placeholder templates are implemented
- WARN: Binary reproducibility verification deferred to pdftract-build-binaries implementation

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 18:13:14 -04:00
jedarden
660a9401ef feat(pdftract-59zz): implement MCP bearer token ingress channels and TH-03 enforcement
Implements secure MCP bearer-token ingress channels and TH-03 startup abort
enforcement per plan lines 874, 915-921, 922-924.

## Changes
- Add `--auth-token-file PATH` flag (RECOMMENDED channel)
- Add `PDFTRACT_MCP_TOKEN` env var support
- Reject `--auth-token VALUE` unless `PDFTRACT_INSECURE_CLI_TOKEN=1`
- Enforce TH-03: require token for non-loopback bind addresses (exit 78)
- Loopback exemption for 127.0.0.0/8 and ::1/128

## Files
- crates/pdftract-cli/src/mcp/auth.rs: Token resolution with priority order
- crates/pdftract-cli/src/mcp/bind.rs: TH-03 bind security check
- crates/pdftract-cli/src/mcp/server.rs: MCP server entry point
- crates/pdftract-cli/src/mcp/mod.rs: Module exports
- crates/pdftract-cli/src/main.rs: CLI arguments
- crates/pdftract-cli/Cargo.toml: Add secrecy, tempfile dependencies

## Acceptance Criteria
-  --auth-token-file PATH flag implemented
-  PDFTRACT_MCP_TOKEN env var resolved
-  --auth-token VALUE rejected (exit 64) unless PDFTRACT_INSECURE_CLI_TOKEN=1
-  mcp --bind ADDR with non-loopback ADDR and no token: aborts with exit 78
-  mcp --bind ADDR with loopback ADDR and no token: succeeds
-  mcp --bind ADDR with token: succeeds regardless of address
- ⏸️ Inspector token: Phase 7.9 (not yet implemented)
- ⏸️ TH-03 test: separate bead

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 02:47:54 -04:00
jedarden
b535638104 feat(pdftract-2bsfc): implement document catalog parser with PageLabels number tree
Implement the document catalog parser (/Root traversal) for PDF documents.
The catalog parser extracts all key entries from the document catalog
including Pages, Outlines, MarkInfo, StructTreeRoot, AcroForm, Names,
Metadata, PageLabels, OCProperties, OpenAction, AA, and Version.

Key structures:
- MarkInfo: parses /MarkInfo dictionary with is_tagged, user_properties, suspects
- PageLabelStyle: enum for all label styles (D, R, r, A, a)
- PageLabel: single page label with style, prefix, and start value
- PageLabelsTree: number tree parser for /PageLabels with /Nums and /Kids support
- OcProperties: stub for OCG implementation (delegated to dedicated bead)
- Catalog: main catalog struct with all required and optional fields

Number tree implementation:
- Parses /Nums arrays (leaf nodes with alternating key-value pairs)
- Supports /Kids arrays (internal nodes for recursive tree traversal)
- Provides get_label_with_start() and get_label() methods for lookup
- Correctly formats roman numerals (uppercase/lowercase) and letter sequences

All 27 tests pass including proptests for fuzzing robustness (INV-8).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 23:45:45 -04:00