Documents the implementation of the pdftract-crates-publish WorkflowTemplate
in jedarden/declarative-config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The OCG implementation was already complete in ocg.rs. All 20 tests pass:
- BaseState parsing (ON/OFF/Unchanged)
- /ON and /OFF array override handling
- OCMD policy preservation (AllOn, AnyOn, AllOff, AnyOff)
- INV-8 compliance verified via proptests
Phase 3 will consume OcProperties via is_visible() to suppress
glyphs in /OC /OCGRef BDC blocks when the referenced OCG is OFF.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Add details about the BytesSource cleanup bug fix and clarify that the
contract defines 7 error kinds, not 8 as initially stated in the task.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add source Source parameter to invoke, invokeJSON, invokeString, invokeStream
- Change BytesSource from []byte type to struct with data and tmpPath fields
- Add proper cleanup of temporary files after subprocess execution
- Fix source parameter pass-through in Extract, ExtractText, ExtractMarkdown, GetMetadata, Hash, Classify
This ensures BytesSource temporary files are cleaned up after use, preventing
file descriptor leaks. The BytesSource now creates a temp file on demand and
cleans it up automatically via defer in the invoke methods.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implement the publish-if-tag step in pdftract-ci that activates on
version tags (v*.*.*) and publishes cross-compiled binaries to
GitHub Releases.
Changes:
- Add tools/extract-release-notes.sh script for CHANGELOG parsing
- Update publish-if-tag template in pdftract-ci.yaml:
- Downloads all 5 build artifacts from build-matrix
- Generates SHA256SUMS checksums
- Extracts release notes from CHANGELOG.md
- Creates GitHub Release via gh CLI
- Supports both stable and pre-release tags (--prerelease flag)
- Uses --clobber for idempotent re-runs
The step uses Chainguard's gh:latest image and authenticates via
github-pdftract-release Secret (GH_TOKEN key). Optional signing
infrastructure is deferred to Release Engineering epic.
Co-Authored-By: Claude Code (glm-4.7) <noreply@anthropic.com>
Add quality-matrix implementation to pdftract-ci with msrv-check step
using rust:1.78-slim to detect usage of newer Rust features.
Changes:
- .ci/argo-workflows/pdftract-ci.yaml: Implement quality-matrix DAG with
msrv-check, clippy-fmt, and cargo-audit templates
- CHANGELOG.md: New file documenting MSRV bump policy (MINOR version
event, warning period, update checklist)
The MSRV gate prevents silent drift that would break downstream consumers
on older toolchains. Any Rust 1.79+ feature (e.g., let-else, core::error::Error)
will fail the msrv-check step, triggering a policy review.
See notes/pdftract-2w02.md for acceptance criteria verification.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Add MSRV (Minimum Supported Rust Version) pinning to 1.78 for
pdftract-core and pdftract-cli. The MSRV gate prevents silent
absorption of newer Rust features that would break downstream
consumers on older toolchains.
Changes:
- CI: Add quality-matrix DAG with msrv-check step (rust:1.78-slim)
- CI: Add clippy-check, fmt-check, cargo-audit, cargo-deny templates
- README: Add MSRV badge (shields.io)
- clippy.toml: Enable msrv=1.78 for MSRV-aware lints
- CONTRIBUTING.md: Document MSRV bump policy (MINOR version event)
The rust-version was already declared in workspace Cargo.toml;
this bead adds the CI enforcement and documentation.
Refs: pdftract-2w02
Add comprehensive verification note for forward_scan_xref implementation.
The function was already implemented in xref.rs; this note documents
verification of all bead requirements.
Also fix duplicate ObjRef import in parser/mod.rs (ObjRef is defined in
diagnostics module and re-exported).
Bead: pdftract-46lw
The apostrophe in 'banker's_rounding' is invalid Rust 2021 syntax.
Changed to 'bankers_rounding' to fix compilation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The stub template was already created in commit 642949b in
jedarden/declarative-config. This note documents the acceptance
criteria verification status.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Crypt filter was already implemented in the codebase. This note
documents the verification of acceptance criteria and test coverage.
Acceptance criteria verified:
- /Identity crypt passes through unchanged
- Custom crypt returns ENCRYPTION_UNSUPPORTED
- Missing /DecodeParms defaults to /Identity
- Works correctly with FlateDecode
- Comprehensive test coverage including proptests
- INV-8 maintained (no panics)
Also add missing malformed fixture entries to PROVENANCE.md.
Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit implements the Cargo.lock policy for reproducible builds
across all workspace members (pdftract-core, pdftract-cli, pdftract-py).
Changes:
- Add CONTRIBUTING.md with lockfile-update workflow documentation
- Add .renovaterc.json for weekly lockfile-only PRs (human-gated)
- Add crates/pdftract-core/README.md with rationale for checked-in lockfiles
- Add notes/pdftract-49f8.md with verification note
The Argo workflow updates (pdftract-ci.yaml) are committed separately
in the declarative-config repo.
Acceptance criteria:
- PASS: Cargo.lock tracked by git, not in .gitignore
- PASS: Argo workflow templates document --locked/--frozen requirements
- WARN: Enforcement to be completed when placeholder templates are implemented
- WARN: Binary reproducibility verification deferred to pdftract-build-binaries implementation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The workspace-level Cargo.lock is checked into version control
for reproducible builds. All Argo build steps enforce --locked
--frozen to ensure dependency versions match exactly.
This commit includes lockfile updates for new dependencies
(lzw, memchr) added during development.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Configure workspace with pdftract-core, pdftract-cli, pdftract-py members
- Add workspace.package metadata: version, edition, rust-version (1.78), license (MIT OR Apache-2.0)
- Add workspace.dependencies for shared external deps (anyhow, flate2, lzw, memchr, secrecy, serde, thiserror, tracing)
- Create .cargo/config.toml with CI and development build aliases
- All member crates reference workspace metadata via workspace = true
- pdftract-py configured as cdylib with pyo3 extension-module feature
Acceptance criteria:
- PASS: 3 workspace members listed by cargo metadata
- PASS: All crates use workspace metadata references
- WARN: cargo build fails due to code compilation errors (separate concern)
Refs: pdftract-279, plan lines 3343-3367
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The structural token lexer was already fully implemented. All 84 lexer
tests pass, covering all acceptance criteria:
- Array/dict delimiters ([], <<>>)
- Keywords (true, false, null, obj, endobj, stream, endstream, R)
- Hex string vs dict ambiguity (< vs <<)
- Stream header validation (\n or \r\n only, lone \r is invalid)
- Case-sensitive keyword matching
This commit fixes a pre-existing compilation error in xref.rs where
forward_scan_memory() called parse_obj_header_at_memory() which didn't
exist. Added the missing function as a byte-slice variant of
parse_obj_header_at() for efficient memory-based scanning.
Verification: notes/pdftract-5upi.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implement TH-07 password ingress channels for CLI:
- --password-stdin flag (reads one line from stdin)
- PDFTRACT_PASSWORD env var
- --password VALUE (rejected unless PDFTRACT_INSECURE_CLI_PASSWORD=1)
Exit code 64 for insecure password usage with stderr hint.
Stderr warning emitted when --password VALUE accepted via opt-in.
Priority order: stdin > env var > value (opt-in) > none.
Empty password (bare newline) treated as no password.
Acceptance criteria:
- --password-stdin: PASS
- PDFTRACT_PASSWORD: PASS
- --password VALUE rejection (exit 64): PASS
- Stderr warning on opt-in: PASS
- Exit codes: PASS
- Python/MCP/Serve: N/A (crates don't exist yet)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Fix Token::Keyword to use b"..." .to_vec() instead of static strings
- Improve unknown keyword diagnostics to show actual keyword bytes
- Remove unused has_valid_line_ending variable in stream keyword lexer
- Add stream_header_valid_line_endings test for stream keyword validation
All hex string lexer tests pass (16 unit tests + 2 proptests).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bead-Id: pdftract-2hm4
Add two proptests for the PDF hex string lexer to verify robustness
and correctness:
1. proptest_hex_string_never_panics_on_random_bytes: Random byte
sequences starting with '<' (not '<<') never cause panics.
2. proptest_hex_string_roundtrip_via_reencode: Hex decode + re-encode
roundtrip property validates that encoding and decoding are
inverse operations.
The hex string lexer implementation was already present and correct,
with proper handling of odd-length zero padding (<4> -> \x40, not \x04).
All acceptance criteria pass:
- Empty hex string: <> -> b""
- Odd-length single nibble: <4> -> b"\x40" (critical test)
- Standard decoding: <48656C6C6F> -> b"Hello"
- Mixed case: <aBcD> -> b"\xAB\xCD"
- Whitespace ignored: <48 65> -> b"\x48\x65"
- Unterminated with diagnostic: <48 -> b"\x48" + STRUCT_UNTERMINATED_STRING
- Proptests pass: random bytes never panic, roundtrip property holds
- INV-8 maintained: all error paths use diagnostics, no panics
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The PDF name object lexer was already fully implemented with
all acceptance criteria passing. Added verification note documenting
test results.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Bead-Id: pdftract-5dng
Rename all DiagCode enum variants in the lexer to use the STRUCT_ prefix
to match the specification. This clarifies that these diagnostics relate
to structural/lexical issues in PDF documents.
Changes:
- InvalidName -> StructInvalidName
- InvalidHex -> StructInvalidHex
- InvalidOctal -> StructInvalidOctal
- InvalidStreamHeader -> StructInvalidStreamHeader
- UnexpectedEof -> StructUnexpectedEof
- UnterminatedString -> StructUnterminatedString
The hex string lexer implementation was already correct, with proper
handling of:
- Hex digit pair decoding
- Embedded whitespace (PDF spec 7.2.2)
- Odd-length zero padding: <4> -> \x40 (dangling nibble is HIGH)
- Invalid character diagnostics
- Unterminated string diagnostics
All 16 hex string tests pass, including critical tests for odd-length
padding and error handling.
See: notes/pdftract-2hm4.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add verify_receipt method support to Go templates:
- client.go.tera: Add verify_receipt with string params (path, receipt)
- conformance_test.go.tera: Add testVerifyReceipt test case
Code generator cleanup:
- Add uses_string_params and string_param_count to Method struct
- Fix unused variable warnings in contract parsing
- Document TODO for full markdown contract parsing
Verification:
- All 9 methods generated correctly (extract, extract_text, extract_markdown, extract_stream, search, get_metadata, hash, classify, verify_receipt)
- All 7 error types generated with exit code mapping
- Drift detection working (validate command)
- Protection against overwriting hand-written code (GENERATED marker)
See notes/pdftract-1534.md for full acceptance criteria status.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bead-Id: pdftract-1534
Add verify_receipt method support to Go templates:
- client.go.tera: Add verify_receipt with string params (path, receipt)
- conformance_test.go.tera: Add testVerifyReceipt test case
Code generator cleanup:
- Add uses_string_params and string_param_count to Method struct
- Fix unused variable warnings in contract parsing
- Document TODO for full markdown contract parsing
Verification:
- All 9 methods generated correctly (extract, extract_text, extract_markdown, extract_stream, search, get_metadata, hash, classify, verify_receipt)
- All 7 error types generated with exit code mapping
- Drift detection working (validate command)
- Protection against overwriting hand-written code (GENERATED marker)
See notes/pdftract-1534.md for full acceptance criteria status.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two fixes:
1. Hex string lexer now flushes dangling nibble when encountering invalid
characters. For `<4X8Y>`, the X and Y are invalid, so we flush nibble 4
as 0x40, then flush nibble 8 as 0x80, producing `\x40\x80`.
2. Fixed skip_whitespace_and_comments() to properly handle whitespace
after comments. The previous logic only continued looping if the next
byte was `%`, missing cases where whitespace follows a comment.
All 52 lexer tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the conformance test runner pattern for all 10 SDKs as specified
in the plan (line 3547). Each SDK now has a dedicated conformance test runner.
Created:
- tests/sdk-conformance/report-schema.json: JSON schema for conformance reports
- docs/notes/sdk-conformance-runner.md: Pattern documentation and reference
- crates/pdftract-cli/tests/conformance.rs: Rust cargo test target
- tests/conformance/test_conformance.py: Python pytest harness
- tests/conformance/conformance.test.ts: Node.js vitest runner
- tests/conformance/conformance_test.go: Go go test runner
- tests/conformance/ConformanceTest.java: Java JUnit 5 runner
- tests/conformance/ConformanceTests.cs: .NET xUnit runner
- tests/conformance/conformance.c: C standalone binary
- tests/conformance/conformance_test.rb: Ruby minitest runner
- tests/conformance/ConformanceTest.php: PHP PHPUnit runner
- tests/conformance/ConformanceTests.swift: Swift XCTest runner
All runners implement:
- Loading of tests/sdk-conformance/cases.json
- Execution of test cases with language-native method invocations
- Comparison of results against expected values with numeric tolerances
- Emission of machine-readable conformance-report.json
- Non-zero exit on failures/errors for CI gating
Acceptance criteria:
- PASS: All 10 SDKs have language-specific runners
- PASS: Runners consume shared cases.json
- PASS: Runners emit JSON reports matching schema
- PASS: Runners exit non-zero on failure
- WARN: README integration pending SDK repo creation
- WARN: Stub implementations return placeholder results
References:
- Plan line 3547: "Every SDK has a pdftract-sdk-conformance test runner"
- Plan line 3589: "Conformance suite results published as Argo artifact"
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bead-Id: pdftract-5omc
- Add extraction of pdftract_geomean from tool_geomeans array for regression gate
- Fix vector geomean calculation to properly pass bash array values to Python
The benchmark infrastructure was complete but had two bugs:
1. $pdftract_geomean was used but never set (line 308)
2. Vector geomean calculation had broken Python code for array expansion
These fixes ensure the regression and 10x-faster gates will work correctly
once the pdftract binary with extract/grep subcommands is available.
Refs pdftract-60h
Implement the conformance test runner pattern that every SDK will
implement to validate against the shared test suite.
- Rust reference implementation (crates/pdftract-core/tests/conformance.rs)
* Full test suite loader and executor
* Comparison engine with min/max, string constraints, tolerances
* Skip logic for unsupported features and schema versions
* Report generation in JSON format
- CLI compare subcommand (crates/pdftract-cli/src/main.rs)
* pdftract compare - Compare actual vs expected with tolerances
* Cross-language comparison tool to avoid reimplementations
- Documentation (docs/conformance/sdk-contract.md)
* Complete pattern specification with pseudocode
* Per-language runner locations
* CI integration requirements
- Python reference stub (tests/python-conformance/test_conformance.py)
* Full pytest-based implementation following the pattern
Closes: pdftract-5omc
Changes:
- Use pdftract-test-glibc:1.78 image (has aws/b2 CLI preinstalled)
- Use b2-readonly secret instead of armor-secrets
- Update env var names to ARMOR_ACCESS_KEY_ID/ARMOR_SECRET_ACCESS_KEY
- Remove apt-get install step (tools already in image)
The cer-diff tool was already implemented in a previous commit.
This commit fixes the image and secret references per the bead spec.
References pdftract-2t9 acceptance criteria:
- regression-corpus step runs on every PR (✓ already in workflow)
- Uses pdftract-test-glibc:1.78 image (✓ fixed)
- Uses b2-readonly secret (✓ fixed)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add tests/sdk-conformance/ containing the shared, language-neutral test
specification for all pdftract SDKs. The suite includes 32 cases covering
all 9 contract methods (extract, extract_text, extract_markdown,
extract_stream, search, get_metadata, hash, classify, verify_receipt)
across vector, scanned, encrypted, fillable-form, mixed, large, broken,
and remote PDFs.
- cases.json: 32 test cases with id, fixture, method, options, expected,
tolerances, feature tags, and min_schema_version
- schema.json: JSON Schema v7 draft for validating test case structure
- validate_suite.py: Validation script that checks structure and fixture
existence
- fixtures/: Test PDFs organized by category (symlinks to classifier
fixtures for shared files)
See notes/pdftract-1527.md for verification details.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The parse_indirect_object() function was already implemented in
crates/pdftract-core/src/parser/object/parser.rs with all required
functionality:
- Reads 3-token preamble (Integer Integer Obj)
- Parses direct object body
- Expects EndObj token
- Returns PdfIndirect { id, obj }
All acceptance criteria PASS:
- Simple null object test ✅
- Stream object test ✅
- Missing endobj recovery ✅
- Integer overflow clamping ✅
- proptest: random bytes never panic ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add CI validation script for checking unauthorized expose_secret() call
sites. The script validates that all uses of expose_secret() are in
approved locations (SecretFingerprint and test code).
Also add verification note summarizing the bead completion status.
Per pdftract-5l9m acceptance criteria:
- CI grep guard rejects unauthorized expose_secret() call sites
- Verification documents existing SecretString wrapping status
Co-Authored-By: Claude Code <noreply@anthropic.com>
Implement Merkle SHA-256 fingerprint algorithm for PDF structural
fingerprinting as specified in Phase 1.7 of the plan.
Components:
- FingerprintInput struct with page data and catalog flags
- Per-page hashing: content streams (normalized), resources (sorted),
geometry (4dp banker's rounding)
- Structure tree hash for tagged PDFs
- Catalog feature flag byte (encryption, JS, XFA, OCG)
Acceptance criteria:
- INV-3: 100% reproducible fingerprints (test passes)
- INV-13: Output format ^pdftract-v1:[0-9a-f]{64}$ (test passes)
- Performance: 100-page PDF in < 1ms (test passes)
- KU-7: WARN - no linearized fixtures available
Closes pdftract-q15sh
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- book.toml with title, authors, build directory, edit-url-template
- src/SUMMARY.md with complete TOC for all planned sections
- src/introduction.md: what pdftract does and doesn't do (Non-Goals)
- src/installation.md: cargo, pip, Homebrew, Docker; KU-12 caveat verbatim
- src/quickstart.md: five-minute walkthrough with executable commands
- 39 draft placeholder files for CLI reference, schema, profiles, SDKs, advanced topics, troubleshooting, FAQ
mdbook build completes cleanly with zero warnings (linkcheck optional).
See notes/pdftract-1g87.md for verification details.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
