Commit graph

9 commits

Author SHA1 Message Date
jedarden
62a36ea756 docs(pdftract-3eohy): add rustdoc examples to Glyph and Span types
- Add worked example to Glyph struct showing all 11 fields
- Add worked example to Span struct showing all 10 fields
- Examples use rust,no_run for internal dependencies
- cargo doc passes with docs.rs feature set
- Verification note added at notes/pdftract-3eohy.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 01:16:24 -04:00
jedarden
80dbf0f703 feat(profiles): add profile infrastructure and initial fixtures
- Add profile source modules: apply_profile, extraction, extraction_loader, field_extractor, match_eval
- Add profiles CLI subcommand (profiles_cmd.rs)
- Update all 9 built-in profile YAMLs (invoice, receipt, contract, scientific_paper, slide_deck, form, bank_statement, legal_filing, book_chapter)
- Add 50 invoice fixture PDFs
- Add 2 receipt fixture PDFs

Part of: pdftract-3a310 (Phase 7.10 coordinator)
2026-05-31 15:10:51 -04:00
jedarden
bb7146cffe fix(pdftract-2uk9z): wrap native module results in typed Python objects
The native PyO3 module returns raw dicts via pythonize, but the Python SDK
API expects typed dataclass objects (Document, Page, Metadata, etc.) to be
consistent with the subprocess fallback and test expectations.

Updated wrapper functions in __init__.py to convert native results:
- extract(): wraps dict in Document.from_dict()
- extract_stream(): wraps yielded page dicts in Page.from_dict()
- get_metadata(): wraps dict in Metadata()
- hash(): wraps string in Fingerprint.from_string()
- classify(): wraps dict in Classification()
- search(): wraps yielded match dicts in Match

The native PyO3 entry points (extract, extract_text, extract_stream) were
already implemented with:
- extract: uses extract_pdf + pythonize for PyDict conversion
- extract_text: uses extract_text for plain String return
- extract_stream: uses extract_pdf_streaming with custom StreamIterator

All kwargs parsing with strict validation (unknown kwargs raise TypeError)
was already in place.

Acceptance criteria:
- pdftract.extract() returns Document object with pages/metadata
- pdftract.extract_text() returns plain text string
- pdftract.extract_stream() yields Page objects
- Unknown kwarg raises TypeError
2026-05-28 21:18:38 -04:00
jedarden
adaf27be85 feat(pdftract-64p5): implement classify CLI subcommand and --auto flag
- Implement pdftract classify command with JSON output
- Load built-in profiles + custom profiles from --profiles DIR
- Output format: {"document_type":"invoice","confidence":0.87,"reasons":[...],"runner_up":"receipt","runner_up_confidence":0.42}
- Support --top-k, --exit-on-unknown, --pretty flags
- Implement --auto flag for extract subcommand
- Add path traversal protection for profiles directory
- Add load_profiles_from_file() and load_profiles_from_dir() to profiles/loader

Closes: pdftract-64p5
2026-05-24 15:16:56 -04:00
jedarden
71705ed77b feat(profiles): implement built-in classification profiles (5.6.4)
Add 9 built-in classification profile definitions as YAML files bundled
via include_str! for the document type classifier (Phase 5.6).

- Create profiles/builtin/classification/{invoice,receipt,contract,scientific_paper,slide_deck,form,bank_statement,legal_filing,book_chapter}.yaml
- Implement load_builtins() in profiles module with profiles feature gate
- Each profile uses MatchPredicate schema with text patterns, structural signals, page counts
- Add comprehensive unit tests for profile loading and feature gate

Closes: pdftract-5sdd

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-24 15:04:43 -04:00
jedarden
51cb277535 feat(pdftract-49cn): implement feature signal extraction for classifier
Implements Phase 5.6.3: FeatureSignals extraction computed during Phase 4 assembly.

- Added profiles/signals.rs module with PageSignalAccumulator and extract_feature_signals()
- Predefined text patterns: currency symbols, ISO dates, INVOICE, WHEREAS, Abstract, References, page numbers, bullets, math operators
- Per-page signal extraction: text content, fonts, table count, heading depth, glyph density
- Document-level aggregation: page count, font diversity, presence flags (signature field, form field, math operators, bullet lists, footer page numbers)
- All regex patterns compiled once via OnceLock for performance
- 23 unit tests covering all functionality

Closes: pdftract-49cn
2026-05-24 11:01:18 -04:00
jedarden
865429d5f6 feat(pdftract-2iyk): implement classifier engine
Implements Phase 5.6.2 classifier engine that evaluates document type
profiles against extracted feature signals.

- ClassifierEngine: evaluates profiles, computes normalized scores,
  returns highest-scoring profile above threshold
- FeatureSignals: struct containing all metrics for predicate matching
- ClassificationResult: document_type, confidence, reasons, runner_up
- Score normalization: matched_weight / total_weight to [0, 1]
- Predicate evaluation: all MatchPredicate variants supported
- Regex caching: OnceLock-based cache for TextMatchesRegex
- Unit tests: 28 tests covering invoice, scientific_paper, unknown
  classification, score normalization, tie-breaking, determinism

Closes: pdftract-2iyk
2026-05-24 10:23:58 -04:00
jedarden
7df83c64dd feat(pdftract-51bk): implement ProfileType, Profile, MatchPredicate types
- Add ProfileType enum with 10 variants (invoice, receipt, contract, etc.)
- Add Profile struct with name, type, predicates, threshold (default 0.6)
- Add MatchPredicate enum with 12 predicate kinds (text_contains, text_matches_regex, structural_has_table, etc.)
- All types support serde YAML serialization/deserialization
- ProfileType uses snake_case for YAML compatibility
- MatchPredicate uses tagged enum representation (kind field)
- Comprehensive unit tests for all variants and roundtrip serialization

Closes: pdftract-51bk
2026-05-24 09:34:40 -04:00
jedarden
0dcae8766e feat(pdftract-kdp6): implement profile loader secret key hardening
Add PROFILE_SECRETS_FORBIDDEN diagnostic and enhanced profile validation
to prevent accidental publication of credentials in profile YAML files.

Changes:
- Add DiagCode::ProfileSecretsForbidden to diagnostics catalog
- Create pdftract-core/src/profiles/ module with loader.rs
- Implement separator-tolerant key matching (api_key/apiKey/api-key/api.key)
- Expand forbidden keys from 7 to 17 entries
- Add line number detection for error reporting
- Update ProfilePathCheck to use enhanced validation

Closes: pdftract-kdp6

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-24 04:41:04 -04:00