Add explicit enum constraints to page_type, severity, and confidence_source
fields in the generated JSON Schema for better validation.
Changes:
- Modified xtask/src/bin/gen_schema.rs to add explicit enum constraints
during schema generation via add_enum_constraints() function
- page_type enum: ["text", "scanned", "mixed", "broken_vector", "blank", "figure_only"]
- severity enum: ["info", "warning", "error", "fatal"]
- confidence_source enum: ["native", "heuristic", "ocr"]
- Regenerated docs/schema/v1.0/pdftract.schema.json with enum constraints
- Added .github/workflows/schema-gen.yml CI workflow for schema validation
The CI workflow validates:
1. Generated schema matches committed file (fails on diff)
2. JSON syntax is valid
3. Schema structure is correct ($id, $schema, title, $defs)
4. Enum constraints are present and have correct values
This ensures schema changes are reviewable in PRs and forces
developers to commit the updated schema when type definitions change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The schema now reflects the latest doc comments from the Rust types,
including updated descriptions for annotations and other fields.
Changes:
- AnnotationJson description updates (phase 7.6.4 reference)
- Format consistency updates (float vs double)
- Subtype-specific field documentation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add test_page_json_with_page_labels_roman_numerals: verifies page_label
serialization with roman numeral values (i, ii, iii, etc)
- Add test_page_json_without_page_labels_absent: verifies page_label is
absent (null) when PDF has no /PageLabels
- Add test_page_json_page_index_and_page_number_both_present: verifies
both page_index and page_number are always present and page_number = page_index + 1
- Add test_page_json_roundtrip_with_all_fields: verifies full roundtrip
serde preservation of all PageJson fields
- Update docs/schema/v1.0/pdftract.schema.json PageResult definition:
- Add page_number field (1-based, = page_index + 1)
- Add page_label field (optional, from /PageLabels number tree)
- Add width and height fields (page geometry in points)
- Add rotation field (0, 90, 180, 270 degrees)
- Add type field with enum: text, scanned, mixed, broken_vector, blank, figure_only
- Update required fields to include all page-level fields
Acceptance criteria:
✅ Page serializes with both page_index AND page_number
✅ PDF with /PageLabels [{S: "r"}] produces page_label "i", "ii", "iii" etc
✅ PDF without /PageLabels -> page_label absent
✅ JSON Schema enum for page_type includes all values
✅ Roundtrip serde Page test passes
Closes: pdftract-4c8qu
Phase 7.7.3: Add threads field to ExtractionResult with ThreadJson schema integration.
Changes:
- Added ThreadJson and BeadJson structs to schema/mod.rs
- Added thread_to_json() function to threads/mod.rs
- Added build_page_ref_to_index() helper to parser/pages.rs
- Added threads field to ExtractionResult in extract.rs
- Implemented Phase 7.7 extraction logic with discover_threads/walk_beads
- Added threads_to_markdown() and collapse_page_ranges() to markdown.rs
- Updated JSON schema with ThreadJson and BeadJson definitions
- Added thread_to_py() and bead_to_py() conversions in pdftract-py
- Exported ThreadJson, BeadJson from lib.rs
All 32 threads module tests pass. All 35 markdown tests pass.
Verification: notes/pdftract-3h9xo.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implement the --json output sink for pdftract grep with JSON-Lines
format (one match per line). Includes MatchEvent, FileOnlyEvent,
CountEvent structs and JsonSink line-buffered writer.
Key features:
- MatchEvent with all fields (path, page_index, bbox, match_text,
span_text, span_confidence, pdf_fingerprint, crosses_spans)
- crosses_spans omitted when false via skip_serializing_if
- NaN/Infinity in span_confidence replaced with null
- page_index is 0-based (machine convention)
- FileOnlyEvent for -l mode, CountEvent for -c mode
- Line-buffered writes with immediate flush
- JSON schema at docs/schema/v1.0/grep-jsonl.schema.json
Closes: pdftract-5ls35
Implement the xtask gen-schema binary at xtask/src/bin/gen_schema.rs that
derives JSON Schema Draft 2020-12 from the Rust ExtractionResult type via
the schemars crate.
Changes:
- Add stable key sorting (sort_keys_recursive) for byte-identical output
- Set $id to stable URL: https://pdftract.com/schema/v1.0/pdftract.schema.json
- Set title to "pdftract Output v1.0"
- Add cargo alias `gen-schema` for convenient invocation
- Emit schema to docs/schema/v1.0/pdftract.schema.json
The schema is generated from the Rust types with schemars derives, ensuring
the JSON schema is always in sync with the source types.
Acceptance criteria:
- cargo gen-schema regenerates docs/schema/v1.0/pdftract.schema.json
- Generated schema validates against JSON Schema Draft 2020-12
- Schema $id is the stable URL
- Title is "pdftract Output v1.0"
- Stable ordering: regenerating twice produces byte-identical output
- All expected types appear in $defs (BlockJson, SpanJson, PageResult, etc.)
Note: page_type and confidence_source enums are not yet implemented in the
Rust types (marked as TODO in schema/mod.rs). These will be added by sibling
beads pdftract-1ob and pdftract-1f8we respectively.
Closes: pdftract-5nv9h
Phase 7.4.5 implementation: Wire combined Vec<(String, FormFieldValue)> from
combiner into document-level /form_fields JSON output with tagged union schema.
- Add FormFieldJson, FormFieldTypeJson, FormFieldValueJson, ChoiceValueJson to schema
- Add form_fields: Vec<FormFieldJson> to ExtractionResult (always emitted, empty when none)
- Implement acro_field_to_value() converter for Phase 7.4.2 type-specific extraction
- Wire form field extraction in extract_pdf(): walk AcroForm, extract XFA, combine with XFA-wins
- Add convert_form_field_to_json() helper for FormFieldValue → FormFieldJson conversion
- Update docs/schema/v1.0/pdftract.schema.json with form_fields $defs and required field
- Add form_fields_to_markdown() to markdown module for Form Fields footer table
Schema shape: /form_fields is array of {name, type, value, default?, page_index?, rect?,
required, read_only, multiline?, max_length?, options?, multi_select?, selected?,
state_name?, pushbutton?, radio?}. Type field is tagged enum: "text", "button", "choice",
"signature". Value field varies by type (string|boolean|string|array|uint|null).
Closes: pdftract-5qca
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
