jedarden 2f010c51fb feat(pdftract-206o6): implement scientific_paper profile with fixtures and tests

Author profiles/builtin/scientific_paper.yaml per Phase 7.10 YAML schema:
- Match predicates: text_contains (Abstract, References, doi:, arXiv:, Bibliography)
- Structural predicates: has_math, heading_depth, page_count
- Extraction tuning: xy_cut reading order for 2-column layout
- Fields: title, authors, abstract, doi, journal, publication_date, references

Add 5 fixtures covering diverse scientific paper types:
- arXiv preprint (CC-BY license)
- PLOS ONE journal article
- IEEE-style 2-column paper
- Nature-style single-column with sidebar
- ACM/IEEE conference proceedings

Add comprehensive regression tests in test_scientific_paper.rs:
- Profile schema validation
- Fixture structure verification
- Expected output consistency checks
- Match predicate validation
- Fixture diversity verification
- xy_cut reading order verification
- DOI regex format validation

Co-Authored-By: Claude Code (claude-opus-4-7) <noreply@anthropic.com>

2026-05-27 20:19:10 -04:00

2.2 KiB

Raw Blame History

Scientific Paper Profile Fixtures

This directory contains test fixtures for the scientific paper document profile.

Fixture Types

arxiv_paper - arXiv preprint with CC-BY license, typical academic structure with Abstract, Introduction, Methods, Results, Discussion, References
plos_one_paper - PLOS ONE journal article with DOI, open access formatting, single-column layout
ieee_paper - IEEE-style 2-column journal article with mathematical equations, numbered references
nature_paper - Nature-style single-column article with sidebar layout, Received/Accepted dates
conference_paper - ACM/IEEE conference proceedings with DOI, author affiliations, structured references

Expected Output Format

Each fixture should have a corresponding *-expected.json file with the following structure:

{
  "metadata": {
    "document_type": "scientific_paper",
    "document_type_confidence": 0.XX,
    "document_type_reasons": [...],
    "profile_name": "scientific_paper",
    "profile_version": "1.0.0",
    "profile_fields": {
      "title": "...",
      "authors": ["..."],
      "abstract": "...",
      "doi": "...",
      "journal": "...",
      "publication_date": "YYYY-MM-DD",
      "references": "..."
    }
  }
}

Profile Fields

The scientific paper profile extracts the following fields:

title: Paper title (region: top_quarter, pick: largest_font)
authors: Author list (region: top_quarter, pick: nearest_below)
abstract: Abstract text (near: "Abstract", region: top_half)
doi: Digital Object Identifier (regex match)
journal: Journal or publication name (region: top_eighth)
publication_date: Publication date (near: "Published", "Received", "Accepted")
references: References section (region: bottom_half, after "References" heading)

Provenance

All fixtures should be sourced from publicly available academic papers with appropriate licenses or created synthetically with clear provenance documentation. See PROVENANCE.md for details on each fixture.

TODO

Acquire or create PDF files for each fixture type
Validate extraction accuracy against expected outputs
Document extraction limitations (e.g., 3-column layouts, unusual author formats)

2.2 KiB Raw Blame History