pdftract/profiles/community/CONTRIBUTING.md

Contributing Document Profiles

Thank you for your interest in contributing document profiles to pdftract! Community profiles help make pdftract more useful for everyone by supporting specialized document types and industry-specific formats.

This guide covers how to create, test, and submit document profiles for inclusion in the pdftract community profile collection.

Overview

Document profiles (pdftract) define how to recognize and extract structured data from specific types of documents (invoices, receipts, contracts, scientific papers, etc.). Each profile specifies:

• Match criteria: How to recognize this document type
• Extraction strategy: How to read and parse the document content
• Field definitions: What structured data to extract (dates, amounts, line items, etc.)

Community profiles are stored in profiles/community/ and can be used by anyone with pdftract.

Quick Start

If you're new to profile authoring, start with an existing profile:

# Export a built-in profile to use as a template
pdftract profiles export invoice > my-profile.yaml

# Edit it to match your document type
vim my-profile.yaml

# Validate your profile
pdftract profiles validate my-profile.yaml

# Test it on a sample document
pdftract extract --profile my-profile sample.pdf

File Naming Convention

Profile directories in profiles/community/ should follow these conventions:

Directory Structure

Each profile should be organized as a directory:

profiles/community/
├── .gitkeep                    # Placeholder file (empty)
├── CONTRIBUTING.md             # This file
└── <profile-name>/
    ├── profile.yaml            # Required: Profile definition
    ├── README.md               # Required: Profile documentation
    └── fixtures/               # Optional: Test documents
        ├── sample-01.pdf
        └── sample-02.pdf

Profile Name Requirements

• Use lowercase, hyphen-separated names: vendor-invoice, w2-form, packing-slip
• Be specific: If a profile is for a specific region or industry, include it in the name
  • ✅ eu-invoice-vat, medical-billing-statement
  • ❌ invoice, statement
• Avoid trademarked terms: Don't use company names or product names in profile names
  • ✅ saas-subscription-invoice
  • ❌ salesforce-invoice, aws-invoice

File Format

• profile.yaml: YAML format (see Profile Authoring Guide)
• README.md: Markdown format with specific sections (see below)
• fixtures/: Sample PDFs for testing (optional but recommended)

Required Profile Fields

Every profile.yaml must include these top-level fields:

name: profile-name              # Unique identifier (lowercase, hyphens)
description: One-line summary   # What this profile extracts
priority: 50                    # 1-100, higher = more specific

match:                          # Document recognition rules
  all:
    - ...                       # All conditions must match
  any:
    - ...                       # At least one must match
  none:
    - ...                       # None of these may match

extraction:                     # Extraction strategy
  reading_order: line_dominant
  table_detection: strict_borders
  # ... (see docs for full schema)

fields:                         # Fields to extract
  field_name:
    type: string|date|decimal|int|bool|array
    extraction:
      # ... extraction strategy

Schema Requirements

• name: Must match directory name
• description: Clear, concise description (50-100 characters)
• priority: 1-100, where higher values indicate more specific profiles
  • 10-30: Generic/broad profiles (e.g., document)
  • 40-60: Standard document types (e.g., invoice, receipt)
  • 70-100: Highly specific profiles (e.g., w2-tax-form-2024)
• match: Must define at least one condition
• fields: Must define at least one field

Validation Rules

Profiles are automatically validated on submission. Common validation failures:

• ❌ YAML syntax errors (indentation, unescaped characters)
• ❌ Missing required fields (name, description, priority, match, fields)
• ❌ Forbidden keys (api_key, password, token, secret, credential)
• ❌ Invalid field type combinations (type: decimal with parse: bool)
• ❌ Malformed regex patterns or YAML structure

See Profile Authoring Guide for common YAML pitfalls.

Local Testing Workflow

Before submitting a profile, test it thoroughly:

1. Validate the Profile

pdftract profiles validate profiles/community/<profile-name>/profile.yaml

Fix any validation errors before proceeding.

2. Test Against Sample Documents

# Test extraction on a sample document
pdftract extract --profile profiles/community/<profile-name>/profile.yaml sample.pdf

# Inspect the output
pdftract extract --profile profiles/community/<profile-name>/profile.yaml sample.pdf | jq .

3. Test Edge Cases

Test your profile against:

• ✅ Documents that should match (positive cases)
• ❌ Documents that should NOT match (negative cases)
• 📄 Variations (different layouts, regions, fonts)
• 🚫 Edge cases (missing fields, corrupted pages, blank pages)

4. Performance Check

Ensure extraction completes in reasonable time:

time pdftract extract --profile profiles/community/<profile-name>/profile.yaml sample.pdf

Typical targets:

• Simple profiles: < 2 seconds per page
• Complex profiles (tables, multi-page): < 5 seconds per page

README.md Template

Every profile should include a README.md with these sections:

# <Profile Name> Profile

<One-line description>

## Match Criteria Summary

A document matches this profile when... (describe the key indicators)

## Extracted Fields

| Field | Type | Description | Example Value | Source Hint |
|-------|------|-------------|----------------|-------------|
| field1 | string | Description | "example" | regex/near/region |
| field2 | decimal | Description | 123.45 | table: largest_table |

## Known Limitations

- Limitation 1
- Limitation 2

## Sample Input

Example fixtures demonstrating this profile are available in `fixtures/`.

## Configuration Tips

(Optional) Tips for customizing or overriding this profile.

---

*This README was auto-generated from `profile.yaml`.*

See invoice README for a complete example.

PR Submission Process

1. Fork and Branch

# Fork the repository and create a feature branch
git checkout -b add-profile-<profile-name>

2. Add Your Profile

# Create profile directory
mkdir -p profiles/community/<profile-name>

# Add profile.yaml and README.md
# ... (edit files)

# Commit
git add profiles/community/<profile-name>
git commit -m "Add profile: <profile-name>"

3. Submit Pull Request

Submit a PR to the main repository with:

• Title: Add profile: <profile-name>
• Description: Brief summary of what the profile extracts
• Linked issues: Reference any related issues or discussions

4. Review Process

Maintainers will review your profile for:

• ✅ Functional correctness (extraction accuracy)
• ✅ YAML validity and schema compliance
• ✅ Documentation completeness
• ✅ Edge case handling
• ✅ Performance characteristics

Review typically takes 1-3 business days.

License Requirements for Fixtures

If you include sample PDFs in fixtures/, ensure:

• Public domain or permissive license: PDFs must be usable under Apache-2.0 or MIT
• No personal information: Remove or redact PII (names, addresses, account numbers)
• No confidential data: Don't include real business data, trade secrets, or proprietary information
• Synthetic data preferred: Use generated/fabricated sample documents where possible

Redaction Examples

❌ Don't include unredacted documents:

Invoice #: INV-2024-001
Customer: John Doe
Account: 1234-5678-9012

✅ Use synthetic/redacted data:

Invoice #: INV-DEMO-001
Customer: [REDACTED]
Account: XXXX-XXXX-XXXX

Review Checklist

Before submitting, confirm:

• Validation passes: pdftract profiles validate returns success
• README.md is complete: Includes all required sections
• Field documentation: Every field has a description and example
• Positive cases work: Profile extracts correctly from intended document types
• Negative cases reject: Profile doesn't match unrelated documents
• Edge cases handled: Gracefully handles missing/empty/malformed fields
• Performance acceptable: Extraction completes in reasonable time
• Fixtures licensed: Sample PDFs use permissive licenses or are synthetic
• No secrets: Profile contains no API keys, passwords, or credentials
• Name follows conventions: Lowercase, hyphen-separated, non-trademarked

Profile Quality Standards

Profiles should meet these quality standards:

Accuracy

• Extract correct values in 95%+ of test cases
• Handle common layout variations (centered, left-aligned, multi-column)
• Gracefully degrade when fields are missing

Robustness

• Don't crash on malformed input
• Provide sensible defaults when extraction fails
• Log warnings for ambiguous cases

Clarity

• Profile name clearly indicates purpose
• Field names are self-documenting
• README explains match criteria and limitations

Performance

• No O(n²) or worse algorithms in match logic
• Reasonable memory usage (< 100MB per page)
• Fast enough for batch processing

Getting Help

• Documentation: Profile Authoring Guide
• Examples: Built-in profiles
• Issues: Open a GitHub issue for questions or problems
• Discussions: Use GitHub Discussions for design feedback

Profile Maintenance

Profile authors are encouraged to:

• Respond to feedback during review
• Fix bugs reported by users
• Update profiles for new pdftract releases
• Add test fixtures for edge cases

Maintainers may update profiles for compatibility or correctness.

---

Thank you for contributing to pdftract! 🎉