pdftract/tests/fixtures/grep-corpus

pdftract grep-corpus

Benchmark corpus for pdftract-grep-1000 CI benchmark.

Purpose

This corpus contains 1000 PDFs (~100 MB total) used to benchmark and validate the grep feature's performance and correctness.

Structure

tests/fixtures/grep-corpus/
├── corpus/              # Actual PDF files
├── manifest.csv         # File metadata and expected match counts
├── regenerate.sh        # Script to rebuild the corpus
└── README.md            # This file

Usage

Running the benchmark

cargo bench --bench grep_1000

Regenerating the corpus

The corpus can be regenerated using the Makefile targets:

# Generate/regenerate the corpus (default: 1000 PDFs)
make download-grep-corpus

# Generate a specific count
make download-grep-corpus COUNT=500

# Validate corpus integrity
make validate-corpus

The download-grep-corpus target generates synthetic PDFs using ReportLab, which ensures:

• Determinism: Same script produces identical corpus
• Known license: All synthetic PDFs are public domain
• Controlled variety: Mix of page counts (1-20 pages per PDF)
• Fast regeneration: No network dependencies after initial setup

The validate-corpus target verifies corpus integrity by checking:

• All files listed in manifest.csv exist
• File sizes match the manifest
• SHA256 checksums are correct
• License information is present
• Total counts (files, pages, size) are accurate

Manual regeneration (advanced)

For advanced usage, you can also run the scripts directly:

# Generate corpus
bash scripts/download-grep-corpus.sh 1000

# Regenerate manifest from existing corpus
bash scripts/grep-corpus-generate-manifest.sh

# Validate corpus
bash scripts/validate-corpus.sh tests/fixtures/grep-corpus

Corpus Requirements

The corpus must satisfy:

• Size: 1000 PDF files, ~100 MB total
• Content: Mix of vector and scanned PDFs
• License: Public domain or permissive (CC BY-SA, MIT, etc.)
• Determinism: Regenerable from source (no manual uploads)

CI Gates

The benchmark enforces these gates on every PR:

1. Throughput: ≥ 50 MB/s on 4-core CI machine
2. vs pdfgrep: ≥ 2× faster
3. vs pdftotext+ripgrep: ≥ 3× faster
4. Regression: ≤ 10% vs historical main

Status

TODO: Populate corpus (blocks on 7.8.1-7.8.9 grep implementation).

Sources (TODO)

Potential corpus sources:

• arXiv API (public domain metadata)
• Wikipedia article exports (CC BY-SA)
• Synthetic PDFs via pdfjoin

Manifest Format

filename,source_url,page_count,file_size,checksum,license
doc001.pdf,https://example.com/doc001.pdf,10,102400,abc123...,public-domain
doc002.pdf,https://arxiv.org/pdf/1234.5678.pdf,15,98304,def456...,cc-by-4.0
...

Fields

• filename: Relative path from corpus/ directory (e.g., doc001.pdf)
• source_url: URL where the PDF was downloaded from
• page_count: Number of pages in the PDF
• file_size: File size in bytes
• checksum: SHA256 hash of the file contents (for integrity verification)
• license: License identifier:
  • public-domain: Public domain
  • cc-by-4.0: Creative Commons Attribution 4.0
  • cc-by-sa-4.0: Creative Commons Attribution-ShareAlike 4.0
  • mit: MIT License
  • other: Other permissive license (document in source_url comments)