b970bc2d66
- Create benches/results/.gitkeep and README.md - Create benches/competitors/run_all.py Python orchestration script - Script runs competitor benchmarks (pdftract, pdfminer.six, pypdf, pdfplumber) - Emits benches/results/<commit-sha>.json with throughput metrics - Add tier4-competitor-runner to pdftract-ci Argo WorkflowTemplate - Runs only on main branch to track performance over time - Acceptance criteria: ratio_pdfminer ≥ 10.0, ratio_pypdf ≥ 5.0 Closes bf-2f5ew Files created: - benches/results/.gitkeep - benches/results/README.md - benches/competitors/run_all.py - notes/bf-2f5ew.md Files modified: - .ci/argo-workflows/pdftract-ci.yaml |
||
|---|---|---|
| .. | ||
| .gitkeep | ||
| README.md | ||
Competitive Benchmark Results
This directory stores per-commit benchmark results tracking pdftract performance against competitor PDF extraction libraries.
Output Format
Each commit produces a JSON file named <commit-sha>.json with the following schema:
{
"commit": "abc123...",
"date": "2026-07-05T12:00:00Z",
"pdftract_mbs": 125.5,
"pdfminer_mbs": 12.3,
"pypdf_mbs": 25.7,
"pdfplumber_mbs": 11.8,
"ratio_pdfminer": 10.2,
"ratio_pypdf": 4.9
}
Metrics
*_mbs: Median throughput in megabytes per second (MB/s) for each toolratio_pdfminer:pdftract_mbs / pdfminer_mbs(target: ≥ 10.0)ratio_pypdf:pdftract_mbs / pypdf_mbs(target: ≥ 5.0)
Acceptance Criteria
Per the plan (Tier 4 competitive benchmarks), pdftract must achieve:
- ≥ 10× faster than pdfminer.six on vector PDFs (
ratio_pdfminer ≥ 10.0) - ≥ 5× faster than pypdf on vector PDFs (
ratio_pypdf ≥ 5.0)
Usage
# Run all competitive benchmarks
python3 benches/competitors/run_all.py
# Results are written to benches/results/<commit-sha>.json
cat benches/results/$(git rev-parse HEAD).json | jq '.'
CI Integration
The benchmark runs as a Tier 4 step in pdftract-ci Argo WorkflowTemplate, executing only on the main branch to track performance over time.