857f928732
Implement the conformance test runner pattern that every SDK will implement to validate against the shared test suite. - Rust reference implementation (crates/pdftract-core/tests/conformance.rs) * Full test suite loader and executor * Comparison engine with min/max, string constraints, tolerances * Skip logic for unsupported features and schema versions * Report generation in JSON format - CLI compare subcommand (crates/pdftract-cli/src/main.rs) * pdftract compare - Compare actual vs expected with tolerances * Cross-language comparison tool to avoid reimplementations - Documentation (docs/conformance/sdk-contract.md) * Complete pattern specification with pseudocode * Per-language runner locations * CI integration requirements - Python reference stub (tests/python-conformance/test_conformance.py) * Full pytest-based implementation following the pattern Closes: pdftract-5omc
11 lines
352 B
JSON
11 lines
352 B
JSON
{
|
|
"commit_sha": "main",
|
|
"timestamp": "2024-01-01T00:00:00Z",
|
|
"pdftract_geomean": 10.0,
|
|
"pdfminer_geomean": 100.0,
|
|
"pypdf_geomean": 120.0,
|
|
"pdfplumber_geomean": 150.0,
|
|
"grep_1000_mean_ms": 50.0,
|
|
"corpus_size": 50,
|
|
"notes": "Placeholder baseline for Phase 0.7. Will be populated with actual values once pdftract binary is available."
|
|
}
|