diff --git a/.needle-predispatch-sha b/.needle-predispatch-sha index ee46070..7f402ec 100644 --- a/.needle-predispatch-sha +++ b/.needle-predispatch-sha @@ -1 +1 @@ -30e02437ed27b17cba89afb74f72753939f44e37 +5fbd5a682372981ecaad8b473280dbbab685df12 diff --git a/notes/pdftract-2zw.md b/notes/pdftract-2zw.md index f4702a2..97378b0 100644 --- a/notes/pdftract-2zw.md +++ b/notes/pdftract-2zw.md @@ -23,7 +23,7 @@ Each fixture includes: ### 2. Integration Tests -Created `crates/pdftract-core/tests/page_classification.rs` with 4 tests: +Created `crates/pdftract-core/tests/page_classification.rs` with 5 tests: 1. **test_page_classification_fixtures**: Validates all fixtures classify correctly - Checks class matches expected @@ -43,6 +43,10 @@ Created `crates/pdftract-core/tests/page_classification.rs` with 4 tests: - Checks confidence_min in [0.0, 1.0] - Validates class names +5. **test_reproducibility_gate_with_perturbation**: Verifies reproducibility gate fails on perturbation + - Intentionally perturbs a confidence value + - Asserts the reproducibility check fails with clear diff + ### 3. CI Integration The tests are automatically run in CI via the Argo Workflows pipeline: @@ -56,20 +60,22 @@ The tests are automatically run in CI via the Argo Workflows pipeline: | Criterion | Status | Notes | |-----------|--------|-------| | 4 fixtures present | ✅ PASS | vector_pure, scanned_single, brokenvector_pdfa, hybrid_header_body | -| cargo test passes | ✅ PASS | 4/4 tests passing | -| Reproducibility gate | ✅ PASS | test_page_classification_reproducibility verifies byte-identical JSON | +| cargo test passes | ✅ PASS | 5/5 tests passing | +| Reproducibility gate | ✅ PASS | test_page_classification_reproducibility + test_reproducibility_gate_with_perturbation | | Fixtures < 1 MB | ✅ PASS | Total: 3.6 KB | +| Gate fails on perturbation | ✅ PASS | test_reproducibility_gate_with_perturbation verifies this | ## Test Output ``` -running 4 tests +running 5 tests test test_expected_json_validity ... ok test test_fixture_files_exist_and_size ... ok test test_page_classification_fixtures ... ok test test_page_classification_reproducibility ... ok +test test_reproducibility_gate_with_perturbation ... ok -test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out +test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ``` ## References