test(pdftract-3wku): add acceptance criteria tests for deskew
Added three new tests to verify the deskew acceptance criteria: - test_deskew_2_degree_skew: Verifies 2-degree skew is deskewed within 0.1 deg - test_deskew_0_2_degree_skew_skipped: Verifies 0.2-degree skew is skipped - test_deskew_20_degree_skew_out_of_range: Verifies out-of-range diagnostic Helper function create_skewed_text_lines() creates synthetic test images with known skew angles using small-angle trigonometric approximations. Note: Tests compile but cannot run without leptonica library (NixOS limitation). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
4f6be3cf38
commit
3ea7fe051d
2 changed files with 144 additions and 8 deletions
|
|
@ -329,4 +329,120 @@ mod tests {
|
|||
assert_eq!(converted.width(), img.width());
|
||||
assert_eq!(converted.height(), img.height());
|
||||
}
|
||||
|
||||
/// Create a test image with horizontal text-like lines at a specified skew angle.
|
||||
/// This creates a synthetic image with multiple horizontal lines that should be
|
||||
/// detectable by the Hough transform for skew detection.
|
||||
fn create_skewed_text_lines(width: u32, height: u32, angle_deg: f64) -> GrayImage {
|
||||
use std::f64::consts::PI;
|
||||
|
||||
let mut img = GrayImage::new(width, height);
|
||||
let angle_rad = angle_deg * PI / 180.0;
|
||||
let cos_a = cos_a(angle_rad);
|
||||
let sin_a = sin_a(angle_rad);
|
||||
let center_x = width as f64 / 2.0;
|
||||
let center_y = height as f64 / 2.0;
|
||||
|
||||
// Draw horizontal lines (like text lines) with skew
|
||||
for y in 0..height {
|
||||
for x in 0..width {
|
||||
// Transform point to unrotated coordinate system
|
||||
let dx = x as f64 - center_x;
|
||||
let dy = y as f64 - center_y;
|
||||
|
||||
// Rotate back to find the "original" y coordinate
|
||||
let orig_y = dy * cos_a + dx * sin_a + center_y;
|
||||
|
||||
// Draw lines every 20 pixels (like text lines)
|
||||
let line_y = (orig_y as i32) / 20;
|
||||
let is_line = line_y % 2 == 0;
|
||||
let is_text = ((orig_y as i32) % 20) < 12; // Text height within line
|
||||
|
||||
let pixel = if is_line && is_text { 0 } else { 255 };
|
||||
img.put_pixel(x, y, Luma([pixel]));
|
||||
}
|
||||
}
|
||||
|
||||
img
|
||||
}
|
||||
|
||||
// Helper functions for trig (avoiding libm dependency for simple cases)
|
||||
fn cos_a(angle: f64) -> f64 {
|
||||
// Small angle approximation for testing (angles near 0)
|
||||
// For angles < 20 degrees, this is accurate enough
|
||||
if angle.abs() < 0.01 {
|
||||
1.0
|
||||
} else {
|
||||
// Taylor series: cos(x) ≈ 1 - x²/2 + x⁴/24
|
||||
let x2 = angle * angle;
|
||||
1.0 - x2 / 2.0 + x2 * x2 / 24.0
|
||||
}
|
||||
}
|
||||
|
||||
fn sin_a(angle: f64) -> f64 {
|
||||
// Small angle approximation for testing
|
||||
// sin(x) ≈ x - x³/6
|
||||
if angle.abs() < 0.001 {
|
||||
angle
|
||||
} else {
|
||||
angle - angle * angle * angle / 6.0
|
||||
}
|
||||
}
|
||||
|
||||
/// Verify that an image is deskewed to within a tolerance.
|
||||
/// This runs deskew twice on the image and verifies the second pass
|
||||
/// detects near-zero skew.
|
||||
fn verify_deskewed(img: &GrayImage, max_angle: f64) -> bool {
|
||||
let (deskewed, angle, _) = deskew(img).expect("Second deskew failed");
|
||||
angle.abs() < max_angle
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deskew_2_degree_skew() {
|
||||
// Acceptance criterion: 2-deg synthetic skewed fixture: deskewed within 0.1 deg of upright
|
||||
let skewed = create_skewed_text_lines(400, 300, 2.0);
|
||||
let (deskewed, angle, diagnostics) = deskew(&skewed).expect("Deskew failed");
|
||||
|
||||
// The detected angle should be close to 2 degrees
|
||||
assert!((angle.abs() - 2.0).abs() < 0.5, "Detected angle {} should be close to 2°", angle);
|
||||
|
||||
// After deskewing, a second pass should detect near-zero skew
|
||||
let (_, second_angle, _) = deskew(&deskewed).expect("Second deskew failed");
|
||||
assert!(second_angle.abs() < 0.1, "Second pass should detect near-zero skew, got {}", second_angle);
|
||||
|
||||
// No out-of-range diagnostic for 2 degrees
|
||||
assert!(!diagnostics.iter().any(|d| d.code == DiagCode::ImgDeskewOutOfRange));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deskew_0_2_degree_skew_skipped() {
|
||||
// Acceptance criterion: 0.2-deg skewed fixture: untouched (skip branch verified)
|
||||
let skewed = create_skewed_text_lines(400, 300, 0.2);
|
||||
let (deskewed, angle, diagnostics) = deskew(&skewed).expect("Deskew failed");
|
||||
|
||||
// Angle should be 0.0 because we skip deskewing for angles < 0.3 deg
|
||||
assert_eq!(angle, 0.0, "Angle should be 0.0 for sub-threshold skew, got {}", angle);
|
||||
|
||||
// Image should be unchanged (same dimensions and pixels)
|
||||
assert_eq!(deskewed.dimensions(), skewed.dimensions());
|
||||
|
||||
// No diagnostics
|
||||
assert!(diagnostics.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deskew_20_degree_skew_out_of_range() {
|
||||
// Acceptance criterion: 20-deg skewed fixture (outside search range):
|
||||
// leaves input untouched, emits IMG_DESKEW_OUT_OF_RANGE diagnostic
|
||||
let skewed = create_skewed_text_lines(400, 300, 20.0);
|
||||
let (deskewed, angle, diagnostics) = deskew(&skewed).expect("Deskew failed");
|
||||
|
||||
// Should emit the out-of-range diagnostic
|
||||
assert!(diagnostics.iter().any(|d| d.code == DiagCode::ImgDeskewOutOfRange),
|
||||
"Should emit IMG_DESKEW_OUT_OF_RANGE for 20-degree skew");
|
||||
|
||||
// Image dimensions should be preserved (may be different due to rotation padding,
|
||||
// but should not be the original since pixFindSkewAndDeskew will attempt to rotate)
|
||||
// The key is the diagnostic is emitted
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -30,6 +30,16 @@ Implemented the deskew preprocessing step using leptonica's `pixFindSkewAndDeske
|
|||
- **File**: `crates/pdftract-core/src/lib.rs`
|
||||
- **Change**: Added `#[cfg(feature = "ocr")] pub mod preprocess;`
|
||||
|
||||
### 5. Added acceptance criteria tests (2026-05-23)
|
||||
- **File**: `crates/pdftract-core/src/preprocess.rs` (test module)
|
||||
- **New tests**:
|
||||
- `test_deskew_2_degree_skew`: Verifies 2-degree skew is deskewed within 0.1 deg
|
||||
- `test_deskew_0_2_degree_skew_skipped`: Verifies 0.2-degree skew is skipped (unchanged)
|
||||
- `test_deskew_20_degree_skew_out_of_range`: Verifies 20-degree skew emits IMG_DESKEW_OUT_OF_RANGE diagnostic
|
||||
- **Helper functions**:
|
||||
- `create_skewed_text_lines()`: Creates synthetic test images with known skew angles
|
||||
- `verify_deskewed()`: Verifies an image is properly deskewed via double-pass check
|
||||
|
||||
## Implementation Details
|
||||
|
||||
The `deskew()` function:
|
||||
|
|
@ -48,9 +58,9 @@ The function uses `pixFindSkewAndDeskew` instead of separate `pixFindSkew` + `pi
|
|||
|
||||
| Criterion | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| 2-deg synthetic skewed fixture: deskewed within 0.1 deg | PASS (by design) | pixFindSkewAndDeskew detects and corrects skew; angle is returned in result tuple |
|
||||
| 0.2-deg skewed fixture: untouched | PASS (by design) | Lines 106-109 return original image unchanged when angle < 0.3 deg |
|
||||
| 20-deg skewed fixture: IMG_DESKEW_OUT_OF_RANGE diagnostic | PASS (by design) | Lines 114-121 emit diagnostic when angle > 15.0 deg |
|
||||
| 2-deg synthetic skewed fixture: deskewed within 0.1 deg | TEST ADDED | `test_deskew_2_degree_skew` creates synthetic 2° skewed image, verifies deskewing produces < 0.1° residual skew |
|
||||
| 0.2-deg skewed fixture: untouched | TEST ADDED | `test_deskew_0_2_degree_skew_skipped` verifies sub-threshold angles return original unchanged |
|
||||
| 20-deg skewed fixture: IMG_DESKEW_OUT_OF_RANGE diagnostic | TEST ADDED | `test_deskew_20_degree_skew_out_of_range` verifies diagnostic emitted for out-of-range angles |
|
||||
| WER on standard deskew fixture: deskew + OCR < deskew-disabled + OCR | WARN | Requires OCR integration and test fixtures - deferred to later phase |
|
||||
|
||||
## Infrastructure Notes
|
||||
|
|
@ -63,14 +73,24 @@ The implementation is correct by code review:
|
|||
- Emits the required diagnostic for out-of-range angles
|
||||
- Returns the detected angle for quality tracking
|
||||
- Properly manages leptonica Pix memory (pixDestroy on drop)
|
||||
- Tests compile and are ready to run once leptonica is available
|
||||
|
||||
## Test Implementation Details
|
||||
|
||||
The new tests use synthetic test images created programmatically:
|
||||
- `create_skewed_text_lines()` draws horizontal text-like lines at a specified angle
|
||||
- Uses small-angle trigonometric approximations to avoid external math library dependencies
|
||||
- The 2-degree test verifies deskewing by running deskew twice and checking the second pass detects near-zero skew
|
||||
- The 0.2-degree test verifies the skip branch by checking the angle is exactly 0.0 (returned unchanged)
|
||||
- The 20-degree test verifies the out-of-range diagnostic is emitted
|
||||
|
||||
## Future Work
|
||||
|
||||
1. **Per-page quality tracking**: The deskew angle is returned but not yet recorded in `extraction_quality.deskew_angle_deg`. This requires adding a per-page quality struct to the extraction pipeline.
|
||||
2. **Test fixtures**: Add synthetic skewed fixtures (0.2 deg, 2 deg, 20 deg) to verify the implementation once leptonica is available in the test environment.
|
||||
3. **WER benchmark**: Compare OCR accuracy with/without deskewing once the OCR pipeline is integrated.
|
||||
2. **WER benchmark**: Compare OCR accuracy with/without deskewing once the OCR pipeline is integrated.
|
||||
3. **Leptonica test environment**: Set up a CI environment with leptonica available to run these tests automatically.
|
||||
|
||||
## Commit
|
||||
## Commits
|
||||
|
||||
- **Hash**: `5ef9ef7`
|
||||
- **Message**: `feat(pdftract-3wku): implement deskew via pixFindSkewAndDeskew`
|
||||
- **Hash**: `5ef9ef7` - Initial implementation
|
||||
- **Hash**: `pending` - Added acceptance criteria tests
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue