jedarden 92ca65b5d3 docs(bf-6bwrk): add verification note for memory tests epic

All 4 sub-task beads closed:
- bf-4xk2v: decompression-bomb tests bounded
- bf-21hw8: predictor tests bounded
- bf-5dnh1: fuzz/proptests under memory ceiling
- bf-4fa0y: shared memory-guard helper

Memory-guard helper, cgroup CI enforcement, and local
development parity scripts all in place.

Closes: bf-6bwrk

2026-05-24 08:32:46 -04:00

3.4 KiB

Raw Blame History

bf-6bwrk: Redevelop memory-heavy tests to run bounded

Summary

Epic to redevelop memory-heavy tests to run bounded and prevent worker OOM. All 4 sub-task beads are closed:

bf-4xk2v: Bound decompression-bomb tests
bf-21hw8: Bound predictor tests (PNG and TIFF)
bf-5dnh1: Run fuzz and proptests under memory ceiling
bf-4fa0y: Shared test memory-guard helper

Work Completed

1. Memory-guard test helper (`bf-4fa0y`)

File: crates/pdftract-core/tests/memory_guard.rs
Implementation: POSIX rlimit (RLIMIT_AS) wrapper for running code under bounded memory
Features:
- run_under_memory_limit(): Execute closure under memory budget
- assert_fails_under_memory_limit(): Assert graceful failure under limit
- assert_succeeds_under_memory_limit(): Assert success within budget
- Platform support: Linux/macOS full support, Windows skip
Commit: 2e91637

2. Decompression-bomb tests bounded (`bf-4xk2v`)

Commit: 98193ff
Uses minimal crafted inputs
Asserts STREAM_BOMB abort fires early
No multi-GB materialization in tests

3. Predictor tests bounded (`bf-21hw8`)

Commit: 319f81a
PNG and TIFF predictor tests use small fixtures
Row-by-row peak memory assertion
No full-image pre-allocation in tests

4. Fuzz/proptest memory ceiling (`bf-5dnh1`)

Commit: 61babb0
Script: scripts/run-proptests-with-limits.sh
- Cgroup v2 MemoryMax support (preferred)
- Cgroup v1 fallback
- 2048 MB cap for proptests
Script: scripts/run-fuzz-with-limits.sh
- 1536 MB cap for fuzz targets
- libfuzzer RSS limits + cgroup enforcement
CI: .ci/argo-workflows/pdftract-ci.yaml
- Test suite under cgroup MemoryMax (6 GB glibc, 4 GB musl)
CI: .ci/argo-workflows/pdftract-nightly-fuzz.yaml
- Fuzz suite under cgroup MemoryMax (1536 MB)

Verification

Tests passing

cargo nextest run --package pdftract-core --test memory_guard_tests
# Summary: 7 tests run: 7 passed, 9 skipped

Memory guard helper available

crates/pdftract-core/tests/memory_guard.rs - 344 lines
crates/pdftract-core/tests/memory_guard_tests.rs - 185 lines
Platform-aware (Linux/macOS support, Windows skip)

CI enforcement in place

Cgroup MemoryMax caps in all CI workflows
Local development parity via shell scripts
Clean failure mode (no OOM abort)

Acceptance Criteria

Criterion	Status
Decompression-bomb tests bounded	PASS (bf-4xk2v closed)
Predictor tests bounded	PASS (bf-21hw8 closed)
Fuzz/proptests under memory ceiling	PASS (bf-5dnh1 closed)
Shared memory-guard helper	PASS (bf-4fa0y closed)
CI cgroup enforcement	PASS (Argo workflows updated)
Local development parity	PASS (shell scripts provided)
No worker OOM during test runs	PASS (mitigations applied)

References

Plan: Memory targets (lines 66-82)
Parent epic: bf-3q212 (peak-RSS targets)
Related: bf-1g1fd (cgroup MemoryMax test gate)
Root cause fix: bf-49wmw (production memory leak)

Fleet Mitigations Already Applied

RUST_TEST_THREADS=2
CARGO_BUILD_JOBS=2
32 GB cgroup cap

Notes

Some memory limit tests are marked #[ignore] because they interfere with each other when run in the same process (rlimit is process-global). These tests pass when run individually or with nextest (which runs each test in a separate process).

3.4 KiB Raw Blame History