pdftract/crates/pdftract-core/src/output/ndjson
jedarden 85a502c346 fix(pdftract-31bum): implement smarter backpressure for OutOfOrderBuffer
The OutOfOrderBuffer had a deadlock issue where:
1. Buffer fills with 8 pages from workers
2. Next expected page (e.g., page 0) is missing
3. All workers block trying to push more pages
4. Deadlock because no one can push page 0

Fix: Implement smarter backpressure that:
- Blocks when buffer is full AND next expected page is missing
- Allows push if we're pushing the missing next expected page
- Allows push if next expected page is already in buffer

Also add pop_next_in_order_blocking() for multi-threaded scenarios.

Acceptance criteria:
- Unit test: push pages 3,1,4,1,5,9,2,6 -> pop in 0..=9 order PASS
- Backpressure test: 9th push blocks until page 0 arrives PASS
- Concurrency stress test: 8 workers + 1 consumer, 1000 pages PASS
- finish() test: producer finished, heap drained -> pop returns None PASS

Closes: pdftract-31bum

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 17:15:06 -04:00
..
buffer.rs fix(pdftract-31bum): implement smarter backpressure for OutOfOrderBuffer 2026-05-26 17:15:06 -04:00
frames.rs feat(pdftract-2kpm0): implement NdjsonFrame enum with internal-tag discriminator and write_frame helper 2026-05-25 11:24:08 -04:00
mod.rs feat(pdftract-2kpm0): implement NdjsonFrame enum with internal-tag discriminator and write_frame helper 2026-05-25 11:24:08 -04:00
pipeline.rs feat(pdftract-5izq5): implement NDJSON streaming pipeline infrastructure 2026-05-25 02:15:39 -04:00