pdftract/tests/lexer/fixtures/bom_utf16_string.pdf.in at 99317e90106abcde40ccabe4886980101099bdb9 - jedarden/pdftract - Forgejo

jedarden/pdftract

jedarden 585d861efc test(pdftract-sy8x): implement lexer proptest harness and curated corpus

Add property-based testing infrastructure for the lexer module with 6+
property tests covering INV-8 (no panic), string/hex roundtrips, name
length bounds, and position monotonicity. Create 8 curated fixture files
with golden token outputs for critical edge cases including EC-01 empty
file test and whitespace-only inputs.

Changes:
- Add prop_string_roundtrip to tests/proptest/lexer.rs
- Create tests/lexer/fixtures/ with 8 fixtures + .tokens.txt golden files
- Add gen_lexer_golden.rs binary for regenerating golden outputs
- Fix missing ObjRef import in marked_content_operators.rs

Acceptance criteria:
- cargo test --features proptest -p pdftract-core: 105 lexer tests pass
- tests/lexer/fixtures/ contains 8 fixtures with .tokens.txt outputs
- EC-01 empty file test: 0-byte input -> Token::Eof, no panic
- Whitespace-only file test passes
- INV-8 verified by prop_lexer_never_panics

Closes: pdftract-sy8x

2026-05-24 02:36:37 -04:00

1 line

No EOL

32 B

Text

Raw Blame History

(text with ţ˙ UTF-16 BOM prefix)