585d861efc
Add property-based testing infrastructure for the lexer module with 6+ property tests covering INV-8 (no panic), string/hex roundtrips, name length bounds, and position monotonicity. Create 8 curated fixture files with golden token outputs for critical edge cases including EC-01 empty file test and whitespace-only inputs. Changes: - Add prop_string_roundtrip to tests/proptest/lexer.rs - Create tests/lexer/fixtures/ with 8 fixtures + .tokens.txt golden files - Add gen_lexer_golden.rs binary for regenerating golden outputs - Fix missing ObjRef import in marked_content_operators.rs Acceptance criteria: - cargo test --features proptest -p pdftract-core: 105 lexer tests pass - tests/lexer/fixtures/ contains 8 fixtures with .tokens.txt outputs - EC-01 empty file test: 0-byte input -> Token::Eof, no panic - Whitespace-only file test passes - INV-8 verified by prop_lexer_never_panics Closes: pdftract-sy8x
12 lines
131 B
Text
12 lines
131 B
Text
<4>
|
|
<41>
|
|
<48 65 6C 6C 6F>
|
|
<48656C6C6F>
|
|
<oddlength>
|
|
<with whitespace>
|
|
<mixedCase>
|
|
<MixedCase>
|
|
<
|
|
> (unterminated)
|
|
<4 5 6>
|
|
<4A6F6B65>
|