585d861efc
Add property-based testing infrastructure for the lexer module with 6+ property tests covering INV-8 (no panic), string/hex roundtrips, name length bounds, and position monotonicity. Create 8 curated fixture files with golden token outputs for critical edge cases including EC-01 empty file test and whitespace-only inputs. Changes: - Add prop_string_roundtrip to tests/proptest/lexer.rs - Create tests/lexer/fixtures/ with 8 fixtures + .tokens.txt golden files - Add gen_lexer_golden.rs binary for regenerating golden outputs - Fix missing ObjRef import in marked_content_operators.rs Acceptance criteria: - cargo test --features proptest -p pdftract-core: 105 lexer tests pass - tests/lexer/fixtures/ contains 8 fixtures with .tokens.txt outputs - EC-01 empty file test: 0-byte input -> Token::Eof, no panic - Whitespace-only file test passes - INV-8 verified by prop_lexer_never_panics Closes: pdftract-sy8x
11 lines
299 B
Text
11 lines
299 B
Text
(n\\n) (newline escape)
|
|
(n\\r) (carriage return escape)
|
|
(n\\t) (tab escape)
|
|
(n\\\\) (backslash escape)
|
|
(n\\() (open paren escape)
|
|
(n\\)) (close paren escape)
|
|
(n\\101) (octal A)
|
|
(n\\101\\102\\103) (multiple octal)
|
|
(line1\
|
|
line2) (line continuation)
|
|
(bare\\rcarriage) (line ending normalization test)
|