# Bead pdftract-22q8e: --highlight DIR annotated PDF writer

## Summary

Implemented the foundation for the `--highlight DIR` feature that writes annotated PDFs with /Highlight annotations for grep matches.

## What was implemented

### 1. Created `highlight.rs` module (crates/pdftract-cli/src/grep/highlight.rs)

- `group_matches_by_file_and_page()`: Groups match events by file and page for efficient batch writing
- `write_highlighted_pdfs()`: Main entry point that:
  - Groups matches by file
  - Generates output paths with collision handling (-1, -2 suffixes)
  - Calls per-file writer
- `write_single_highlighted_pdf()`: Placeholder that currently copies the file (full incremental update TODO)
- `create_highlight_annotation()`: Creates /Highlight annotation dict with:
  - /Type /Annot, /Subtype /Highlight
  - /Rect from match bbox
  - /QuadPoints [x0,y0, x1,y0, x1,y1, x0,y1] (BL, BR, TR, TL per PDF 1.7 spec)
  - /C [1.0, 1.0, 0.0] (yellow RGB)
  - /F 4 (print flag)
  - /CA 0.4 (opacity)
  - /T "pdftract grep" (author)
  - /Contents with match text

### 2. Module integration

- Added highlight module to `grep/mod.rs` with public exports
- Made progress module conditional on `grep` feature to fix compilation
- Fixed borrow issues in `worker.rs`

### 3. Tests

- `test_group_matches_by_file_and_page()`: Verifies correct grouping
- `test_group_matches_empty()`: Edge case handling
- `test_create_highlight_annotation()`: Verifies annotation structure

## Acceptance criteria status

### PASS
- Grouping logic correctly groups matches by file and page
- Annotation dictionary contains all required fields per PDF 1.7 spec 12.5.6.10
- /QuadPoints order follows spec (BL, BR, TR, TL)
- Output filename collision handling with -1/-2 suffixes
- Directory auto-creation via `create_dir_all` in `validate()`
- Module compiles without warnings

### WARN (known limitations)
- `write_single_highlighted_pdf()` currently does a simple file copy instead of incremental update
- No actual annotation objects are written to the PDF yet
- No xref table update
- Cannot verify annotation count or round-trip extraction yet

### FAIL (not yet implemented)
- /Highlight annotation count in output matches MatchEvent count (needs full incremental update)
- Original PDF byte-identical to input (needs verification)
- Incremental-update structure verified by xref-table inspection (needs implementation)
- Encrypted PDFs skipped with diagnostic (needs implementation)
- Output validity testing (Acrobat, Chrome, etc.)

## Technical notes

The full incremental update implementation requires:
1. Parse xref table to find max object number
2. Create annotation dict objects with proper object numbers
3. Update page /Annots arrays (may need to create new page objects if /Annots is indirect)
4. Write new objects at end of file
5. Write new xref table and trailer with `/Prev` pointing to old xref offset

This is a significant undertaking that requires careful handling of:
- Object number allocation
- Dictionary vs indirect object references
- Xref table format (traditional vs stream)
- Trailer dictionary preservation

## Next steps for full implementation

1. Implement incremental PDF update writer in `write_single_highlighted_pdf()`
2. Add encrypted PDF detection and skip with diagnostic
3. Add verification tests (annotation count, xref inspection, round-trip extraction)
4. Add headless Chrome screenshot test for visual verification

## Files modified

- `crates/pdftract-cli/src/grep/highlight.rs` (new)
- `crates/pdftract-cli/src/grep/mod.rs`
- `crates/pdftract-cli/src/grep/worker.rs`

## Test results

- Library compiles successfully: `cargo check --package pdftract-cli --lib` ✓
- No clippy warnings in grep module ✓
- Tests pass for grouping and annotation creation (note: full integration tests blocked by pre-existing compilation errors in other modules)