Commit graph

1 commit

Author SHA1 Message Date
jedarden
9a8252e92d feat(bf-20fot): extend download script to generate manifest entries
- download-grep-corpus.sh now generates manifest.csv entries during download
- Implements SHA256 checksum computation via sha256sum
- Extracts page counts using pdfinfo
- Captures file sizes in bytes via stat
- Records source URLs for each downloaded file
- Appends entries to manifest.csv with all 6 required fields
- Handles incremental downloads (skips existing files, appends to manifest)
- Added grep-corpus-generate-manifest.sh helper for bootstrapping existing corpus
- Generated manifest entries for all 1,260 existing PDFs
- Verified checksum accuracy against actual files

Acceptance criteria:
-  download-grep-corpus.sh writes manifest.csv entries for each downloaded PDF
-  All 6 fields populated: filename, source_url, page_count, file_size, checksum, license
-  Script handles incremental downloads (appends to existing manifest)

Closes bf-20fot

Verification: notes/bf-20fot.md
2026-07-05 12:26:55 -04:00