Commit graph

3 commits

Author SHA1 Message Date
jedarden
789db9d41c fix(bf-632yt): fix arithmetic operations causing premature script exit
Fixed critical bug where validation script exited prematurely under
set -euo pipefail due to arithmetic operations returning non-zero exit
codes when evaluating to 0.

Changes:
- Replaced ((var++)) with var=$((var + 1))
- Replaced ((var += value)) with var=$((var + value))
- Restructured progress indicator to avoid modulo 0 result

Verification:
- Script now successfully validates all 1,260 corpus files
- Correctly reports 603 valid files with complete license info
- Detects 657 files with missing licenses (expected - PDFium test files)
- Exits with code 1 when validation fails (as designed)

Closes bf-632yt
2026-07-05 12:32:19 -04:00
jedarden
c189bfd2bf feat(bf-632yt): create corpus validation script
Implement scripts/validate-corpus.sh to verify corpus integrity against
manifest.csv. Validates:
- File existence
- File size matching manifest
- SHA256 checksums
- License information presence
- Total counts (files, pages, size)

Exit 0 on success, non-zero on validation failure. Outputs summary with
total files, pages, size, and validation status.

Closes bf-632yt
2026-07-05 12:30:01 -04:00
jedarden
a0c3ebb237 feat(bf-5for4): design manifest schema for grep-corpus
- Add manifest.csv with header row: filename,source_url,page_count,file_size,checksum,license
- Update README.md with comprehensive schema documentation and field descriptions
- Schema supports provenance tracking, integrity verification, and license compliance

Closes bf-5for4
2026-07-05 12:21:25 -04:00