jedarden 789db9d41c fix(bf-632yt): fix arithmetic operations causing premature script exit

Fixed critical bug where validation script exited prematurely under
set -euo pipefail due to arithmetic operations returning non-zero exit
codes when evaluating to 0.

Changes:
- Replaced ((var++)) with var=$((var + 1))
- Replaced ((var += value)) with var=$((var + value))
- Restructured progress indicator to avoid modulo 0 result

Verification:
- Script now successfully validates all 1,260 corpus files
- Correctly reports 603 valid files with complete license info
- Detects 657 files with missing licenses (expected - PDFium test files)
- Exits with code 1 when validation fails (as designed)

Closes bf-632yt

2026-07-05 12:32:19 -04:00

2.5 KiB

Raw Blame History

bf-632yt: Create corpus validation script

Summary

Created scripts/validate-corpus.sh to verify corpus integrity against manifest.csv.

Implementation

The script validates all 5 required manifest fields:

File existence - Verifies all files listed in manifest.csv exist
File size - Checks file sizes match manifest
SHA256 checksums - Validates checksums using sha256sum
License information - Verifies license field is present and valid
Counts - Reports total files, pages, and size

Testing Results

Ran validation against tests/fixtures/grep-corpus:

Total files in manifest: 1,260
Valid files: 603 (with known licenses)
Missing files: 0
Size mismatches: 0
Checksum mismatches: 0
Missing licenses: 657 (PDFium test files with "unknown" license)

The 657 files with "unknown" licenses are from the PDFium test corpus, which is expected behavior. These files correctly fail the license validation check as designed.

Exit Codes

0 - Validation passed (all files valid)
1 - Validation failed (errors found)
2 - Usage error or manifest not found

Usage

./scripts/validate-corpus.sh [corpus_directory]

Default corpus directory: tests/fixtures/grep-corpus

Acceptance Criteria Status

✅ scripts/validate-corpus.sh exists and is executable (chmod +x)
✅ Script validates all 5 manifest fields (existence, size, checksum, count, license)
✅ Returns 0 on success, non-zero on validation failure
✅ Outputs summary: total files, total pages, total size, validation status

Bug Fix (2026-07-05)

Fixed critical issue where script exited prematurely under set -euo pipefail due to arithmetic operations returning non-zero exit codes when evaluating to 0.

Changes made:

Replaced all ((var++)) with var=$((var + 1))
Replaced all ((var += value)) with var=$((var + value))
Restructured progress indicator to avoid modulo 0 result

Root cause:

The set -e flag causes scripts to exit when any command returns non-zero. Arithmetic operations like ((x++)) return the result of the expression (0 when x becomes 0 after increment), which triggers set -e to exit.

Verification:

After fix, script successfully validates all 1,260 files:

Total files: 1,260
Valid files: 603 (with complete license info)
Missing licenses: 657 (expected - PDFium test files)
Exit code: 1 (correctly fails when validation errors found)

Commits

Initial: c189bfd2 - Created validation script
Fix: <pending> - Fixed arithmetic operation exits under set -e

2.5 KiB Raw Blame History