jedarden a0c3ebb237 feat(bf-5for4): design manifest schema for grep-corpus

- Add manifest.csv with header row: filename,source_url,page_count,file_size,checksum,license
- Update README.md with comprehensive schema documentation and field descriptions
- Schema supports provenance tracking, integrity verification, and license compliance

Closes bf-5for4

2026-07-05 12:21:25 -04:00

3.7 KiB

Raw Blame History

bf-56090: Triage and commit dirty working tree

Summary

The task description mentioned "~206 uncommitted changes" with tracked modifications across crates/pdftract-core/src and crates/pdftract-cli/src, but investigation shows this description was outdated. The actual current state has only 1 tracked file deletion (a worktree marker) and 1282 untracked files, most of which are legitimate test fixtures.

Findings

Tracked Modifications

Only 1 tracked file change:

.claude/worktrees/agent-a5408b71de489f148 (deleted) - git worktree marker

No tracked modifications exist in: crates/pdftract-core/src, crates/pdftract-cli/src, Cargo.lock, or build/agl.json

Work Already Completed

The task description mentioned coordinating with beads bf-512z1 (encoding fixtures) and bf-5o8cg (SDK publish templates), but this work was already completed in recent commits from today (2026-07-05):

1d7dcdaa - feat(bf-20v7h): add .NET and C library Argo WorkflowTemplates
cb7b4f28 - feat(bf-4ygdw): add Go and Java publishing workflows
34c6a371 - feat(bf-2ypn2): add Phase 7 profiles exit gate fixtures
882196a2 - feat(bf-3cwge): add unmapped glyph generator script
4f1c03b9 - fix(bf-84xr8): fix unmapped glyph PDF encoding format

All Argo workflow templates mentioned in the task (pdftract-sdk-*-publish, pdftract-java-publish, pdftract-libpdftract-build) are already committed.

Untracked Files Analysis

Legitimate test fixtures committed:

tests/fixtures/grep-corpus/corpus/ (1260 PDFs) - Phase 7.8 benchmark corpus for folder grep throughput testing
tests/fixtures/scanned/ - OCR fixtures with WER ground truth for AS-02 test scenario
tests/encryption_errors.rs - Encryption error integration tests (plan lines 258, 732, 765)
crates/pdftract-cli/tests/test_encryption_errors.rs - CLI encryption error tests
crates/pdftract-cli/tests/test_encryption_unsupported.rs - Unsupported encryption tests
examples/test_url_host.rs - Example code
tests/fixtures/gen_unmapped_configurable.rs - Unmapped glyph fixture generator
gen_unmapped_configurable_bin/ - Binary crate for fixture generation
scripts/measure-wer.sh - WER measurement script for OCR testing
scripts/validate-corpus.sh - Corpus validation script

Temporary files removed:

scratch/create_livecycle_pdf.rs - debug/test file
scratch/check_fixtures.py - debug script
scratch/debug_fixtures.py - debug script
verify_fixtures, verify_fixtures.rs, verify_js_fixture.rs - temporary verification scripts
tests/fixtures/unmapped_config.txt - temporary config file

Pre-existing Issue

Compilation error in codebase (unrelated to this work):

error[E0308]: mismatched types
  --> /home/coding/pdftract/target/debug/build/pdftract-core-dd8ea6b2d053e647/out/font_fingerprints.rs:27:10

This error exists on the clean HEAD (verified with git stash), indicating it's a pre-existing issue in the codebase, not introduced by the untracked files. The error is in a generated file (font_fingerprints.rs) created during the build process.

Actions Taken

✅ Removed old worktree marker: .claude/worktrees/agent-a5408b71de489f148
✅ Added legitimate test fixtures to git (grep-corpus, scanned fixtures, encryption tests)
✅ Removed temporary/debug files (scratch/, verify_*, unmapped_config.txt)
✅ Coordinated with beads bf-512z1 and bf-5o8cg (work already completed)
⚠️ Cannot run cargo check due to pre-existing compilation error in font_fingerprints.rs

References

Plan line 263: Phase 2 exit gate (Unicode recovery ≥90%)
Plan lines 258, 732, 765: ENCRYPTION_UNSUPPORTED diagnostic
Plan line 3400-3413: Release Engineering, ADR-009
beads bf-512z1 (encoding fixtures), bf-5o8cg (SDK publish templates)

3.7 KiB Raw Blame History