jedarden/pdftract
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pdftract/tests/fixtures/classifier/MANIFEST.tsv
jedarden 633eba61b1 test(classifier): add 200-document labeled corpus for Phase 5.6 
- Create tests/fixtures/classifier/ with 200 synthetic PDFs:
  - 50 invoices with bill-to/ship-to, item tables, totals
  - 50 scientific papers with abstracts, sections, references
  - 50 contracts with clauses, legal terminology, signatures
  - 50 misc documents (8 receipts, 8 forms, 7 bank statements,
    7 slide decks, 7 legal filings, 6 book excerpts, 7 magazines)

- Add MANIFEST.tsv mapping each document to its expected type
  with source URL and license (all MIT-0 synthetic data)

- Add scripts/generate_test_corpus.py to regenerate the corpus
  using reportlab for PDF generation

- Add tests/test_classifier_corpus.rs with validation harness:
  - test_corpus_manifest_validity: verifies manifest structure
    and file existence (PASSES)
  - test_classifier_corpus_accuracy: will validate precision/
    recall/F1 when classifier is implemented (SKIP for now)
  - test_classifier_reproducibility: will verify deterministic
    classification (SKIP for now)

- Add tests/fixtures/classifier/README.md documenting corpus
  structure, generation process, and acceptance criteria

Total corpus size: ~0.4 MB (each PDF < 5 KB)

Acceptance criteria (from plan.md Phase 5.6):
- Per-class precision and recall >= 0.85
- Macro-F1 >= 0.88
- Reproducibility: identical output for same document

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 07:16:02 -04:00

201 lines
19 KiB
Text
Raw Blame History

path expected_document_type source_url license
invoice/01.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/02.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/03.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/04.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/05.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/06.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/07.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/08.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/09.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/10.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/11.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/12.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/13.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/14.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/15.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/16.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/17.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/18.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/19.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/20.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/21.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/22.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/23.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/24.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/25.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/26.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/27.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/28.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/29.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/30.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/31.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/32.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/33.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/34.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/35.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/36.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/37.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/38.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/39.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/40.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/41.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/42.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/43.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/44.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/45.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/46.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/47.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/48.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/49.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
invoice/50.pdf invoice Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/01.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/02.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/03.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/04.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/05.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/06.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/07.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/08.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/09.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/10.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/11.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/12.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/13.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/14.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/15.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/16.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/17.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/18.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/19.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/20.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/21.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/22.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/23.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/24.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/25.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/26.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/27.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/28.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/29.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/30.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/31.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/32.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/33.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/34.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/35.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/36.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/37.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/38.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/39.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/40.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/41.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/42.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/43.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/44.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/45.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/46.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/47.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/48.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/49.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
scientific_paper/50.pdf scientific_paper Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/01.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/02.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/03.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/04.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/05.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/06.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/07.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/08.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/09.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/10.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/11.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/12.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/13.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/14.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/15.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/16.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/17.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/18.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/19.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/20.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/21.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/22.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/23.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/24.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/25.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/26.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/27.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/28.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/29.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/30.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/31.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/32.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/33.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/34.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/35.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/36.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/37.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/38.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/39.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/40.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/41.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/42.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/43.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/44.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/45.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/46.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/47.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/48.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/49.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
contract/50.pdf contract Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/01.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/02.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/03.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/04.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/05.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/06.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/07.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/08.pdf receipt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/09.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/10.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/11.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/12.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/13.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/14.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/15.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/16.pdf form Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/17.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/18.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/19.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/20.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/21.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/22.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/23.pdf bank_statement Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/24.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/25.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/26.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/27.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/28.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/29.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/30.pdf slide_deck Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/31.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/32.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/33.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/34.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/35.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/36.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/37.pdf legal_filing Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/38.pdf book_excerpt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/39.pdf book_excerpt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/40.pdf book_excerpt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/41.pdf book_excerpt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/42.pdf book_excerpt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/43.pdf book_excerpt Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/44.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/45.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/46.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/47.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/48.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/49.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
misc/50.pdf magazine Synthetic test data generated by scripts/generate_test_corpus.py MIT-0
Reference in a new issue View git blame Copy permalink