a5a80bf319
Create comprehensive documentation of the cleaned tests/fixtures/ structure following the three-bead cleanup effort (bf-1iefu, bf-xqib3, bf-2yhak). Changes: - Create: tests/fixtures/STRUCTURE.md with complete directory organization - Directory structure with annotations - Cleanup history and justification - List of 10 KEEP generators (co-located with fixtures) - List of 5 removed DELETE generators - List of 2 relocated RELOCATE tools - Maintenance guidelines - Update: tools/README.md with cleanup summary section - Three-bead coordination overview - Final state statistics - Links to related documentation - Create: notes/bf-620xp.md verification note Verification: - ✅ No compiled artifacts remain - ✅ No obsolete generators remain - ✅ 10 KEEP generators remain (actively maintained, co-located with fixtures) - ✅ All RELOCATE tools in tools/ with documentation - ✅ Binary .bin files are legitimate test data (LZW compression, malformed fixtures) Closes bf-620xp
171 lines
8 KiB
Markdown
171 lines
8 KiB
Markdown
# tests/fixtures/ Structure
|
|
|
|
This document describes the structure and organization of test fixtures in the pdftract project.
|
|
|
|
## Overview
|
|
|
|
The `tests/fixtures/` directory contains PDF test data files organized by test category. As of 2026-07-05, the directory contains both fixture data files and a small set of actively maintained generator scripts co-located with the fixtures they generate.
|
|
|
|
## Cleanup History (2026-07-05)
|
|
|
|
Three beads completed the cleanup and organization:
|
|
|
|
1. **bf-1iefu**: Categorized all generator scripts into KEEP/DELETE/RELOCATE
|
|
2. **bf-xqib3**: Removed obsolete generators and compiled artifacts (5 files deleted)
|
|
3. **bf-2yhak**: Relocated general-purpose tools to tools/ with documentation
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
tests/fixtures/
|
|
├── cjk/ # CJK encoding fixtures
|
|
├── classifier/ # Classifier training data
|
|
├── encoding/ # Unicode recovery test fixtures
|
|
│ ├── generate_unmapped_glyphs.py [KEEP: generates unmapped glyph tests]
|
|
│ └── create_unmapped_comprehensive.py [KEEP: generates comprehensive encoding tests]
|
|
├── encrypted/ # (deprecated - moved to security/)
|
|
├── encrypted-old/ # Legacy encrypted PDF tests
|
|
├── fonts/ # Font subset fixtures
|
|
├── forms/ # AcroForm and XFA fixtures
|
|
│ └── generate_form_fixtures.rs [KEEP: generates form field fixtures]
|
|
├── grep-corpus/ # CER testing corpus
|
|
├── json_schema/ # JSON schema validation fixtures
|
|
├── lzw_*.bin # LZW compression test data (binary fixtures)
|
|
├── malformed/ # Malformed PDF fixtures
|
|
│ ├── random_bytes.bin [fixture: random corruption data]
|
|
│ ├── compression-bomb.bin [fixture: decompression bomb test]
|
|
│ └── gen_bomb.py [KEEP: generates malformed fixtures]
|
|
├── no-mapping.md # Encoding mapping reference
|
|
├── ocr/ # OCR ground truth data
|
|
├── page_class/ # Page classification fixtures
|
|
├── perf/ # Performance test fixtures
|
|
├── preprocess/ # Preprocessing test data
|
|
├── profiles/ # Color profile fixtures
|
|
├── PROVENANCE.md # Fixture generation log
|
|
├── remote_100page.pdf # Remote testing fixture
|
|
├── sample.pdf # Minimal valid PDF
|
|
├── scanned/ # Scanned document fixtures
|
|
│ ├── generate_scanned_fixtures.py [KEEP: generates scan simulation fixtures]
|
|
│ ├── run_gen.sh [KEEP: nix-shell wrapper for generator]
|
|
│ ├── calculate_wer.py [KEEP: WER/CER calculation utility]
|
|
│ └── wer_gate_stub.rs [KEEP: WER gate test stub]
|
|
├── security/ # Security and encryption fixtures
|
|
│ └── generate_sensitive_fixture.rs [KEEP: generates security test fixtures]
|
|
├── STRUCTURE.md # This file
|
|
├── tagged-*.pdf # Tagged PDF fixtures
|
|
├── test-minimal.pdf # Minimal test fixture
|
|
├── valid-minimal.pdf # Valid minimal PDF
|
|
└── vector/ # Vector text extraction fixtures
|
|
└── generate_vector_cer_corpus.py [KEEP: generates CER test corpus]
|
|
```
|
|
|
|
## Generator Scripts Remaining in tests/fixtures/
|
|
|
|
As of 2026-07-05, **10 generator scripts** remain in tests/fixtures/. These were categorized as **KEEP** in bf-1iefu because they generate fixtures for specific test categories and are actively maintained:
|
|
|
|
| Script | Category | Purpose |
|
|
|--------|----------|---------|
|
|
| `encoding/generate_unmapped_glyphs.py` | KEEP | Unmapped glyph test generation |
|
|
| `encoding/create_unmapped_comprehensive.py` | KEEP | Comprehensive encoding tests |
|
|
| `forms/generate_form_fixtures.rs` | KEEP | AcroForm/XFA fixtures |
|
|
| `malformed/gen_bomb.py` | KEEP | Decompression bomb fixtures |
|
|
| `scanned/generate_scanned_fixtures.py` | KEEP | OCR scan simulation fixtures |
|
|
| `scanned/run_gen.sh` | KEEP | Nix-shell dependency wrapper |
|
|
| `scanned/calculate_wer.py` | KEEP | WER/CER calculation utility |
|
|
| `scanned/wer_gate_stub.rs` | KEEP | WER gate test stub |
|
|
| `security/generate_sensitive_fixture.rs` | KEEP | TH-08 log audit fixtures |
|
|
| `vector/generate_vector_cer_corpus.py` | KEEP | CER testing corpus |
|
|
|
|
These scripts are **actively maintained** (last modified May-July 2026) and are **not redundant** - each generates unique fixtures for specific test scenarios.
|
|
|
|
## Removed Scripts (bf-xqib3)
|
|
|
|
The following obsolete generators were removed on 2026-07-05:
|
|
|
|
- `scanned/generate_scanned_fixtures.rs` (Rust stub, use Python version)
|
|
- `security/generate_sensitive_fixture.py` (duplicate, use Rust version)
|
|
- `encoding/generate_unmapped_glyphs.rs` (superseded by Python version)
|
|
- `malformed/gen-bomb-10k-2g.sh` (superseded by Python version)
|
|
- `forms/generate_form_fixtures.d` (orphaned Makefile dependency)
|
|
|
|
## Relocated Scripts (bf-2yhak)
|
|
|
|
The following general-purpose tools were relocated to `tools/` on 2026-07-05:
|
|
|
|
- `scanned/convert_to_scanned.sh` → `tools/convert_pdf_to_scanned.sh` (general PDF conversion utility)
|
|
- `grep-corpus/regenerate.sh` → deleted (incomplete stub)
|
|
|
|
## Binary Test Data
|
|
|
|
The `.bin` files in the fixtures root are **legitimate test fixtures**, not compiled artifacts:
|
|
|
|
- `lzw_*.bin` - LZW compression stream test data (14 files)
|
|
- `malformed/random_bytes.bin` - Random corruption data
|
|
- `malformed/compression-bomb.bin` - Decompression bomb test fixture
|
|
|
|
## Fixture Categories
|
|
|
|
### Encoding Tests (`encoding/`)
|
|
Unicode recovery test fixtures with custom encodings and glyph mappings.
|
|
|
|
### Form Tests (`forms/`)
|
|
AcroForm and XFA fixtures for form field extraction.
|
|
|
|
### Malformed PDFs (`malformed/`)
|
|
Intentionally malformed PDFs for robustness testing.
|
|
|
|
### OCR Tests (`scanned/`)
|
|
Scan simulation fixtures with ground truth text for OCR validation.
|
|
|
|
### Security Tests (`security/`)
|
|
Encrypted PDFs and security policy validation fixtures.
|
|
|
|
### Vector Tests (`vector/`)
|
|
Clean vector PDFs with embedded text for CER/accuracy testing.
|
|
|
|
### Performance Tests (`perf/`)
|
|
Large-page-count PDFs for memory ceiling validation.
|
|
|
|
### CJK Tests (`cjk/`)
|
|
Chinese, Japanese, Korean text encoding fixtures.
|
|
|
|
## Maintenance Guidelines
|
|
|
|
### When Adding New Fixtures
|
|
|
|
1. Place fixtures in the appropriate category subdirectory
|
|
2. If writing a generator, add it to the same subdirectory as the fixtures it generates
|
|
3. Update `PROVENANCE.md` with generation details
|
|
4. Add documentation comments explaining the generator's purpose
|
|
|
|
### When Modifying Existing Fixtures
|
|
|
|
1. Regenerate fixtures using the co-located generator script
|
|
2. Update `PROVENANCE.md` with new generation date and SHA256
|
|
3. Commit with message referencing the bead ID
|
|
|
|
### Generator Script Guidelines
|
|
|
|
- Generators should be co-located with the fixtures they generate
|
|
- General-purpose tools belong in `tools/` (not `tests/fixtures/`)
|
|
- Add docstring/comments explaining: what it generates, how to run it, dependencies
|
|
- Prefer Python for encoding/OCR fixtures, Rust for security/crypto fixtures
|
|
|
|
## Related Documentation
|
|
|
|
- [`PROVENANCE.md`](PROVENANCE.md) - Detailed generation log for each fixture
|
|
- [`tools/README.md`](../tools/README.md) - General-purpose tools catalog
|
|
- [`notes/bf-1iefu.md`](../notes/bf-1iefu.md) - Generator categorization rationale
|
|
- [`notes/bf-xqib3.md`](../notes/bf-xqib3.md) - Obsolete generator removal
|
|
- [`notes/bf-2yhak.md`](../notes/bf-2yhak.md) - Tool relocation documentation
|
|
|
|
## Verification Status
|
|
|
|
Last verified: 2026-07-05 (bf-620xp)
|
|
|
|
✅ All DELETE-category generators removed
|
|
✅ All RELOCATE-category generators moved to tools/
|
|
✅ No compiled artifacts (`.o`, `.so`, `.a`, `.dylib`, `.dll`) present
|
|
✅ No obsolete scripts remaining
|
|
✅ 10 KEEP generators remain (actively maintained, co-located with fixtures)
|
|
✅ tools/README.md comprehensive documentation present
|