jedarden/pdftract
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pdftract/tests/fixtures/PROVENANCE.md
jedarden 7788719e0a docs(bf-ad2pp): analyze unmapped glyph fixture requirements 
- Read design documents notes/bf-68f9i-design.md and notes/bf-68f9i-glyphs.md
- Identified 7 unmapped glyphs across 4 failure categories
- Identified 3 mapped AGL glyphs for comparison
- Verified existing generator implementation (gen_unmapped_comprehensive.rs)
- Confirmed fixture files exist and are correctly structured
- Documented implementation analysis in notes/bf-ad2pp.md

Closes bf-ad2pp
2026-07-03 19:17:02 -04:00

370 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# EC-04-rc4-encrypted.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 1.7, RC4 encryption (V=1, R=2), 40-bit key, user password: "user40"
Generated: 2026-05-28
# EC-05-aes128-encrypted.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 1.7, AES-128 encryption (V=2, R=3), 128-bit key, user password: "user128"
Generated: 2026-05-28
# EC-06-aes256-encrypted.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 2.0, AES-256 encryption (V=5, R=5), 256-bit key, user password: "user256"
Generated: 2026-05-28
# EC-empty-password.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 1.7, no encryption (control fixture)
Generated: 2026-05-28
# EC-04-rc4-encrypted.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 1.7, RC4 encryption (V=1, R=2), 40-bit key, user password: "user40"
Generated: 2026-05-28
# EC-05-aes128-encrypted.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 1.7, AES-128 encryption (V=2, R=3), 128-bit key, user password: "user128"
Generated: 2026-05-28
# EC-06-aes256-encrypted.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 2.0, AES-256 encryption (V=5, R=5), 256-bit key, user password: "user256"
Generated: 2026-05-28
# EC-empty-password.pdf
Generated by tests/fixtures/generate_encrypted_fixtures.py
PDF 1.7, no encryption (control fixture)
Generated: 2026-05-28
# sample.pdf
Copied from valid-minimal.pdf for SDK examples default path
Minimal valid PDF v1.4 fixture for contract method examples
Generated: 2026-05-31
# json_schema/simple_invoice.pdf
Simple invoice PDF for JSON schema validation tests
Generated: 2026-06-01
# json_schema/EC-04-rc4-encrypted.pdf
Copied from fixtures/EC-04-rc4-encrypted.pdf for JSON schema validation
PDF 1.7, RC4 encryption (V=1, R=2), 40-bit key, user password: "user40"
Generated: 2026-06-01
# json_schema/EC-05-aes128-encrypted.pdf
Copied from fixtures/EC-05-aes128-encrypted.pdf for JSON schema validation
PDF 1.7, AES-128 encryption (V=2, R=3), 128-bit key, user password: "user128"
Generated: 2026-06-01
# json_schema/valid-minimal.pdf
Minimal valid PDF v1.4 fixture for JSON schema validation tests
Generated: 2026-05-28
# json_schema/sample.pdf
Copied from valid-minimal.pdf for SDK examples default path
Minimal valid PDF v1.4 fixture for contract method examples
Generated: 2026-05-31
# vector/academic-paper/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Academic paper on machine learning - Abstract, Introduction, Methods, Results, Conclusion
Generated: 2026-06-01
# vector/technical-documentation/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
API documentation with Getting Started, Authentication, Endpoints, Rate Limits
Generated: 2026-06-01
# vector/legal-contract/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Service Agreement with Services, Term, Compensation, Confidentiality, Termination, Governing Law
Generated: 2026-06-01
# vector/scientific-report/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Climate Research Report with Executive Summary, Data Collection, Analysis, Findings, Recommendations
Generated: 2026-06-01
# vector/user-manual/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Product User Manual with Quick Start Guide, Unboxing, Setup, Features, Troubleshooting, Support
Generated: 2026-06-01
# vector/financial-report/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Q1 Financial Report with Revenue, Expenses, Net Income, Outlook, Risk Factors
Generated: 2026-06-01
# vector/conference-proceedings/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Conference Proceedings with Keynote Address, Paper Session, Panel Discussion, Workshop
Generated: 2026-06-01
# vector/medical-research/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Clinical Trial Results with Background, Methodology, Results, Discussion, Conclusion
Generated: 2026-06-01
# vector/multi-page-academic/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Multi-page academic paper (3 pages) - Abstract, Introduction, Conclusion
Generated: 2026-06-01
# vector/code-documentation/source.pdf
Generated by tests/fixtures/vector/generate_vector_cer_corpus.py
Clean vector PDF with embedded text for CER testing (PDF 1.4, Type1 Helvetica, WinAnsiEncoding)
Code library documentation with Installation, Quick Example, API Reference, Supported Formats, Limitations, License
Generated: 2026-06-01
# scanned/receipt/receipt-300dpi.pdf
Generated by tests/fixtures/scanned/generate_scanned_fixtures.py
Source PDF for scan simulation at 300 DPI
Supermarket receipt with items, prices, totals (Helvetica 10pt, Letter, 14pt line spacing)
Generated: 2026-06-01
# scanned/receipt/receipt-300dpi-scanned.pdf
Generated by pdftoppm + img2pdf from receipt-300dpi.pdf at 300 DPI
Scan simulation for OCR testing (rasterized image-only PDF)
Generated: 2026-06-01
# scanned/documents/invoice-300dpi.pdf
Generated by tests/fixtures/scanned/generate_scanned_fixtures.py
Source PDF for scan simulation at 300 DPI
Service invoice with line items, totals, payment terms (Helvetica 11pt, Letter, 16pt line spacing)
Generated: 2026-06-01
# scanned/documents/invoice-300dpi-scanned.pdf
Generated by pdftoppm + img2pdf from invoice-300dpi.pdf at 300 DPI
Scan simulation for OCR testing (rasterized image-only PDF)
Generated: 2026-06-01
# scanned/documents/form-300dpi.pdf
Generated by tests/fixtures/scanned/generate_scanned_fixtures.py
Source PDF for scan simulation at 300 DPI
Employment application form with fields and checkboxes (Helvetica 11pt, Letter, 18pt line spacing)
Generated: 2026-06-01
# scanned/documents/form-300dpi-scanned.pdf
Generated by pdftoppm + img2pdf from form-300dpi.pdf at 300 DPI
Scan simulation for OCR testing (rasterized image-only PDF)
Generated: 2026-06-01
# scanned/multi-page/doc-10page-300dpi.pdf
Generated by tests/fixtures/scanned/generate_scanned_fixtures.py
Source PDF for scan simulation at 300 DPI (10 pages with diverse content)
Times-Roman 12pt, Letter, 18pt line spacing, "Page N:" markers
Generated: 2026-06-01
# scanned/multi-page/doc-10page-300dpi-scanned.pdf
Generated by pdftoppm + img2pdf from doc-10page-300dpi.pdf at 300 DPI
Scan simulation for OCR testing (rasterized image-only PDF, 10 pages)
Generated: 2026-06-01
# scanned/receipt/receipt-300dpi.pdf
Generated by tests/fixtures/scanned/generate_scanned_fixtures.py
Source PDF for scan simulation at 300 DPI
Simple sales receipt with itemized list and totals (Helvetica 11pt, 6.5" x 4", 14pt line spacing)
Generated: 2026-06-01
# scanned/receipt/receipt-300dpi-scanned.pdf
Generated by pdftoppm + img2pdf from receipt-300dpi.pdf at 300 DPI
Scan simulation for OCR testing (rasterized image-only PDF)
Generated: 2026-06-01
# scanned/documents/invoice-300dpi.pdf
Generated by tests/fixtures/scanned/generate_scanned_fixtures.py
Source PDF for scan simulation at 300 DPI
Business invoice with line items, subtotal, tax, and total (Helvetica 11pt, Letter, 16pt line spacing)
Generated: 2026-06-01
# scanned/documents/invoice-300dpi-scanned.pdf
Generated by pdftoppm + img2pdf from invoice-300dpi.pdf at 300 DPI
Scan simulation for OCR testing (rasterized image-only PDF)
# json_schema/simple-text.pdf
Minimal text-only PDF for JSON schema validation tests
Generated: 2026-06-01
# encoding/no-mapping.pdf
Generated by tests/fixtures/generate_unicode_recovery_fixtures.rs
PDF 1.4, Type1 font with custom glyph names, no ToUnicode CMap, no standard encoding
Level 4 Unicode recovery test fixture (worst case: no encoding fallback)
Content: <20><><EFBFBD> (three U+FFFD replacement characters - glyph names /g001/g002/g003 are not in AGL and cannot be recovered)
Generated: 2026-06-09
Regenerated: 2026-07-02 (bf-512z1)
Regenerated: 2026-07-03 (bf-f0xqd: corrected ground truth from "ABC" to U+FFFD)
Regenerated: 2026-07-03 (bf-1m30m: regenerated via generate_unicode_recovery_fixtures.rs)
SHA256: b24f88d3add958bfec1d6b134f2cd030cd41bb1932bedbe99405599bd01fa8f0
# encoding/agl-only.pdf
Generated by tests/fixtures/generate_unicode_recovery_fixtures.rs
PDF 1.4, Type1 font with AGL glyph names only, no ToUnicode CMap
Level 2 Unicode recovery test fixture (Adobe Glyph List fallback)
Content: "Hello\nWorld" (extracted via AGL glyph name mapping)
Generated: 2026-06-09
Regenerated: 2026-07-02 (bf-512z1)
Regenerated: 2026-07-03 (bf-1m30m: regenerated via generate_unicode_recovery_fixtures.rs)
SHA256: c6e8a067a637457ac9852c1d56745f5eac1f0fe441d19af08060d13d862b74b7
# encoding/fingerprint-match.pdf
Generated by tests/fixtures/generate_unicode_recovery_fixtures.rs
PDF 1.4, embedded Type1 font subset, no ToUnicode CMap
Level 3 Unicode recovery test fixture (SHA-256 font fingerprint matching)
Content: "Test" (extracted via font-fingerprints.json fingerprint lookup)
Generated: 2026-06-09
Regenerated: 2026-07-02 (bf-512z1)
Regenerated: 2026-07-03 (bf-1m30m: regenerated via generate_unicode_recovery_fixtures.rs)
SHA256: bd9a3533c0f31fb20bf5532fcff4cb2ae399f8d2e22ea1d09538837410544c4a
# encoding/shape-match.pdf
Generated by tests/fixtures/generate_unicode_recovery_fixtures.rs
PDF 1.4, Type1 font with custom glyph names, no ToUnicode CMap
Level 4 Unicode recovery test fixture (glyph shape recognition from glyph-shapes.json)
Content: "S" (extracted via glyph shape database lookup)
Generated: 2026-06-09
Regenerated: 2026-07-02 (bf-512z1)
Regenerated: 2026-07-03 (bf-1m30m: regenerated via generate_unicode_recovery_fixtures.rs)
SHA256: 6e99cfe2be79f5bcdc2a99b1004ce0e2288be56b081f8e558ecdecd78f702c55
# encoding/unmapped-glyphs.pdf
Generated by tests/fixtures/generate_unmapped_glyphs.rs
PDF 1.4, Type1 font with custom glyph names, no ToUnicode CMap, no standard encoding
GLYPH_UNMAPPED diagnostic test fixture - custom glyph names not in AGL
Content: <20><><EFBFBD><EFBFBD> (four U+FFFD replacement characters - glyph names /CustomAlpha, /CustomBeta, /CustomGamma, /CustomDelta are not in AGL and cannot be recovered)
Generated: 2026-07-03 (bf-68f9i)
SHA256: 5e472e1c5997318c5d180ff4bc4ae21f1231c87e8b905b059546e274e15393c6
# encoding/unmapped-comprehensive.pdf
Generated by tests/fixtures/gen_unmapped_comprehensive.rs
PDF 1.4, Type1 font with custom glyph names, no ToUnicode CMap, custom /Differences encoding
Comprehensive GLYPH_UNMAPPED diagnostic test fixture - 7 unmapped + 3 mapped glyphs
Content: "<22><><EFBFBD>\n<><6E><EFBFBD><EFBFBD>\nAB " (7×U+FFFD for /g001/g002/g003/CustomA/CustomB/NotAGlyph/glyph_0041, then "AB " for /A/B/space)
Unmapped glyphs: /g001, /g002, /g003 (PUA), /CustomA, /CustomB (custom), /NotAGlyph (orphaned), /glyph_0041 (non-AGL)
Mapped glyphs: /A, /B, /space (AGL - Level 2 fallback)
Ground truth: encoding/unmapped-comprehensive.txt (3 lines: 3×U+FFFD, 4×U+FFFD, "AB ")
Generated: 2026-07-03 (bf-5taa6)
SHA256: a4d4c76c3edbc1bb9b870e9d0c453a943700dd4508cfdfe020ba0a28675cc94c
# cjk/cjk-chinese-gb18030.pdf
Generated by tests/fixtures/generate_cjk_with_tounicode.rs
PDF 1.4, Type0 composite font with GBpc-EUC-H CMap encoding and ToUnicode CMap
Phase 2.3 CJK encoding test fixture - Simplified Chinese (GB18030)
Content: "你好世界" (Simplified Chinese, 4 characters)
Ground truth: cjk-chinese-gb18030.txt (12 bytes, UTF-8)
Font: AdobeSongStd-Light (CIDFontType0, Adobe-GB1 CIDSystemInfo)
ToUnicode: Adobe-GB1-UCS2 CMap with custom character mappings
Generated: 2026-06-24
Regenerated: 2026-06-24 (bf-3ourh)
# cjk/cjk-japanese-shiftjis.pdf
Generated by tests/fixtures/generate_cjk_with_tounicode.rs
PDF 1.4, Type0 composite font with 90ms-RKSJ-H CMap encoding and ToUnicode CMap
Phase 2.3 CJK encoding test fixture - Japanese (Shift-JIS)
Content: "こんにちは" (Japanese, 5 characters)
Ground truth: cjk-japanese-shiftjis.txt (15 bytes, UTF-8)
Font: HeiseiMin-W3 (CIDFontType0, Adobe-Japan1 CIDSystemInfo)
ToUnicode: Adobe-Japan1-UCS2 CMap with custom character mappings
Generated: 2026-06-24
Regenerated: 2026-06-24 (bf-3ourh)
# cjk/cjk-korean-euckr.pdf
Generated by tests/fixtures/generate_cjk_with_tounicode.rs
PDF 1.4, Type0 composite font with KSCms-UHC-H CMap encoding and ToUnicode CMap
Phase 2.3 CJK encoding test fixture - Korean (EUC-KR)
Content: "안녕하세요" (Korean, 5 characters)
Ground truth: cjk-korean-euckr.txt (15 bytes, UTF-8)
Font: HYSMyeongJo-Medium (CIDFontType0, Adobe-Korea1 CIDSystemInfo)
ToUnicode: Adobe-Korea1-UCS2 CMap with custom character mappings
Generated: 2026-06-24
Regenerated: 2026-06-24 (bf-3ourh)
# cjk/cjk-tc-big5.pdf
Generated by tests/fixtures/generate_cjk_with_tounicode.rs
PDF 1.4, Type0 composite font with ETen-B5-H CMap encoding and ToUnicode CMap
Phase 2.3 CJK encoding test fixture - Traditional Chinese (Big5)
Content: "你好世界" (Traditional Chinese, 4 characters)
Ground truth: cjk-tc-big5.txt (12 bytes, UTF-8)
Font: PMingLiU-Light (CIDFontType0, Adobe-CNS1 CIDSystemInfo)
ToUnicode: Adobe-CNS1-UCS2 CMap with custom character mappings
Generated: 2026-06-24
Regenerated: 2026-06-24 (bf-3ourh)
# tagged/tagged-ua-simple.pdf
Generated by tests/fixtures/generate_tagged_fixtures.rs
PDF 1.7, PDF/UA-1 (Universal Accessibility) with /MarkInfo /Marked true
Phase 7.1 StructTree extraction test fixture - Minimal tagged PDF
StructTree: Document root with H1 (MCID 0) and P (MCID 1) elements
Content: "Heading Level 1" (H1) and "This is a paragraph under the heading." (P)
Ground truth: tagged-ua-simple.expected.json (structure_tree field with Document/H1/P hierarchy)
Generated: 2026-06-24 (bf-5pyzm)
# tagged/tagged-ua-table.pdf
Generated by tests/fixtures/generate_tagged_fixtures.rs
PDF 1.7, PDF/UA-1 with table structure (Table/TR/TD elements)
Phase 7.1 StructTree extraction test fixture - Tagged table
StructTree: Document root with H1 (MCID 0) and Table (2 rows × 3 columns, MCIDs 1-6)
Content: "Simple Table Example" (H1) and table cells (A1, B1, C1, A2, B2, C2)
Ground truth: tagged-ua-table.expected.json (structure_tree with nested Table/TR/TD hierarchy)
Generated: 2026-06-24 (bf-5pyzm)
# tagged/tagged-a-2a.pdf
Generated by tests/fixtures/generate_tagged_fixtures.rs
PDF 1.7, PDF/A-2a (Level A accessibility) with XMP metadata and StructTree
Phase 7.1 StructTree extraction test fixture - PDF/A-2a compliance
StructTree: Document root with H1 (MCID 0) and P (MCID 1) elements
XMP Metadata: pdfaid:part="2", pdfaid:conformance="A"
Content: "PDF/A-2a Document" (H1) and "This is a PDF/A-2a compliant tagged document." (P)
Ground truth: tagged-a-2a.expected.json (structure_tree with Document/H1/P hierarchy)
Generated: 2026-06-24 (bf-5pyzm)
# tagged/tagged-mcid-ordering.pdf
Generated by tests/fixtures/generate_tagged_fixtures.rs
PDF 1.7, Tagged PDF with MCID-to-structure-element mapping test
Phase 7.1 StructTree extraction test fixture - MCID ordering validation
StructTree: Document root with three P elements (MCIDs 0, 1, 2 in sequence)
Content: "First block", "Second block", "Third block" (each as separate P elements)
Ground truth: tagged-mcid-ordering.expected.json (structure_tree with sequential MCID mapping)
Generated: 2026-06-24 (bf-5pyzm)
# forms/acroform-text-fields.pdf
Generated by tests/fixtures/forms/generate_form_fixtures.py
PDF 1.4, AcroForm with multiple field types
Phase 7.4 form field extraction test fixture - Text, checkbox, radio, and choice fields
Fields: employee_name (Tx, required, maxLen=50), address (Tx, multiline), is_manager (Btn, checkbox), department (Btn, radio group with sales/engineering options), role (Ch, choice/combo with manager/developer/designer options)
Ground truth: forms/acroform-text-fields.json (form_fields array with field names, types, values, flags, and options)
Generated: 2026-07-03 (bf-4sn1j)
# forms/acroform-readonly.pdf
Generated by tests/fixtures/forms/generate_form_fixtures.py
PDF 1.4, AcroForm with read-only field flags
Phase 7.4 form field extraction test fixture - ReadOnly flag validation
Fields: company_name (Tx, ReadOnly=true), contact_email (Tx, ReadOnly=false), verified (Btn, checkbox, ReadOnly=true)
Ground truth: forms/acroform-readonly.json (form_fields array with read_only flags)
Generated: 2026-07-03 (bf-4sn1j)
# forms/acroform-submit.pdf
Generated by tests/fixtures/forms/generate_form_fixtures.py
PDF 1.4, AcroForm with submit and reset button actions
Phase 7.4 form field extraction test fixture - Button actions
Fields: username (Tx, required), submit (Btn, pushbutton with SubmitForm action to https://example.com/submit), reset (Btn, pushbutton with ResetForm action)
Ground truth: forms/acroform-submit.json (form_fields array with action objects for buttons)
Generated: 2026-07-03 (bf-4sn1j)
# forms/xfa-dynamic.pdf
Generated by tests/fixtures/forms/generate_form_fixtures.py
PDF 1.7, AcroForm with XFA data packet placeholder
Phase 7.4 form field extraction test fixture - XFA detection
Fields: xfa_field1 (Tx, fallback field), XFA array with XML datasets/template placeholder (XFA XML parsing not yet implemented)
Ground truth: forms/xfa-dynamic.json (form_fields array with note about XFA parsing status)
Generated: 2026-07-03 (bf-4sn1j)
Reference in a new issue View git blame Copy permalink