jedarden/pdftract

Fork 0

Commit graph

Author	SHA1	Message	Date
jedarden	e25a4fc78d	docs(pdftract-10cf): finalize table structure reconstruction research note v1.0 Added complete pseudo-code listings for: - Line-based grid reconstruction algorithm (path segment collection, collinear merging, intersection finding, cell synthesis) - Borderless table detection via vertical projection profiles and column separator inference - Cell content assignment via centroid containment Also added version history section documenting v0.9 -> v1.0 changes. Closes: pdftract-10cf	2026-05-24 09:58:03 -04:00
jedarden	26bdd255c8	feat(pdftract-ilen): implement header row detection with bold+TH support Implement header row detection for tables using two signals: 1. Bold font detection (fully implemented) 2. StructTree TH detection (stub pending MCID tracking) Bold detection: - is_bold_font(): detects bold fonts from PostScript name patterns - is_cell_bold(): checks if all non-whitespace content in a cell is bold - is_bold_header_row(): validates rows with >=2 bold cells - count_header_rows(): counts contiguous bold headers from top - Cell::mark_header_rows(): sets is_header_row flag on cells TH detection (stub): - is_th_header_row(): placeholder for StructTree TH detection Requires MCID tracking on TableSpan (future work) Will use ParentTree to map MCIDs to StructElems Will verify TR > TH chain structure Combined detection: - is_header_row(): combines bold and TH signals - Bold wins on conflict per body data design principle Documentation: - Updated table-structure-reconstruction.md with full header detection spec - Documented implemented vs pending signals - Added implementation notes for TH detection Tests: - 45 tests covering all bold detection scenarios - Tests for multi-row headers (contiguous from top) - Tests for single-cell row exclusion - Tests for empty/whitespace cell handling - Placeholder tests for TH detection Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-23 23:32:54 -04:00
jedarden	b805593973	Add six research documents covering output-side extraction topics - table-structure-reconstruction: line detection, gap analysis, Hough transform, graph-based cell reconstruction, merged cells, multi-page tables - mathematical-expression-handling: five encoding cases, OpenType MATH table, symbol font recovery, spatial heuristics, LaTeX reconstruction, fallback tiers - language-detection-and-script-handling: UAX #24/#9, Arabic/Hebrew bidi, CJK vertical text, ligature normalization, whatlang/lingua integration - document-classification-and-zone-labeling: margin heuristics, font clustering, cross-page recurrence, footnote/caption/sidebar detection - post-extraction-normalization: hyphen handling, ligature expansion, paragraph reconstruction, Unicode normalization, pipeline ordering - chunking-for-llm-consumption: semantic snapping, heading hierarchy, sliding window overlap, table chunking strategies, token budget, late chunking Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-16 14:56:25 -04:00

Author

SHA1

Message

Date

jedarden

e25a4fc78d

docs(pdftract-10cf): finalize table structure reconstruction research note v1.0

Added complete pseudo-code listings for:
- Line-based grid reconstruction algorithm (path segment collection,
  collinear merging, intersection finding, cell synthesis)
- Borderless table detection via vertical projection profiles
  and column separator inference
- Cell content assignment via centroid containment

Also added version history section documenting v0.9 -> v1.0 changes.

Closes: pdftract-10cf

2026-05-24 09:58:03 -04:00

jedarden

26bdd255c8

feat(pdftract-ilen): implement header row detection with bold+TH support

Implement header row detection for tables using two signals:
1. Bold font detection (fully implemented)
2. StructTree TH detection (stub pending MCID tracking)

Bold detection:
- is_bold_font(): detects bold fonts from PostScript name patterns
- is_cell_bold(): checks if all non-whitespace content in a cell is bold
- is_bold_header_row(): validates rows with >=2 bold cells
- count_header_rows(): counts contiguous bold headers from top
- Cell::mark_header_rows(): sets is_header_row flag on cells

TH detection (stub):
- is_th_header_row(): placeholder for StructTree TH detection
  Requires MCID tracking on TableSpan (future work)
  Will use ParentTree to map MCIDs to StructElems
  Will verify TR > TH chain structure

Combined detection:
- is_header_row(): combines bold and TH signals
- Bold wins on conflict per body data design principle

Documentation:
- Updated table-structure-reconstruction.md with full header detection spec
- Documented implemented vs pending signals
- Added implementation notes for TH detection

Tests:
- 45 tests covering all bold detection scenarios
- Tests for multi-row headers (contiguous from top)
- Tests for single-cell row exclusion
- Tests for empty/whitespace cell handling
- Placeholder tests for TH detection

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-23 23:32:54 -04:00

jedarden

b805593973

Add six research documents covering output-side extraction topics

- table-structure-reconstruction: line detection, gap analysis, Hough
  transform, graph-based cell reconstruction, merged cells, multi-page tables
- mathematical-expression-handling: five encoding cases, OpenType MATH table,
  symbol font recovery, spatial heuristics, LaTeX reconstruction, fallback tiers
- language-detection-and-script-handling: UAX #24/#9, Arabic/Hebrew bidi,
  CJK vertical text, ligature normalization, whatlang/lingua integration
- document-classification-and-zone-labeling: margin heuristics, font
  clustering, cross-page recurrence, footnote/caption/sidebar detection
- post-extraction-normalization: hyphen handling, ligature expansion,
  paragraph reconstruction, Unicode normalization, pipeline ordering
- chunking-for-llm-consumption: semantic snapping, heading hierarchy,
  sliding window overlap, table chunking strategies, token budget, late chunking

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-16 14:56:25 -04:00

3 commits