- Add startPosition option and currentPosition getter to LogTailer so
evicted tailers can be resumed from their last read byte offset
- Rewrite DirectoryTailer with a bounded active set (maxActiveFiles=200):
only the N most-recently-modified files have open watchers; older files
are tracked in a fileInfo Map but not watched
- LRU eviction: when the active set is full and a new file needs activation,
the least-recently-active tailer is stopped (position checkpointed) and
replaced
- Re-activation: the poll loop (default 30 s) detects mtime changes in
inactive files and opens them from their saved position so no bytes are
missed or replayed
- RSS back-pressure: skip new activations when process.memoryUsage().rss
exceeds maxRssBytes (default 400 MB)
- Hot-add new files via fs.watch rename, always reading from position 0
- Add three new tests: 10k-file cap assertion, LRU eviction+reactivation,
and position-checkpoint correctness
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire shared EventDeduplicator across all ingestion paths (JSONL tailer,
OTLP/gRPC receiver, OTLP/HTTP receiver) so duplicate events from dual
ingestion are silently dropped on (session_id, worker_id, sequence).
Also adds docs/needle-exporter-wiring.md (OTLP configuration guide for
NEEDLE), SpanDag React component, EventFilter.eventType field, and
various test/layout fixes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Create src/normalizer.ts with normalize(raw, source): NeedleEvent supporting
all 5 sources: jsonl, otlp-log, otlp-span-start, otlp-span-end, otlp-metric
- Move JSONL parsing logic (canonical, legacy NEEDLE, flat legacy) from
parser.ts into normalizer.ts
- Refactor parser.ts to thin facade delegating to normalizer
- Tailer now imports directly from normalizer (pure source, no parse coupling)
- Add 63 unit tests covering all source paths, edge cases, and cross-source
parity between JSONL and OTLP-log
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements working `fabric tail` command with:
- Log tailer module (src/tailer.ts)
- File watching with fs.watch
- Follow mode for real-time updates
- Buffer management for partial lines
- Graceful shutdown handling
- JSON parser module (src/parser.ts)
- Validates NEEDLE log format
- Extracts optional fields (tool, path, bead, duration_ms, error)
- Human-readable event formatting with color support
- In-memory event store (src/store.ts)
- Stores events with worker indexing
- Supports querying with filters
- Auto-updates worker status based on events
- Working CLI (src/cli.ts)
- Filters by worker (-w) and level (-l)
- Supports --json output
- Shows existing lines with -n flag
Resolves HUMAN bead bd-17q (worker starvation) by providing
working implementation and creating beads for remaining phases.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
