Commit graph

9 commits

Author SHA1 Message Date
jedarden
f67868bea5 feat(bd-h40): extract Normalizer component — decouple parse from ingest
- Create src/normalizer.ts with normalize(raw, source): NeedleEvent supporting
  all 5 sources: jsonl, otlp-log, otlp-span-start, otlp-span-end, otlp-metric
- Move JSONL parsing logic (canonical, legacy NEEDLE, flat legacy) from
  parser.ts into normalizer.ts
- Refactor parser.ts to thin facade delegating to normalizer
- Tailer now imports directly from normalizer (pure source, no parse coupling)
- Add 63 unit tests covering all source paths, edge cases, and cross-source
  parity between JSONL and OTLP-log

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 12:34:30 -04:00
jedarden
10146b6415 feat(bd-cj8): adopt NeedleEvent canonical schema + publish docs/schema.md
Add NeedleEvent interface as the canonical internal shape versioned at
schema-version 1. Parser validates wire format and asserts schema version
on incoming events. Legacy LogEvent retained as backward-compatible adapter.
docs/schema.md documents all fields, the (worker_id, sequence) ordering
contract, and the full event taxonomy cross-referenced with NeedleEventType.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 11:57:19 -04:00
jedarden
374be30dac fix(bd-vqd): Align NeedleLogEntry and parser with NEEDLE telemetry format
- NeedleLogEntry.worker: accept string | NeedleWorkerObject (NEEDLE emits
  flat runner-provider-model-identifier strings)
- Add optional level field to NeedleLogEntry (NEEDLE always includes it)
- parseNeedleFormat: reconstruct worker as runner-provider-model-identifier
  from legacy object form (was missing provider and model)
- parseNeedleFormat: gate provider/model extraction on object form (was
  accessing .provider/.model unconditionally, yielding undefined)
- parseNeedleFormat: prefer entry.level over inferred level
- inferLogLevel: match NEEDLE's _needle_telemetry_infer_level rules exactly
  (prefix/suffix matching: error.*, *.failed, *.retry, debug.*)
- Add NeedleEventType, NeedleWorkerStatus types to types.ts
- Add session/provider/model optional fields to LogEvent
- Fix store.ts status detection: match on exact NEEDLE event types instead
  of substring-based heuristics
- Update all tests to match corrected behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 22:39:04 -04:00
default
ee90eb05a3 feat(bd-2bt): add POST /api/events endpoint to ingest NEEDLE telemetry
Add HTTP POST endpoint to receive NEEDLE telemetry events from the
fabric.sh forwarder. This bridges NEEDLE and FABRIC, enabling real-time
event ingestion via HTTP.

Changes:
- Add parseEventObject() to parser.ts for parsing JSON objects directly
- Add POST /api/events endpoint with JSON body parser (64KB limit)
- Validate required fields (ts, event) before processing
- Store events and broadcast to WebSocket clients in real-time
- Return 201 Created on success, 400 for invalid payloads

Acceptance criteria met:
- NEEDLE events sent via curl POST arrive in FABRIC's event store
- Events are broadcast to WebSocket clients in real-time
- Invalid payloads return appropriate error codes (400)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:06:55 +00:00
jeda
7ccfc96603 fix: support both string and object worker formats in parser
Handle NEEDLE's aligned format (worker as flat string) alongside
legacy format (worker as nested object) for backwards compatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 23:38:25 +00:00
jeda
f1a6bb6691 feat(bd-2fxx): Align parser to NEEDLE's actual log schema
- Accept ISO timestamp strings, convert to Unix milliseconds for internal use
- Use 'event' field as event type (map to 'msg' for internal use)
- Flatten worker object to string: ${runner}-${identifier}
- Infer log level from event name (error/warn/info/debug)
- Extract bead_id, duration_ms from data payload
- Add session, provider, model fields from NEEDLE format
- Maintain backward compatibility with legacy format
- Add comprehensive NEEDLE format test cases

Verified with actual NEEDLE log files:
- 146/146 lines parsed successfully
- Correctly identifies error-level events (claim_exhausted)
- Correctly identifies warn-level events (claim_retry)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Worker <noreply@anthropic.com>
2026-03-04 23:19:21 +00:00
jeda
1a331eafaa feat(bd-122): Parse conversation events from NEEDLE logs
Add ConversationEvent types and parsing functions to extract conversation
events from NEEDLE log entries:

Types added to types.ts:
- ConversationRole: system, user, assistant, tool
- ConversationEventType: prompt, response, thinking, tool_call, tool_result
- PromptEvent: User input/prompt
- ResponseEvent: Assistant response text
- ThinkingEvent: Internal reasoning/thinking block
- ToolCallEvent: Tool being called with arguments
- ToolResultEvent: Result from a tool call
- ConversationSession: Complete conversation session
- ConversationParseOptions: Options for parsing

Functions added to parser.ts:
- isConversationEvent(): Check if log event contains conversation content
- parseConversationEvent(): Parse single log event to conversation event
- parseConversationEvents(): Parse multiple log events
- parseConversationLine(): Parse single log line
- parseConversationContent(): Parse multi-line log content
- formatConversationEvent(): Format for display

Features:
- Supports explicit conversation fields (conversation_role, conversation_type)
- Supports content fields (prompt, response, thinking, tool_result)
- Supports tool call parsing with arguments normalization
- Content truncation for large responses
- Human-readable tool summaries (e.g., "Read /src/main.ts")
- Filtering options for thinking blocks and tool results

Comprehensive unit tests added for all conversation parsing functions.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Worker <noreply@anthropic.com>
2026-03-04 04:23:09 +00:00
jeda
f8e17ee2ab feat(bd-3jl): FileHeatmap integration complete + build fixes
FileHeatmap component is fully integrated into main TUI app:
- Keyboard shortcut 'H' toggles heatmap view
- Real-time file access aggregation from event store
- Shows most-touched files with multiple sort modes
- All tests passing (943 tests, including 130 FileHeatmap tests)

Additional fixes:
- Fixed ErrorGroupPanel options to support bottom property
- Added vitest globals configuration for proper TypeScript support
- Fixed type assertions in server.test.ts
- Created comprehensive integration documentation

Co-Authored-By: Claude Worker <noreply@anthropic.com>
2026-03-04 04:18:13 +00:00
jeda
a6699db07f feat: implement Phase 1 core infrastructure for FABRIC
Implements working `fabric tail` command with:

- Log tailer module (src/tailer.ts)
  - File watching with fs.watch
  - Follow mode for real-time updates
  - Buffer management for partial lines
  - Graceful shutdown handling

- JSON parser module (src/parser.ts)
  - Validates NEEDLE log format
  - Extracts optional fields (tool, path, bead, duration_ms, error)
  - Human-readable event formatting with color support

- In-memory event store (src/store.ts)
  - Stores events with worker indexing
  - Supports querying with filters
  - Auto-updates worker status based on events

- Working CLI (src/cli.ts)
  - Filters by worker (-w) and level (-l)
  - Supports --json output
  - Shows existing lines with -n flag

Resolves HUMAN bead bd-17q (worker starvation) by providing
working implementation and creating beads for remaining phases.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-03 04:58:42 +00:00