From f164e7e25318d72412fe9e306ed55780169c1692 Mon Sep 17 00:00:00 2001 From: jeda Date: Mon, 2 Mar 2026 01:52:04 +0000 Subject: [PATCH] docs: refocus FABRIC on worker visualization via logging/telemetry - Shift focus from bead tracking to NEEDLE worker visibility - Emphasize TUI and HTML output formats as primary deliverables - Add CLI interface examples (fabric tui, fabric html, fabric logs) - Define worker event data model and log format contract - Update README to reflect worker-centric purpose Co-Authored-By: Claude Opus 4.6 --- README.md | 42 +++++---- docs/plan.md | 256 ++++++++++++++++++++++++++++++++------------------- 2 files changed, 187 insertions(+), 111 deletions(-) diff --git a/README.md b/README.md index ed43445..169d496 100644 --- a/README.md +++ b/README.md @@ -2,32 +2,40 @@ **Flow Analysis & Bead Reporting Interface Console** -A visualization and dashboard system for monitoring [NEEDLE](../claude-config)'s bead orchestration output. +A visualization system for surfacing NEEDLE worker activity through TUI and HTML dashboards. ## Purpose -FABRIC provides real-time and historical insights into bead processing workflows orchestrated by NEEDLE. It transforms raw orchestration data into actionable visualizations, enabling: +FABRIC consumes logging and telemetry output from NEEDLE workers, transforming raw execution data into reviewable visualizations: -- **Flow Analysis**: Track bead lifecycle from creation through completion -- **Bead Reporting**: Aggregate metrics on processing times, success rates, and bottlenecks -- **Interface Console**: Interactive dashboard for monitoring active and historical workflows +- **Flow Analysis**: Visualize worker execution timelines and patterns +- **Bead Reporting**: Surface what workers are doing and how they're performing +- **Interface Console**: Both TUI (terminal) and HTML dashboards for review -## Architecture +## Output Formats -FABRIC integrates with NEEDLE's output streams to create a comprehensive view of: -- Bead state transitions -- Worker assignment and execution patterns -- Dependency graphs and blocking relationships -- Processing throughput and latency metrics +### TUI Dashboard +Real-time terminal interface showing: +- Active worker status grid +- Live log streaming with filtering +- Worker detail views and session history +- Keyboard-driven navigation + +### HTML Reports +Static and interactive browser-based views: +- Session timeline visualizations (Gantt-style) +- Metrics charts (API calls, tokens, duration) +- Searchable log explorer +- Shareable, self-contained reports ## Relationship to NEEDLE -While NEEDLE orchestrates bead processing, FABRIC answers questions like: -- Which beads are currently blocked and why? -- What's the average time-to-completion for different bead types? -- How many beads are in each state (pending, active, completed)? -- Which workers are processing which beads? -- Where are the bottlenecks in the workflow? +NEEDLE orchestrates workers; FABRIC surfaces their activity: +- What is each worker currently doing? +- How long are tasks taking? +- What errors are occurring? +- What's the API/token usage? +- What does the execution timeline look like? ## Status diff --git a/docs/plan.md b/docs/plan.md index 5514c5b..81802a9 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -4,146 +4,214 @@ ## Overview -FABRIC will provide visualization and analytics for NEEDLE's bead orchestration system. This document outlines the implementation roadmap. +FABRIC provides visualization of NEEDLE worker activity by consuming logging and telemetry output. It generates both TUI (terminal) and HTML visualizations for reviewing worker execution patterns, performance, and output. ## Goals -1. **Real-time Monitoring**: Live dashboard showing active bead processing -2. **Historical Analysis**: Query and visualize past workflow executions -3. **Performance Metrics**: Identify bottlenecks and optimization opportunities -4. **Debugging Support**: Trace individual bead lifecycles and dependencies +1. **Worker Visibility**: Surface what NEEDLE workers are doing in real-time +2. **Log Aggregation**: Collect and present worker logging output in digestible formats +3. **Telemetry Analysis**: Visualize performance metrics, execution timelines, and resource usage +4. **Dual Output**: Generate both TUI dashboards for terminal users and HTML reports for browser review ## Data Sources -FABRIC will consume data from: +FABRIC consumes NEEDLE's logging and telemetry output: ### Primary Sources -- **Bead JSONL files**: Read `.beads/*.jsonl` for current state -- **Worker logs**: Parse execution logs for timing and error data -- **NEEDLE orchestration events**: Real-time event stream (if available) +- **Worker stdout/stderr**: Captured execution output +- **Structured logs**: JSON-formatted log events from workers +- **Telemetry streams**: Timing, resource usage, API call metrics +- **Session transcripts**: Worker conversation/execution history ### Data Model ```typescript -interface BeadRecord { - id: string; - type: string; - status: 'pending' | 'active' | 'completed' | 'failed' | 'blocked'; - priority: string; - created_at: timestamp; - started_at?: timestamp; - completed_at?: timestamp; - assigned_worker?: string; - dependencies?: string[]; - blocking?: string[]; +interface WorkerEvent { + worker_id: string; + session_id: string; + timestamp: number; + event_type: 'log' | 'metric' | 'state_change' | 'tool_call' | 'error'; + level?: 'debug' | 'info' | 'warn' | 'error'; + message?: string; metadata?: Record; } + +interface WorkerSession { + worker_id: string; + session_id: string; + started_at: number; + ended_at?: number; + status: 'running' | 'completed' | 'failed' | 'idle'; + task_description?: string; + events: WorkerEvent[]; + metrics: WorkerMetrics; +} + +interface WorkerMetrics { + api_calls: number; + tool_invocations: number; + tokens_in: number; + tokens_out: number; + duration_ms: number; + errors: number; +} ``` ## Architecture Components -### 1. Data Ingestion Layer -- **Watcher**: Monitor `.beads/*.jsonl` for changes -- **Parser**: Extract and normalize bead records -- **Event Stream**: Convert file changes to real-time events +### 1. Log Collector +- **Stream Reader**: Consume NEEDLE worker output streams +- **Log Parser**: Extract structured data from log lines +- **Event Normalizer**: Convert various log formats to unified schema -### 2. Processing Layer -- **State Aggregator**: Maintain current view of all beads -- **Metrics Calculator**: Compute throughput, latency, success rates -- **Dependency Resolver**: Build bead relationship graphs +### 2. Telemetry Aggregator +- **Metrics Accumulator**: Track counters, gauges, histograms +- **Timeline Builder**: Construct execution timelines per worker +- **Session Tracker**: Group events by worker session -### 3. Storage Layer -- **Time-series DB**: Store metrics for historical analysis (InfluxDB/TimescaleDB) -- **Graph Store**: Bead dependency relationships (in-memory or Neo4j) -- **Cache**: Fast access to current state (Redis/Valkey) +### 3. Visualization Renderers -### 4. API Layer -- **REST API**: Query beads, metrics, and relationships -- **WebSocket**: Real-time updates for dashboard -- **GraphQL** (optional): Flexible querying for complex visualizations +#### TUI Renderer +- **Live Dashboard**: Real-time terminal display using blessed/ink/textual +- **Log Viewer**: Scrollable, filterable log output +- **Worker Grid**: At-a-glance status of all workers +- **Detail Pane**: Deep-dive into specific worker sessions -### 5. Visualization Layer -- **Web Dashboard**: React/Vue-based UI -- **CLI Tool**: Terminal-based monitoring (`fabric status`, `fabric trace `) -- **Metrics Export**: Prometheus-compatible endpoint +#### HTML Renderer +- **Static Reports**: Self-contained HTML files for sharing/archiving +- **Interactive Dashboard**: Browser-based live view +- **Timeline Visualization**: Gantt-style execution timelines +- **Log Explorer**: Searchable, syntax-highlighted log viewer + +### 4. Output Formats +- **TUI**: Direct terminal rendering (ncurses/blessed style) +- **HTML**: Static files or served via local HTTP +- **JSON**: Raw data export for external tools +- **Markdown**: Summary reports for documentation ## Key Features ### Phase 1: Foundation (MVP) -- [ ] Read and parse bead JSONL files -- [ ] Display current bead counts by status -- [ ] List active beads with basic details -- [ ] Simple CLI for querying state +- [ ] Consume NEEDLE worker log streams +- [ ] Parse structured JSON log events +- [ ] Simple TUI: list active workers with status +- [ ] Basic HTML: render session logs as static page -### Phase 2: Visualization -- [ ] Web dashboard with real-time updates -- [ ] Bead state distribution charts (pie/bar) -- [ ] Timeline view of bead processing -- [ ] Dependency graph visualization +### Phase 2: TUI Dashboard +- [ ] Real-time worker status grid +- [ ] Live log streaming with filtering +- [ ] Worker detail view (select worker, see history) +- [ ] Keyboard navigation and search -### Phase 3: Analytics -- [ ] Historical trend analysis -- [ ] Performance metrics (avg completion time, throughput) -- [ ] Bottleneck detection (most-blocked beads) -- [ ] Worker utilization statistics +### Phase 3: HTML Visualizations +- [ ] Session timeline visualization (Gantt-style) +- [ ] Metrics charts (API calls, tokens, duration) +- [ ] Searchable log explorer with syntax highlighting +- [ ] Export/share capabilities -### Phase 4: Advanced Features -- [ ] Alerting on stalled workflows -- [ ] Predictive completion estimates -- [ ] Custom dashboard widgets -- [ ] Export capabilities (CSV, JSON, reports) +### Phase 4: Advanced Analytics +- [ ] Cross-session analysis (patterns, trends) +- [ ] Error clustering and root cause hints +- [ ] Performance regression detection +- [ ] Custom dashboard layouts ## Technology Stack -### Backend Options -- **Node.js + TypeScript**: Fast development, good JSONL parsing -- **Python + FastAPI**: Rich data analysis libraries (pandas, plotly) -- **Go**: High performance, good for file watching and streaming +### Log Processing +- **Node.js streams**: Efficient log consumption +- **pino/winston parsers**: Structured log parsing +- **RxJS**: Reactive event stream processing -### Frontend Options -- **React + Recharts**: Component-based, good charting library -- **Vue + Chart.js**: Lightweight, reactive -- **Svelte + D3.js**: Minimal bundle size, powerful visualizations +### TUI Framework Options +- **blessed/blessed-contrib**: Feature-rich terminal UI (Node.js) +- **ink**: React for CLI (Node.js) +- **textual**: Modern TUI framework (Python) +- **bubbletea**: Elegant TUI framework (Go) -### Storage -- **SQLite**: Simple, file-based, good for MVP -- **PostgreSQL + TimescaleDB**: Production-grade time-series -- **Redis/Valkey**: Caching and pub/sub for real-time updates +### HTML Generation +- **Static**: Generate self-contained HTML files with embedded CSS/JS +- **Templates**: Handlebars/EJS for report generation +- **Charts**: Chart.js, Recharts, or Plotly for visualizations +- **Timeline**: vis-timeline or custom D3.js -## Deployment Model +### Serving (Optional) +- **Local HTTP server**: Serve HTML dashboard on localhost +- **WebSocket**: Real-time updates to browser + +## CLI Interface + +```bash +# TUI mode - live dashboard +fabric tui + +# Watch specific worker +fabric tui --worker + +# Generate HTML report for session +fabric html --session --output report.html + +# Generate HTML report for all recent sessions +fabric html --since 1h --output dashboard.html + +# Stream logs in terminal +fabric logs --follow +fabric logs --worker --level error + +# Export raw data +fabric export --format json --output data.json +``` + +## Integration with NEEDLE + +FABRIC reads from NEEDLE's output, requiring: + +1. **Log Format Agreement**: NEEDLE outputs structured JSON logs +2. **Telemetry Events**: NEEDLE emits timing/metric events +3. **Session Boundaries**: Clear start/end markers for worker sessions + +### Expected Log Format +```json +{"ts":1709337600,"worker":"w-abc123","level":"info","msg":"Starting task","task":"Process bead bd-xyz"} +{"ts":1709337601,"worker":"w-abc123","level":"debug","msg":"Tool call","tool":"Read","args":{"path":"/src/main.ts"}} +{"ts":1709337605,"worker":"w-abc123","level":"info","msg":"Task complete","duration_ms":5000} +``` + +## Deployment ### Development -- Run locally alongside NEEDLE -- Watch local `.beads/` directory -- Serve dashboard on `localhost:3000` +```bash +# Run TUI alongside NEEDLE +fabric tui --source ~/.needle/logs/ -### Production (Kubernetes) -- Deploy as sidecar to NEEDLE workers -- Aggregate data from multiple workers -- Expose dashboard via Ingress -- Store metrics in centralized DB +# Generate HTML report +fabric html --source ~/.needle/logs/ --output ./reports/ +``` + +### Production +- Sidecar container reading NEEDLE worker logs +- Periodic HTML report generation to shared storage +- Optional: hosted dashboard with real-time WebSocket updates ## Success Metrics -FABRIC will be successful if it: -1. **Reduces debugging time**: Find problematic beads in <30 seconds -2. **Improves visibility**: All stakeholders can see workflow status -3. **Enables optimization**: Identify and fix bottlenecks based on data -4. **Scales efficiently**: Handle 1000+ beads without performance degradation +1. **Immediate insight**: See worker status within 1 second of running `fabric tui` +2. **Log accessibility**: Find relevant log entries in <10 seconds +3. **Shareable reports**: Generate HTML that works offline, no dependencies +4. **Low overhead**: <5% CPU impact when monitoring NEEDLE workers ## Next Steps -1. **Prototype CLI tool**: Validate JSONL parsing and basic queries -2. **Design data schema**: Finalize storage format for metrics -3. **Build API**: Implement core endpoints for querying state -4. **Create dashboard mockups**: Define UX before implementation -5. **Implement MVP**: Phase 1 features with simple web UI +1. **Define log format contract**: Specify what NEEDLE must output +2. **Prototype TUI**: Basic worker list with status +3. **Prototype HTML**: Static page from sample logs +4. **Integrate with NEEDLE**: Connect to actual worker output +5. **Iterate on UX**: Refine based on real usage ## Open Questions -- Should FABRIC store historical data indefinitely or have retention policies? -- How should we handle multi-workspace scenarios (multiple NEEDLE instances)? -- What's the desired latency for real-time updates (1s, 5s, 10s)? -- Should FABRIC be read-only or allow workflow control (pause, retry, cancel)? +- Where does NEEDLE write logs? (stdout, files, both?) +- What telemetry does NEEDLE currently emit? +- Should FABRIC support multiple NEEDLE instances? +- Retention: how much history should FABRIC keep accessible? ---