diff --git a/README.md b/README.md index 169d496..3f3017a 100644 --- a/README.md +++ b/README.md @@ -2,51 +2,58 @@ **Flow Analysis & Bead Reporting Interface Console** -A visualization system for surfacing NEEDLE worker activity through TUI and HTML dashboards. +A live display for NEEDLE worker activity, available as TUI or web dashboard. ## Purpose -FABRIC consumes logging and telemetry output from NEEDLE workers, transforming raw execution data into reviewable visualizations: +FABRIC tails NEEDLE's logging output and renders it in real-time. It answers: -- **Flow Analysis**: Visualize worker execution timelines and patterns -- **Bead Reporting**: Surface what workers are doing and how they're performing -- **Interface Console**: Both TUI (terminal) and HTML dashboards for review +- What is each worker doing right now? +- What events are happening across all workers? +- Which workers are active, idle, or erroring? -## Output Formats +## Display Modes -### TUI Dashboard -Real-time terminal interface showing: -- Active worker status grid -- Live log streaming with filtering -- Worker detail views and session history -- Keyboard-driven navigation +### TUI (`fabric tui`) -### HTML Reports -Static and interactive browser-based views: -- Session timeline visualizations (Gantt-style) -- Metrics charts (API calls, tokens, duration) -- Searchable log explorer -- Shareable, self-contained reports +Live terminal dashboard: +- Worker status grid +- Scrolling log stream +- Worker detail panel +- Keyboard navigation and filtering + +### Web (`fabric web`) + +Live browser dashboard at `localhost:3000`: +- Worker overview cards +- Real-time activity feed +- Timeline visualization +- WebSocket-powered updates + +## Quick Start + +```bash +# Terminal dashboard +fabric tui + +# Web dashboard +fabric web +``` + +FABRIC reads from `~/.needle/logs/` by default. ## Relationship to NEEDLE -NEEDLE orchestrates workers; FABRIC surfaces their activity: -- What is each worker currently doing? -- How long are tasks taking? -- What errors are occurring? -- What's the API/token usage? -- What does the execution timeline look like? +``` +NEEDLE (orchestrates workers) β†’ logs β†’ FABRIC (displays activity) +``` + +NEEDLE does the work. FABRIC shows you what's happening. ## Status 🚧 **In Development** - See [docs/plan.md](docs/plan.md) for implementation roadmap. -## Getting Started - -_(Coming soon)_ - ## Documentation - [Implementation Plan](docs/plan.md) -- [Architecture](docs/architecture.md) _(coming soon)_ -- [API Reference](docs/api.md) _(coming soon)_ diff --git a/docs/plan.md b/docs/plan.md index 2eee98a..05fc43a 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -4,25 +4,25 @@ ## Overview -FABRIC parses NEEDLE worker output and renders it as TUI or HTML visualizations. No storage layer - it reads logs directly and renders on demand. +FABRIC is a live display for NEEDLE worker activity. It parses NEEDLE's logging output and renders it in real-time as either a TUI (terminal) or web dashboard. ## Goals -1. **Parse**: Read and understand NEEDLE's logging/telemetry output format -2. **Render TUI**: Terminal dashboard for real-time monitoring -3. **Render HTML**: Browser-viewable reports for review and sharing +1. **Live Display**: Real-time visualization of NEEDLE worker activity +2. **Dual Interface**: TUI for terminal users, web app for browser users +3. **Stateless**: Reads and displays - no storage or persistence ## Data Flow ``` -NEEDLE Workers β†’ stdout/log files β†’ FABRIC Parser β†’ TUI or HTML Renderer +NEEDLE Workers β†’ ~/.needle/logs/ β†’ FABRIC β†’ Live TUI or Web Dashboard ``` -FABRIC is stateless - it reads, parses, and renders. That's it. +FABRIC continuously tails NEEDLE's output and updates the display in real-time. -## Input: NEEDLE Output Format +## Input: NEEDLE Log Format -FABRIC expects structured log lines from NEEDLE workers: +FABRIC expects structured JSON log lines from NEEDLE: ```json {"ts":1709337600,"worker":"w-abc123","level":"info","msg":"Starting task","task":"bd-xyz"} @@ -41,51 +41,6 @@ interface LogEvent { } ``` -## Output: Visualizations - -### TUI Mode (`fabric tui`) - -Terminal dashboard showing: -- **Worker List**: Active workers with current status -- **Log Stream**: Live scrolling log output -- **Detail View**: Focus on single worker's activity - -Features: -- Filter by worker, log level, or search term -- Keyboard navigation (j/k scroll, / search, q quit) -- Color-coded log levels - -### HTML Mode (`fabric html`) - -Static HTML file containing: -- **Timeline**: Visual representation of worker activity over time -- **Log Viewer**: Formatted, syntax-highlighted logs -- **Summary**: Worker count, error count, duration - -Features: -- Self-contained (embedded CSS/JS, no external dependencies) -- Works offline -- Shareable as single file - -## CLI Interface - -```bash -# TUI - live terminal dashboard -fabric tui # Read from ~/.needle/logs/ -fabric tui --source /path/to/logs/ # Read from specific path -fabric tui --worker w-abc123 # Filter to one worker - -# HTML - generate static report -fabric html # Output to stdout -fabric html --output report.html # Output to file -fabric html --source ~/.needle/logs/ # Explicit source path - -# Simple log viewing (parsed + formatted) -fabric logs # Pretty-print parsed logs -fabric logs --level error # Filter by level -fabric logs --worker w-abc123 # Filter by worker -``` - ## Default Source FABRIC reads from `~/.needle/logs/` by default. NEEDLE's folder structure: @@ -99,94 +54,136 @@ FABRIC reads from `~/.needle/logs/` by default. NEEDLE's folder structure: └── README.md ``` -The exact log file format within `logs/` will be determined by NEEDLE's output behavior. +## Output: Live Displays + +### TUI Mode (`fabric tui`) + +Live terminal dashboard that continuously updates: + +- **Worker Grid**: Real-time status of all active workers +- **Log Stream**: Scrolling log output as events arrive +- **Detail Panel**: Focus on a specific worker's activity + +Features: +- Auto-updates as new log events arrive +- Filter by worker, log level, or search term +- Keyboard navigation (j/k scroll, / search, Tab switch panels, q quit) +- Color-coded log levels + +### Web Mode (`fabric web`) + +Live browser dashboard served on localhost: + +- **Worker Overview**: Cards showing each worker's current state +- **Activity Feed**: Real-time log stream +- **Timeline**: Visual representation of worker activity + +Features: +- WebSocket-powered real-time updates +- Filter and search controls +- Responsive layout +- Auto-reconnect on connection loss + +## CLI Interface + +```bash +# TUI - live terminal dashboard +fabric tui # Read from ~/.needle/logs/ +fabric tui --source /path/to/logs/ # Read from specific path +fabric tui --worker w-abc123 # Filter to one worker + +# Web - live browser dashboard +fabric web # Serve on http://localhost:3000 +fabric web --port 8080 # Custom port +fabric web --source /path/to/logs/ # Read from specific path + +# Simple log streaming (parsed + formatted, also live) +fabric logs # Stream parsed logs to stdout +fabric logs --level error # Filter by level +fabric logs --worker w-abc123 # Filter by worker +``` ## Architecture ``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ FABRIC β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Input │───▢│ Parser │───▢│ Renderer β”‚ β”‚ -β”‚ β”‚ Reader β”‚ β”‚ β”‚ β”‚ (TUI or HTML) β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ β”‚ β”‚ -β”‚ stdin or stdout or β”‚ -β”‚ log files HTML file β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ FABRIC β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Log │───▢│ Parser │───▢│ Display Renderer β”‚ β”‚ +β”‚ β”‚ Tailer β”‚ β”‚ β”‚ β”‚ (TUI or Web) β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β”‚ β”‚ β”‚ β”‚ +β”‚ ~/.needle/logs/ Terminal or β”‚ +β”‚ (tail -f style) localhost:3000 β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Components -1. **Input Reader**: Read from stdin, file, or directory of log files +1. **Log Tailer**: Continuously read new lines from log files (like `tail -f`) 2. **Parser**: Parse JSON log lines into structured events -3. **Renderer**: Output as TUI dashboard or HTML document +3. **Display Renderer**: Update TUI or push to WebSocket clients ## Implementation Phases -### Phase 1: Core Parser -- [ ] Parse NEEDLE JSON log format -- [ ] Handle malformed lines gracefully -- [ ] Group events by worker +### Phase 1: Core Infrastructure +- [ ] Log tailer that watches `~/.needle/logs/` +- [ ] JSON line parser +- [ ] Event emitter for parsed events -### Phase 2: TUI Renderer -- [ ] Basic worker list display -- [ ] Log streaming view -- [ ] Filtering and search -- [ ] Keyboard controls +### Phase 2: TUI Display +- [ ] Worker list panel +- [ ] Live log stream panel +- [ ] Worker detail panel +- [ ] Keyboard controls and filtering -### Phase 3: HTML Renderer -- [ ] Generate self-contained HTML -- [ ] Timeline visualization -- [ ] Formatted log display -- [ ] Summary statistics (computed on render, not stored) +### Phase 3: Web Display +- [ ] HTTP server with WebSocket support +- [ ] Real-time event streaming to browser +- [ ] React/Svelte dashboard UI +- [ ] Worker cards and activity feed ## Technology Options -### Parser -- Node.js with streaming JSON line parser -- Go with bufio scanner +### Log Tailer +- Node.js `fs.watch` + readline +- Go `fsnotify` + bufio +- `tail` module (Node.js) ### TUI - **blessed** (Node.js) - Full-featured terminal UI - **ink** (Node.js) - React for CLIs - **bubbletea** (Go) - Elegant TUI framework -### HTML -- Template literals or simple templating -- Inline CSS/JS for self-contained output -- SVG for timeline visualization +### Web +- **Express + ws** (Node.js) - Simple HTTP + WebSocket +- **Fastify** (Node.js) - Fast HTTP server +- **Fiber** (Go) - Express-style for Go +- **React/Svelte** - Frontend framework ## Example Usage ```bash -# Run from workspace with .needle/ folder -cd /path/to/workspace +# Start TUI dashboard fabric tui -# Pipe NEEDLE output directly -needle run task.md | fabric tui +# Start web dashboard +fabric web +# Then open http://localhost:3000 in browser -# Generate HTML report from .needle/ artifacts -fabric html --output report.html - -# Quick log review -fabric logs --level error - -# Review specific worker -fabric logs --worker w-abc123 +# Stream logs to terminal (simple mode) +fabric logs --follow ``` ## Non-Goals +- ❌ Static report generation - ❌ Storing metrics or historical data - ❌ Database or persistence layer -- ❌ Real-time alerting -- ❌ Multi-instance aggregation -- ❌ Web server (HTML is static file output) +- ❌ Alerting system -FABRIC is a parser and renderer. NEEDLE handles orchestration and logging; FABRIC makes that output human-readable. +FABRIC is a live display. It shows what NEEDLE is doing right now. ---