docs: simplify FABRIC to stateless parser/renderer
Remove storage, metrics accumulation, databases, and retention concerns. FABRIC just reads NEEDLE output, parses it, and renders TUI or HTML. - Add clear data flow diagram - Define explicit non-goals - Simplify architecture to reader → parser → renderer - Add piping examples (needle | fabric tui) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jeda
parent
f164e7e253
commit
25ddfb6733
1 changed files with 129 additions and 173 deletions
302
docs/plan.md
302
docs/plan.md
|
|
@ -4,214 +4,170 @@
|
|||
|
||||
## Overview
|
||||
|
||||
FABRIC provides visualization of NEEDLE worker activity by consuming logging and telemetry output. It generates both TUI (terminal) and HTML visualizations for reviewing worker execution patterns, performance, and output.
|
||||
FABRIC parses NEEDLE worker output and renders it as TUI or HTML visualizations. No storage layer - it reads logs directly and renders on demand.
|
||||
|
||||
## Goals
|
||||
|
||||
1. **Worker Visibility**: Surface what NEEDLE workers are doing in real-time
|
||||
2. **Log Aggregation**: Collect and present worker logging output in digestible formats
|
||||
3. **Telemetry Analysis**: Visualize performance metrics, execution timelines, and resource usage
|
||||
4. **Dual Output**: Generate both TUI dashboards for terminal users and HTML reports for browser review
|
||||
1. **Parse**: Read and understand NEEDLE's logging/telemetry output format
|
||||
2. **Render TUI**: Terminal dashboard for real-time monitoring
|
||||
3. **Render HTML**: Browser-viewable reports for review and sharing
|
||||
|
||||
## Data Sources
|
||||
## Data Flow
|
||||
|
||||
FABRIC consumes NEEDLE's logging and telemetry output:
|
||||
```
|
||||
NEEDLE Workers → stdout/log files → FABRIC Parser → TUI or HTML Renderer
|
||||
```
|
||||
|
||||
### Primary Sources
|
||||
- **Worker stdout/stderr**: Captured execution output
|
||||
- **Structured logs**: JSON-formatted log events from workers
|
||||
- **Telemetry streams**: Timing, resource usage, API call metrics
|
||||
- **Session transcripts**: Worker conversation/execution history
|
||||
FABRIC is stateless - it reads, parses, and renders. That's it.
|
||||
|
||||
### Data Model
|
||||
## Input: NEEDLE Output Format
|
||||
|
||||
FABRIC expects structured log lines from NEEDLE workers:
|
||||
|
||||
```json
|
||||
{"ts":1709337600,"worker":"w-abc123","level":"info","msg":"Starting task","task":"bd-xyz"}
|
||||
{"ts":1709337601,"worker":"w-abc123","level":"debug","msg":"Tool call","tool":"Read","path":"/src/main.ts"}
|
||||
{"ts":1709337605,"worker":"w-abc123","level":"info","msg":"Task complete","duration_ms":5000}
|
||||
```
|
||||
|
||||
### Parsed Structure
|
||||
```typescript
|
||||
interface WorkerEvent {
|
||||
worker_id: string;
|
||||
session_id: string;
|
||||
timestamp: number;
|
||||
event_type: 'log' | 'metric' | 'state_change' | 'tool_call' | 'error';
|
||||
level?: 'debug' | 'info' | 'warn' | 'error';
|
||||
message?: string;
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
|
||||
interface WorkerSession {
|
||||
worker_id: string;
|
||||
session_id: string;
|
||||
started_at: number;
|
||||
ended_at?: number;
|
||||
status: 'running' | 'completed' | 'failed' | 'idle';
|
||||
task_description?: string;
|
||||
events: WorkerEvent[];
|
||||
metrics: WorkerMetrics;
|
||||
}
|
||||
|
||||
interface WorkerMetrics {
|
||||
api_calls: number;
|
||||
tool_invocations: number;
|
||||
tokens_in: number;
|
||||
tokens_out: number;
|
||||
duration_ms: number;
|
||||
errors: number;
|
||||
interface LogEvent {
|
||||
ts: number;
|
||||
worker: string;
|
||||
level: 'debug' | 'info' | 'warn' | 'error';
|
||||
msg: string;
|
||||
[key: string]: any; // Additional fields
|
||||
}
|
||||
```
|
||||
|
||||
## Architecture Components
|
||||
## Output: Visualizations
|
||||
|
||||
### 1. Log Collector
|
||||
- **Stream Reader**: Consume NEEDLE worker output streams
|
||||
- **Log Parser**: Extract structured data from log lines
|
||||
- **Event Normalizer**: Convert various log formats to unified schema
|
||||
### TUI Mode (`fabric tui`)
|
||||
|
||||
### 2. Telemetry Aggregator
|
||||
- **Metrics Accumulator**: Track counters, gauges, histograms
|
||||
- **Timeline Builder**: Construct execution timelines per worker
|
||||
- **Session Tracker**: Group events by worker session
|
||||
Terminal dashboard showing:
|
||||
- **Worker List**: Active workers with current status
|
||||
- **Log Stream**: Live scrolling log output
|
||||
- **Detail View**: Focus on single worker's activity
|
||||
|
||||
### 3. Visualization Renderers
|
||||
Features:
|
||||
- Filter by worker, log level, or search term
|
||||
- Keyboard navigation (j/k scroll, / search, q quit)
|
||||
- Color-coded log levels
|
||||
|
||||
#### TUI Renderer
|
||||
- **Live Dashboard**: Real-time terminal display using blessed/ink/textual
|
||||
- **Log Viewer**: Scrollable, filterable log output
|
||||
- **Worker Grid**: At-a-glance status of all workers
|
||||
- **Detail Pane**: Deep-dive into specific worker sessions
|
||||
### HTML Mode (`fabric html`)
|
||||
|
||||
#### HTML Renderer
|
||||
- **Static Reports**: Self-contained HTML files for sharing/archiving
|
||||
- **Interactive Dashboard**: Browser-based live view
|
||||
- **Timeline Visualization**: Gantt-style execution timelines
|
||||
- **Log Explorer**: Searchable, syntax-highlighted log viewer
|
||||
Static HTML file containing:
|
||||
- **Timeline**: Visual representation of worker activity over time
|
||||
- **Log Viewer**: Formatted, syntax-highlighted logs
|
||||
- **Summary**: Worker count, error count, duration
|
||||
|
||||
### 4. Output Formats
|
||||
- **TUI**: Direct terminal rendering (ncurses/blessed style)
|
||||
- **HTML**: Static files or served via local HTTP
|
||||
- **JSON**: Raw data export for external tools
|
||||
- **Markdown**: Summary reports for documentation
|
||||
|
||||
## Key Features
|
||||
|
||||
### Phase 1: Foundation (MVP)
|
||||
- [ ] Consume NEEDLE worker log streams
|
||||
- [ ] Parse structured JSON log events
|
||||
- [ ] Simple TUI: list active workers with status
|
||||
- [ ] Basic HTML: render session logs as static page
|
||||
|
||||
### Phase 2: TUI Dashboard
|
||||
- [ ] Real-time worker status grid
|
||||
- [ ] Live log streaming with filtering
|
||||
- [ ] Worker detail view (select worker, see history)
|
||||
- [ ] Keyboard navigation and search
|
||||
|
||||
### Phase 3: HTML Visualizations
|
||||
- [ ] Session timeline visualization (Gantt-style)
|
||||
- [ ] Metrics charts (API calls, tokens, duration)
|
||||
- [ ] Searchable log explorer with syntax highlighting
|
||||
- [ ] Export/share capabilities
|
||||
|
||||
### Phase 4: Advanced Analytics
|
||||
- [ ] Cross-session analysis (patterns, trends)
|
||||
- [ ] Error clustering and root cause hints
|
||||
- [ ] Performance regression detection
|
||||
- [ ] Custom dashboard layouts
|
||||
|
||||
## Technology Stack
|
||||
|
||||
### Log Processing
|
||||
- **Node.js streams**: Efficient log consumption
|
||||
- **pino/winston parsers**: Structured log parsing
|
||||
- **RxJS**: Reactive event stream processing
|
||||
|
||||
### TUI Framework Options
|
||||
- **blessed/blessed-contrib**: Feature-rich terminal UI (Node.js)
|
||||
- **ink**: React for CLI (Node.js)
|
||||
- **textual**: Modern TUI framework (Python)
|
||||
- **bubbletea**: Elegant TUI framework (Go)
|
||||
|
||||
### HTML Generation
|
||||
- **Static**: Generate self-contained HTML files with embedded CSS/JS
|
||||
- **Templates**: Handlebars/EJS for report generation
|
||||
- **Charts**: Chart.js, Recharts, or Plotly for visualizations
|
||||
- **Timeline**: vis-timeline or custom D3.js
|
||||
|
||||
### Serving (Optional)
|
||||
- **Local HTTP server**: Serve HTML dashboard on localhost
|
||||
- **WebSocket**: Real-time updates to browser
|
||||
Features:
|
||||
- Self-contained (embedded CSS/JS, no external dependencies)
|
||||
- Works offline
|
||||
- Shareable as single file
|
||||
|
||||
## CLI Interface
|
||||
|
||||
```bash
|
||||
# TUI mode - live dashboard
|
||||
fabric tui
|
||||
# TUI - live terminal dashboard
|
||||
fabric tui # Read from stdin or default log path
|
||||
fabric tui --source ~/.needle/logs/ # Read from specific path
|
||||
fabric tui --worker w-abc123 # Filter to one worker
|
||||
|
||||
# Watch specific worker
|
||||
fabric tui --worker <worker-id>
|
||||
# HTML - generate static report
|
||||
fabric html # Output to stdout
|
||||
fabric html --output report.html # Output to file
|
||||
fabric html --source session.log # From specific log file
|
||||
|
||||
# Generate HTML report for session
|
||||
fabric html --session <session-id> --output report.html
|
||||
|
||||
# Generate HTML report for all recent sessions
|
||||
fabric html --since 1h --output dashboard.html
|
||||
|
||||
# Stream logs in terminal
|
||||
fabric logs --follow
|
||||
fabric logs --worker <worker-id> --level error
|
||||
|
||||
# Export raw data
|
||||
fabric export --format json --output data.json
|
||||
# Simple log viewing (parsed + formatted)
|
||||
fabric logs # Pretty-print parsed logs
|
||||
fabric logs --level error # Filter by level
|
||||
fabric logs --worker w-abc123 # Filter by worker
|
||||
```
|
||||
|
||||
## Integration with NEEDLE
|
||||
## Architecture
|
||||
|
||||
FABRIC reads from NEEDLE's output, requiring:
|
||||
|
||||
1. **Log Format Agreement**: NEEDLE outputs structured JSON logs
|
||||
2. **Telemetry Events**: NEEDLE emits timing/metric events
|
||||
3. **Session Boundaries**: Clear start/end markers for worker sessions
|
||||
|
||||
### Expected Log Format
|
||||
```json
|
||||
{"ts":1709337600,"worker":"w-abc123","level":"info","msg":"Starting task","task":"Process bead bd-xyz"}
|
||||
{"ts":1709337601,"worker":"w-abc123","level":"debug","msg":"Tool call","tool":"Read","args":{"path":"/src/main.ts"}}
|
||||
{"ts":1709337605,"worker":"w-abc123","level":"info","msg":"Task complete","duration_ms":5000}
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ FABRIC │
|
||||
├─────────────────────────────────────────────────────┤
|
||||
│ ┌─────────┐ ┌─────────┐ ┌─────────────────┐ │
|
||||
│ │ Input │───▶│ Parser │───▶│ Renderer │ │
|
||||
│ │ Reader │ │ │ │ (TUI or HTML) │ │
|
||||
│ └─────────┘ └─────────┘ └─────────────────┘ │
|
||||
│ │ │ │
|
||||
│ stdin or stdout or │
|
||||
│ log files HTML file │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Deployment
|
||||
### Components
|
||||
|
||||
1. **Input Reader**: Read from stdin, file, or directory of log files
|
||||
2. **Parser**: Parse JSON log lines into structured events
|
||||
3. **Renderer**: Output as TUI dashboard or HTML document
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Core Parser
|
||||
- [ ] Parse NEEDLE JSON log format
|
||||
- [ ] Handle malformed lines gracefully
|
||||
- [ ] Group events by worker
|
||||
|
||||
### Phase 2: TUI Renderer
|
||||
- [ ] Basic worker list display
|
||||
- [ ] Log streaming view
|
||||
- [ ] Filtering and search
|
||||
- [ ] Keyboard controls
|
||||
|
||||
### Phase 3: HTML Renderer
|
||||
- [ ] Generate self-contained HTML
|
||||
- [ ] Timeline visualization
|
||||
- [ ] Formatted log display
|
||||
- [ ] Summary statistics (computed on render, not stored)
|
||||
|
||||
## Technology Options
|
||||
|
||||
### Parser
|
||||
- Node.js with streaming JSON line parser
|
||||
- Go with bufio scanner
|
||||
|
||||
### TUI
|
||||
- **blessed** (Node.js) - Full-featured terminal UI
|
||||
- **ink** (Node.js) - React for CLIs
|
||||
- **bubbletea** (Go) - Elegant TUI framework
|
||||
|
||||
### HTML
|
||||
- Template literals or simple templating
|
||||
- Inline CSS/JS for self-contained output
|
||||
- SVG for timeline visualization
|
||||
|
||||
## Example Usage
|
||||
|
||||
### Development
|
||||
```bash
|
||||
# Run TUI alongside NEEDLE
|
||||
fabric tui --source ~/.needle/logs/
|
||||
# Pipe NEEDLE output directly
|
||||
needle run task.md | fabric tui
|
||||
|
||||
# Generate HTML report
|
||||
fabric html --source ~/.needle/logs/ --output ./reports/
|
||||
# Read from log file
|
||||
fabric tui --source /var/log/needle/session-001.jsonl
|
||||
|
||||
# Generate HTML report after session
|
||||
fabric html --source /var/log/needle/session-001.jsonl --output report.html
|
||||
|
||||
# Quick log review
|
||||
cat session.log | fabric logs --level error
|
||||
```
|
||||
|
||||
### Production
|
||||
- Sidecar container reading NEEDLE worker logs
|
||||
- Periodic HTML report generation to shared storage
|
||||
- Optional: hosted dashboard with real-time WebSocket updates
|
||||
## Non-Goals
|
||||
|
||||
## Success Metrics
|
||||
- ❌ Storing metrics or historical data
|
||||
- ❌ Database or persistence layer
|
||||
- ❌ Real-time alerting
|
||||
- ❌ Multi-instance aggregation
|
||||
- ❌ Web server (HTML is static file output)
|
||||
|
||||
1. **Immediate insight**: See worker status within 1 second of running `fabric tui`
|
||||
2. **Log accessibility**: Find relevant log entries in <10 seconds
|
||||
3. **Shareable reports**: Generate HTML that works offline, no dependencies
|
||||
4. **Low overhead**: <5% CPU impact when monitoring NEEDLE workers
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Define log format contract**: Specify what NEEDLE must output
|
||||
2. **Prototype TUI**: Basic worker list with status
|
||||
3. **Prototype HTML**: Static page from sample logs
|
||||
4. **Integrate with NEEDLE**: Connect to actual worker output
|
||||
5. **Iterate on UX**: Refine based on real usage
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Where does NEEDLE write logs? (stdout, files, both?)
|
||||
- What telemetry does NEEDLE currently emit?
|
||||
- Should FABRIC support multiple NEEDLE instances?
|
||||
- Retention: how much history should FABRIC keep accessible?
|
||||
FABRIC is a parser and renderer. NEEDLE handles orchestration and logging; FABRIC makes that output human-readable.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue