Port TUI ErrorGroupPanel to React — groups errors by signature with
occurrence count, affected workers, time span, severity badges, and
expandable detail cards. Links to similar past errors from fabric.db
error_history via /api/errors/history/similar endpoint.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add OTLP metric ingestion pipeline that persists Sum/Histogram/Gauge
data points to fabric.db with canonical instrument names:
- MetricAccumulator in workerAnalytics.ts accumulates per-worker
running totals for tokens, cost, durations, bead counts
- Schema v2 in historicalStore.ts adds metric_samples and
session_worker_summaries tables with source preference tracking
- flushMetricSamples() in store.ts drains accumulator → SQLite on
every event, upserting session summaries and live session rows
- docs/schema.md documents instrument names and resolution order
Fix source preference ordering (otlp-metric > otlp-span > log-derived)
using CASE-based SQL sort instead of alphabetical ASC which was
inverted. Fix metricsSource detection to not require costUsd > 0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace coarse NeedleWorkerStatus ('idle'|'executing'|'draining'|'starting')
with the real NEEDLE state machine: BOOTING→SELECTING→CLAIMING→WORKING→CLOSING→STOPPED.
- Add NeedleState type, VALID_TRANSITIONS map, needleStateToStatus() helper in types.ts
- Track needleState + lastStateTransition per worker by consuming worker.state_transition events
- Surface all six states in TUI worker cards (WorkerGrid, WorkerDetail) with per-state icons/colors
- Surface all six states in web WorkerGrid.tsx with NEEDLE_STATE_LABELS and NEEDLE_STATE_COLORS
- Add getNeedleStateColor/getNeedleStateIcon to colors.ts
- Rewire stuck detection to fire on state-transition gaps (state_gap pattern in stuckDetection.ts)
- Add sequence-based event ordering via compareEventsBySequence and queryOrdered()
- Legacy event-type fallback preserved for workers not emitting state_transition events
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds Budget panel (B key) to TUI with per-bead/per-worker cost tracking,
EMA-smoothed burn rate, time-series storage, and CostDashboard web component.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
persistSession() still used msg.includes('completed'/'finished'/'closed') to
find bead completion events. With NEEDLE format, msg is the event type string
(e.g. 'bead.completed'), so substring matching can misfire on
'bead.agent_completed'. Match on exact event types 'bead.completed' and
'bead.failed' instead — consistent with the fix already applied to
updateWorkerInfo() and trackTaskForHistory() in bd-vqd.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements persistent storage for historical session analytics:
- Add HistoricalStore class using better-sqlite3
- Schema includes sessions, task_metrics, and error_history tables
- Session management: start/end sessions with worker/task/cost metrics
- Task metrics: record per-bead completion times, costs, tokens
- Error history: track errors with resolution status
- Historical queries: worker comparison across sessions
- Learned recoveries: extract patterns from historical error resolutions
- Integration with EventStore for automatic session persistence
- Historical query methods in WorkerAnalytics
- Enhanced recovery suggestions using historical data in RecoveryManager
- Comprehensive unit tests (20 passing tests)
Database stored at ~/.needle/fabric.db for cross-session persistence.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude Worker <noreply@anthropic.com>
Comprehensive worker performance metrics tracking system:
Features:
- Beads per hour calculation
- Average completion time tracking
- Error rate monitoring
- Cost per bead calculation (USD)
- Idle percentage tracking
- Time-series data storage and retrieval
- Performance trend analysis
- Aggregated analytics across all workers
Implementation:
- Added WorkerAnalytics types to types.ts
- Created workerAnalytics.ts module with full aggregation logic
- Integrated with EventStore for automatic event processing
- 25 comprehensive unit tests (all passing)
- Exported from index.ts for public API access
Technical highlights:
- Automatic activity period detection with 5-minute gap threshold
- Time-series snapshots at configurable intervals (default 1 hour)
- Flexible time window filtering (hour/day/week/all)
- Worker ranking by performance, error rate, and cost efficiency
- Performance trend detection (improving/declining/stable)
Co-Authored-By: Claude Worker <noreply@anthropic.com>
Integrated CrossReferenceManager with EventStore to enable cross-reference
hyperlinking across events, workers, files, and beads. This allows navigation
between related activities in the FABRIC dashboard.
Changes:
- Integrated CrossReferenceManager into InMemoryEventStore
- Added batch processing for cross-reference relationship detection
- Added 11 new API methods to store for cross-reference queries
- Updated web server to use store's cross-reference methods
- Added comprehensive test coverage (11 new tests)
- All 55 tests passing
Features:
- Automatic link detection between events, workers, files, and beads
- Relationship detection (same_bead, same_file, same_worker, temporal_proximity, etc.)
- Navigation path finding between entities
- Cross-reference statistics and queries
- Web API endpoints for cross-reference data
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add recovery playbook feature that suggests recovery actions when workers encounter errors.
Implementation includes:
- RecoverySuggestion, RecoveryAction, RecoveryPlaybookEntry types in types.ts
- recoveryPlaybook.ts utility module with RecoveryManager class
- Predefined playbooks for each error category (network, permission, timeout, etc.)
- Pattern matching to map errors to actionable recovery steps
- Integration with InMemoryEventStore for easy access to suggestions
- RecoveryPanel TUI component for displaying suggestions
- Console formatting utilities for CLI output
Recovery actions include:
- Automated actions (retry, backoff, investigate)
- Manual actions (fix permissions, install dependencies)
- Escalation to human when automated recovery not possible
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement file heatmap visualization that tracks which files are modified
most frequently and by which workers. Helps identify hotspots and potential
collision areas.
Features:
- Track file modifications across all workers
- Heat levels (cold/warm/hot/critical) based on modification frequency
- Worker contribution percentages per file
- Collision risk detection for files with multiple workers
- Sortable by modifications, recent activity, workers, or collisions
- Filter by directory or collision-only files
- Statistics overview with heat distribution
Integration:
- Press 'H' in TUI to toggle heatmap view
- Press 's' to cycle sort modes
- Press 'c' to toggle collision-only filter
- Press 'Esc' to return to default view
Also fixed pre-existing DependencyDag component build issues:
- Created missing dagUtils.ts utility module
- Fixed import paths and type annotations
Tests: 20 new tests for file heatmap, all 154 tests passing
Co-Authored-By: Claude Worker <noreply@anthropic.com>
- Add SessionReplay component with full playback controls
- Add replay command to CLI with filter support
- Fix color type references in SessionReplay
- Add EventFilter type import to CLI
- Add File Heatmap types for future feature
Co-Authored-By: Claude Worker <noreply@anthropic.com>
Implement smart error grouping to cluster similar errors together and reduce
noise in the activity stream. Uses pattern matching on error messages and
stack traces to group related errors.
Features:
- Error categorization into 9 categories (network, permission, validation, etc.)
- Pattern-based fingerprinting for grouping similar errors
- Normalization of variable data (IPs, paths, UUIDs, timestamps)
- Severity calculation based on frequency and recency
- Integration with InMemoryEventStore for automatic error tracking
New files:
- src/errorGrouping.ts - Error grouping module
- src/errorGrouping.test.ts - Comprehensive test suite (34 tests)
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements working `fabric tail` command with:
- Log tailer module (src/tailer.ts)
- File watching with fs.watch
- Follow mode for real-time updates
- Buffer management for partial lines
- Graceful shutdown handling
- JSON parser module (src/parser.ts)
- Validates NEEDLE log format
- Extracts optional fields (tool, path, bead, duration_ms, error)
- Human-readable event formatting with color support
- In-memory event store (src/store.ts)
- Stores events with worker indexing
- Supports querying with filters
- Auto-updates worker status based on events
- Working CLI (src/cli.ts)
- Filters by worker (-w) and level (-l)
- Supports --json output
- Shows existing lines with -n flag
Resolves HUMAN bead bd-17q (worker starvation) by providing
working implementation and creating beads for remaining phases.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
