Fix beadsCompleted vs stuck detection metric discrepancy in /api/workers response.
Problem:
- /api/workers returned contradictory data: beadsCompleted=285 (counts bead.released
events) but stuck=true with "only 1 completion(s)" reason
- stuck detection counted a different metric while beadsCompleted counted bead.released
- When all beads timed out and were deferred, beadsCompleted incremented but stuck
detector saw zero success outcomes and flagged the worker as stuck
Solution:
- Separated beadsCompleted (bead.completed events only) from beadsReleased
(bead.released with release_success, includes timed-out/deferred)
- Updated stuck detection to use beadsCompleted for successful completions
- Added beadsReleased counter to track all processed beads (including timeouts)
- Improved stuck reason to distinguish "processed" vs "successful completions"
- Updated evidence to show both metrics for clarity
Now a worker that processes 100 beads (all timed out) will show:
- beadsReleased: 100
- beadsCompleted: 0
- stuckReason: "Running for Xm with 100 processed but 0 successful completions (all timed out/deferred)"
Acceptance criteria met:
- A worker processing 100 timed-out beads shows clearly that it processed 100 but
completed 0 successfully
- The stuck flag fires with accurate reason text
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Modern NEEDLE emits bead.released with reason=release_success instead of
legacy bead.completed. Update store.ts and workerAnalytics.ts to handle
this event: increment beadsCompleted, set status to idle, and clear
activeBead/activeFiles/activeDirectories.
Also fixes workerAnalytics trackBeadEvent to use explicit isReleasedSuccess
check instead of broad msg.includes('success') which could false-positive.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add side-by-side worker comparison analytics to the TUI analytics panel.
Users can now press 'c' to enter comparison mode and view detailed metrics
comparing two workers across performance, error, cost, and efficiency dimensions.
- Add WorkerComparison type with differences, percent differences, and winner per metric
- Add compareWorkers() method to WorkerAnalytics class
- Extend WorkerAnalyticsPanel with comparison view mode
- Add renderComparison() method with formatted comparison rows
- Add keyboard bindings: [c] toggle comparison, [↑/↓] cycle workers, [←/→] swap selection
Related to docs/plan.md Worker Comparison Analytics section.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Port TUI ErrorGroupPanel to React — groups errors by signature with
occurrence count, affected workers, time span, severity badges, and
expandable detail cards. Links to similar past errors from fabric.db
error_history via /api/errors/history/similar endpoint.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add OTLP metric ingestion pipeline that persists Sum/Histogram/Gauge
data points to fabric.db with canonical instrument names:
- MetricAccumulator in workerAnalytics.ts accumulates per-worker
running totals for tokens, cost, durations, bead counts
- Schema v2 in historicalStore.ts adds metric_samples and
session_worker_summaries tables with source preference tracking
- flushMetricSamples() in store.ts drains accumulator → SQLite on
every event, upserting session summaries and live session rows
- docs/schema.md documents instrument names and resolution order
Fix source preference ordering (otlp-metric > otlp-span > log-derived)
using CASE-based SQL sort instead of alphabetical ASC which was
inverted. Fix metricsSource detection to not require costUsd > 0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds Budget panel (B key) to TUI with per-bead/per-worker cost tracking,
EMA-smoothed burn rate, time-series storage, and CostDashboard web component.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements persistent storage for historical session analytics:
- Add HistoricalStore class using better-sqlite3
- Schema includes sessions, task_metrics, and error_history tables
- Session management: start/end sessions with worker/task/cost metrics
- Task metrics: record per-bead completion times, costs, tokens
- Error history: track errors with resolution status
- Historical queries: worker comparison across sessions
- Learned recoveries: extract patterns from historical error resolutions
- Integration with EventStore for automatic session persistence
- Historical query methods in WorkerAnalytics
- Enhanced recovery suggestions using historical data in RecoveryManager
- Comprehensive unit tests (20 passing tests)
Database stored at ~/.needle/fabric.db for cross-session persistence.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude Worker <noreply@anthropic.com>
Comprehensive worker performance metrics tracking system:
Features:
- Beads per hour calculation
- Average completion time tracking
- Error rate monitoring
- Cost per bead calculation (USD)
- Idle percentage tracking
- Time-series data storage and retrieval
- Performance trend analysis
- Aggregated analytics across all workers
Implementation:
- Added WorkerAnalytics types to types.ts
- Created workerAnalytics.ts module with full aggregation logic
- Integrated with EventStore for automatic event processing
- 25 comprehensive unit tests (all passing)
- Exported from index.ts for public API access
Technical highlights:
- Automatic activity period detection with 5-minute gap threshold
- Time-series snapshots at configurable intervals (default 1 hour)
- Flexible time window filtering (hour/day/week/all)
- Worker ranking by performance, error rate, and cost efficiency
- Performance trend detection (improving/declining/stable)
Co-Authored-By: Claude Worker <noreply@anthropic.com>
