jedarden 455da572a8 feat(retention): add systemd timer for automatic NEEDLE log pruning

Add systemd timer and service for daily log pruning at 03:00 UTC. Includes
manual prune API endpoint, setup script, and updated documentation.

## Changes
- Add `fabric-prune.service` - systemd oneshot service for log pruning
- Add `fabric-prune.timer` - daily timer (03:00 UTC) with persistent=true
- Add `POST /api/retention/prune` - manual prune trigger with auth
- Add `scripts/setup-fabric-prune.sh` - one-shot timer installer
- Update `CLAUDE.md` - document retention policy and usage

## Retention Policy
- `archiveAfterDays: 3` - files older than 3d → archive/
- `maxAgeDays: 7` - files older than 7d → delete (safety net)
- `archiveRetentionDays: 30` - archives older than 30d → delete

## Integration
- Emits `mend.logs_pruned` events to `fabric-mend.jsonl`
- FABRIC DirectoryTailer auto-discovers events
- `/api/retention` endpoint shows current state and last prune

Resolves bd-ch6.2

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-04-30 16:22:16 -04:00

3.4 KiB

Raw Blame History

FABRIC

What This Is

FABRIC is a live dashboard for NEEDLE worker activity — TUI and web modes.

Repo: /home/coding/FABRIC
Stack: TypeScript + Node.js, Express, WebSocket, React frontend (Vite), blessed TUI
Log source: ~/.needle/logs/ (per-worker JSONL files, hot-added via DirectoryTailer)

Running Service

systemctl --user status fabric-web.service   # check status
systemctl --user restart fabric-web.service  # restart

The service runs as fabric web --port 3000 --source ~/.needle/logs --otlp-http :4318. Auth token is loaded from ~/.config/fabric/secrets.env (FABRIC_AUTH_TOKEN).

Remote Access

URL	Notes
`http://localhost:3000`	Local only
`https://hetzner-ex44.tail1b1987.ts.net/`	Tailscale tailnet, TLS, no public internet

The Tailscale HTTPS proxy is configured via tailscale serve --bg http://localhost:3000. To re-apply after a reset: ./scripts/setup-tailscale-serve.sh.

Auth Model

FABRIC_AUTH_TOKEN in ~/.config/fabric/secrets.env protects all POST endpoints
GET endpoints (dashboard UI, workers, events read) are open — read-only, no secret data
Tailscale provides network-level access control (tailnet membership required)

Log Retention Policy

FABRIC automatically manages NEEDLE log file retention to prevent unbounded growth.

Policy

Setting	Default	Description
`archiveAfterDays`	3	Files older than this are archived to `~/.needle/logs/archive/`
`maxAgeDays`	7	Files older than this are deleted (even if not archived)
`archiveRetentionDays`	30	Archive tarballs older than this are deleted

Automatic Execution

A systemd timer runs pruning daily at 03:00 UTC:

systemctl --user status fabric-prune.timer   # check timer status
systemctl --user start fabric-prune.timer    # enable automatic pruning
journalctl --user -u fabric-prune.service    # view prune logs

Manual Pruning

# CLI
fabric prune --dry-run                    # preview what would be pruned
fabric prune --archive-after 7            # customize policy

# API (requires auth)
curl -X POST http://localhost:3000/api/retention/prune \
  -H "Authorization: Bearer $FABRIC_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"dryRun": true}'

Retention State API

curl http://localhost:3000/api/retention

Returns current file count, archive size, and last prune results.

Build & Test

npm run build           # compile TypeScript + Vite frontend
npm test                # vitest unit tests
npm run test:e2e        # Playwright E2E tests
npx tsc --noEmit        # type-check without emitting

Key Files

Path	Purpose
`src/cli.ts`	Entry point; all CLI commands
`src/web/server.ts`	Express HTTP server + WebSocket + auth middleware
`src/web/frontend/`	React SPA (Vite build)
`src/directoryTailer.ts`	Watches `~/.needle/logs/`, hot-adds new JSONL files
`src/store.ts`	In-memory event store + SQLite persistence
`src/logPruner.ts`	Log retention policy implementation
`scripts/fabric-web.service`	systemd unit file for web dashboard
`scripts/fabric-prune.service`	systemd unit file for log pruning
`scripts/fabric-prune.timer`	systemd timer for daily log pruning
`scripts/setup-tailscale-serve.sh`	One-time Tailscale Serve setup
`docs/plan.md`	Full architecture and phase roadmap

3.4 KiB Raw Blame History