Commit graph

10 commits

Author SHA1 Message Date
jedarden
ac80d1f765 P3: Phase 3 Task Registry + Persistence — COMPLETE
Completes Phase 3 of the Miroir implementation: the 14-table task-store
schema from plan §4 with both SQLite and Redis backends.

## What Was Done

### 1. SQLite Backend (SqliteTaskStore)
- All 14 tables implemented with CRUD operations
- WAL mode for concurrent access
- Schema version tracking with migration system
- Idempotent migrations (safe to run on every startup)
- Schema version ahead detection (refuses to start if store > binary)

### 2. Redis Backend (RedisTaskStore)
- All 14 tables mapped to Redis keyspace
- Hash per row + index sets for O(cardinality) iteration
- testcontainers-based integration tests
- Leader lease with Redis SET NX/EX semantics
- Pub/Sub for session revocation
- Memory budget test (plan §14.7)

### 3. Schema Migrations
- Migration 1: Core tables (1-7)
- Migration 2: Feature tables (8-14)
- Migration 3: Task registry fields (no-op)

### 4. Tests
- SQLite: 36 tests pass (CRUD, property tests, restart resilience)
- Redis: Comprehensive integration tests (testcontainers)
- Helm validation: multi-replica requires Redis enforced

### 5. Helm Validation
- values.schema.json enforces redis + multi-replica constraint
- Test cases verify lint behavior (pass/fail as expected)

## Definition of Done — VERIFIED 

- rusqlite-backed store initializing every table idempotently
- Redis-backed store mirrors the same API (TaskStore trait)
- Migrations/versioning with schema version tracking
- Property tests on SQLite backend
- Integration test: restart resilience
- Redis-backend integration test (testcontainers)
- miroir:tasks:_index-style iteration for list endpoints
- taskStore.backend: redis + replicas > 1 enforced by Helm
- Plan §14.7 Redis memory accounting validated

## Files

- crates/miroir-core/src/task_store/mod.rs — TaskStore trait
- crates/miroir-core/src/task_store/sqlite.rs — SQLite impl
- crates/miroir-core/src/task_store/redis.rs — Redis impl
- crates/miroir-core/src/schema_migrations.rs — Migration registry
- crates/miroir-core/src/migrations/*.sql — Migration files
- charts/miroir/values.schema.json — Helm validation
- charts/miroir/tests/*.yaml — Test cases
- notes/miroir-r3j-phase3-completion.md — Completion notes

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 16:50:42 -04:00
jedarden
6e35e420a9 P10.3: SEARCH_UI_JWT_SECRET dual-secret overlap rotation
Implement plan §9 JWT signing-secret rotation with zero-downtime dual-secret
overlap window. Primary secret signs new tokens (kid header identifies it),
optional previous secret validates old tokens during rotation. Validation tries
primary first, falls through to previous on signature mismatch, and propagates
Expired immediately when the correct secret is found.

Key pieces:
- auth.rs: dual-secret JWT validation with kid header, leak response via empty
  previous, full test coverage (62 tests including e2e rotation scenario)
- main.rs: read SEARCH_UI_JWT_SECRET_PREVIOUS, refuse startup without primary
- config: jwt_secret_previous_env + jwt_rotation_buffer_s in SearchUiAuthConfig
- miroir-ctl: rotate-jwt-secret command (5-step dual-secret overlap procedure)
- Helm CronJob: quarterly schedule, suspended by default, Forbid concurrency

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 16:17:33 -04:00
jedarden
26fe2970fc P10.2: nodeMasterKey zero-downtime rotation flow
Add `miroir-ctl key rotate-node-master` command implementing plan §9
4-step zero-downtime rotation: create new admin-scoped key on all
Meilisearch nodes, print K8s Secret update instructions, wait for
rolling restart confirmation, delete old key. Supports --dry-run,
node auto-discovery via topology API, and rollback on step 1 failure.

Add `address` field to topology API NodeInfo for CLI node discovery.
Add runbooks for both nodeMasterKey (zero-downtime) and startup master
key (maintenance window required) rotation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 15:49:40 -04:00
jedarden
7a8742375b P2.6: Complete Phase 2 DoD — dedup, live topology, field stripping, all 14 tests pass
- merger: deduplicate hits by primary key when multiple shards map to same node
- search: use shared AppState with live topology from health checker
- search: strip _miroir_shard always, _rankingScore only when not requested
- search: include facetDistribution only when facets were requested
- credentials: add mutex guards for env-var test isolation
- Add Phase 2 DoD integration tests: shard coverage, dedup, facets, paging,
  degraded writes, error shape parity, topology shape, auth errors, reserved fields

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 09:29:43 -04:00
jedarden
2b1ea87f3e P0.7: Fix cargo fmt and clippy warnings for CI smoke
cargo fmt reformats dump.rs match arms; credentials.rs needs #[allow(dead_code)]
on an unused public helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 22:06:56 -04:00
jedarden
e47c1c2f73 P12.OP3: Validate 2× transient load caveat and add CLI schedule window guard
- Add resharding load simulation model with real router hash functions
- Benchmark confirms storage amplification is exactly 2.0× and dual-write
  amplification is exactly 2.0× across all test matrix scenarios (1KB/10GB,
  10KB/100GB, 1MB/1TB), with hash distribution CV < 5% in all cases
- CLI window guard: resharding.allowed_windows config restricts resharding
  to named time windows (e.g. "02:00-06:00 UTC"), CLI refuses outside
  windows without --force
- Integration tests confirm rejection outside window, --force override,
  no-restriction mode, and disabled config handling

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 22:00:57 -04:00
jedarden
188fd5404c P12.OP5: Add dump import compatibility matrix
Enumerates dump variants that streaming mode can/can't handle.

- Added docs/dump-import/compatibility-matrix.md with comprehensive
  compatibility matrix covering Meilisearch versions, dump variants,
  and workarounds
- Added docs/dump-import/README.md as entry point
- Updated miroir-ctl dump command to reference matrix with helpful
  error messages for unimplemented subcommands (import, export, analyze)

Addresses Open Problem #5: identifies what "can't reconstruct" means
in concrete terms, giving operators clear guidance on when broadcast
fallback is needed and what alternatives exist.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 21:06:46 -04:00
jedarden
78e5fe1acb P0.4: Scaffold miroir-ctl crate
Add miroir-ctl management CLI with:
- clap root CLI with admin-key loading (env → credentials file → flag)
- All 15 subcommand stubs from plan §4
- Unit tests for credential loader priority order
- Clear "not yet implemented" messages pointing to tracking bead

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 21:01:11 -04:00
jedarden
601988829d P0.1: Set up Cargo workspace + toolchain pin
- Update workspace Cargo.toml: explicit members list, edition 2021, MIT license, rust-version 1.87
- Simplify workspace.dependencies to core shared deps
- Update member crates to use explicit dependency versions where workspace inheritance was removed

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 20:52:53 -04:00
jedarden
409f952f59 Add repo hygiene: LICENSE, CHANGELOG, .gitignore
- LICENSE: MIT (per plan §12)
- CHANGELOG.md: Keep a Changelog 1.1.0 skeleton with [Unreleased]
  and [0.1.0] sections matching the awk extractor from plan §7
- .gitignore: Rust target/, editor junk; Cargo.lock kept in VCS

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 20:47:36 -04:00