jedarden/miroir
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

P0.7: Update plan with chaos-test results, sync beads

Browse source 
Verified CI smoke pipeline runs end-to-end in ~5:39 on iad-ci.
All three checks pass: fmt, clippy, test.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jedarden 2026-04-18 23:03:21 -04:00
parent 2b1ea87f3e
commit c30d867d27
3 changed files with 15 additions and 14 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
.beads/issues.jsonl
View file

@ -61,13 +61,13 @@
{"id":"miroir-qjt.6","title":"P8.6 Release mechanics: CHANGELOG parser, version bumps, tag triggers","description":"## What\n\nWire the full release mechanics per plan §7:\n\n- **CHANGELOG extraction** via the plan §7 awk script:\n ```\n NOTES=$(awk \"/^## \\[${TAG#v}\\]/{found=1; next} found && /^## /{exit} found{print}\" CHANGELOG.md)\n ```\n- **Cargo.toml version sync**: workspace version + Chart.yaml appVersion must both bump before tagging\n- **Tag format**: `v[0-9]+.[0-9]+.[0-9]+*` triggers CI — including pre-release suffixes (`-rc.1`, `-alpha.2`)\n- **Pre-release handling**: no `:latest` or float tags for pre-releases\n- **Release checklist in the repo** (plan §7):\n - [ ] All tests pass on `main`\n - [ ] `CHANGELOG.md` updated with new version section\n - [ ] `Cargo.toml` workspace version bumped\n - [ ] `Chart.yaml` `appVersion` updated\n - [ ] Migration notes written if task store schema changed\n\n## Why\n\nPlan §12 commits to SemVer with backward-compat promises from v1.0. Unstructured release processes make those promises impossible to keep. Automation of version sync + release notes prevents the \"we forgot to update Chart.yaml\" class of error.\n\n## Details\n\n**Version-bump script** (`scripts/bump-version.sh`):\n```bash\n#!/bin/bash\nNEW_VERSION=$1\nsed -i \"s/^version = .*/version = \\\"$NEW_VERSION\\\"/\" Cargo.toml\nsed -i \"s/^version: .*/version: $NEW_VERSION/\" charts/miroir/Chart.yaml\nsed -i \"s/^appVersion: .*/appVersion: $NEW_VERSION/\" charts/miroir/Chart.yaml\n```\n\n**Release PR template**: every release PR includes the checklist from plan §7 and a diff of CHANGELOG.md.\n\n**CI enforcement**: a `release-ready` CI step verifies Cargo workspace version, Chart.yaml appVersion, and the CHANGELOG header all agree on the tag. Runs on every PR that modifies any of those files.\n\n**Chart repo publication** (plan §12):\n- `https://jedarden.github.io/miroir` (gh-pages branch with index.yaml)\n- `ghcr.io/jedarden/charts/miroir` (OCI push from Argo Workflow)\n\n## Acceptance\n\n- [ ] `scripts/bump-version.sh 0.2.0` updates all 3 files atomically\n- [ ] Tagging `v0.2.0` fires the CI release path and produces: GitHub release, ghcr image with 4 tags (`v0.2.0, 0.2, 0, latest`), chart published to gh-pages + OCI\n- [ ] Tagging `v0.2.0-rc.1` produces only the exact tag; no `latest`/float tags\n- [ ] `release-ready` check fails a PR that bumps Cargo but not Chart.yaml","status":"open","priority":1,"issue_type":"task","created_at":"2026-04-18T21:43:57.027884427Z","created_by":"coding","updated_at":"2026-04-18T21:44:01.524162086Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-8"],"dependencies":[{"issue_id":"miroir-qjt.6","depends_on_id":"miroir-qjt","type":"parent-child","created_at":"2026-04-18T21:43:57.027884427Z","created_by":"coding","metadata":"{}","thread_id":""},{"issue_id":"miroir-qjt.6","depends_on_id":"miroir-qjt.4","type":"blocks","created_at":"2026-04-18T21:44:01.524106188Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qjt.7","title":"P8.7 Helm values for CDC PVC, Redis, ESO integration","description":"## What\n\nConditional Helm templates + values for optional capabilities:\n\n1. **`miroir-pvc.yaml`** rendered only when `cdc.buffer.primary == \"pvc\"` OR `cdc.buffer.overflow == \"pvc\"` (plan §13.13). Mounts at `/data/cdc`.\n2. **`redis-deployment.yaml`** rendered when `redis.enabled: true`. Simple single-replica Redis for dev; production operators point `taskStore.url` at a managed Redis.\n3. **ESO `ExternalSecret`** example in `examples/eso-external-secret.yaml` (plan §6 ESO section). Pulls from `kv/search/miroir` in OpenBao via `openbao-backend` ClusterSecretStore.\n\n## Why\n\nPlan §13.13: \"Miroir runs from a `scratch` container image with no writable filesystem by default.\" Without the optional PVC template, operators who enable `cdc.buffer.overflow: pvc` get a silent NPE. Making the template conditional on the config value keeps the non-CDC chart tidy.\n\nPlan §9 ESO integration: pulling secrets from OpenBao (rather than baking into values.yaml) is the standard fleet pattern.\n\n## Details\n\n**PVC template**:\n```yaml\n{{- if or (eq .Values.cdc.buffer.primary \"pvc\") (eq .Values.cdc.buffer.overflow \"pvc\") }}\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n name: {{ include \"miroir.fullname\" . }}-cdc\nspec:\n accessModes: [ReadWriteOnce]\n resources:\n requests:\n storage: {{ .Values.cdc.buffer.pvc_size | default \"10Gi\" }}\n{{- end }}\n```\n\n**Redis values** (chart defaults):\n```yaml\nredis:\n enabled: false\n image: redis:7.4-alpine\n persistence:\n enabled: true\n size: 5Gi\n auth:\n enabled: true\n # password comes from K8s Secret `miroir-redis-secrets` / ESO\n```\n\n**ESO example** (plan §6):\n```yaml\napiVersion: external-secrets.io/v1beta1\nkind: ExternalSecret\nmetadata:\n name: miroir-secrets\nspec:\n refreshInterval: 1h\n secretStoreRef:\n name: openbao-backend\n kind: ClusterSecretStore\n target:\n name: miroir-secrets\n creationPolicy: Owner\n data:\n - secretKey: masterKey\n remoteRef: { key: kv/search/miroir, property: master_key }\n - secretKey: nodeMasterKey\n remoteRef: { key: kv/search/miroir, property: node_master_key }\n - secretKey: adminApiKey\n remoteRef: { key: kv/search/miroir, property: admin_api_key }\n```\n\n## Acceptance\n\n- [ ] With `cdc.buffer.overflow: pvc` → PVC manifest rendered; helm install mounts at /data/cdc\n- [ ] With default values → no PVC manifest rendered\n- [ ] `redis.enabled: true` → redis-deployment.yaml + service rendered; Miroir ConfigMap points `taskStore.url` at it\n- [ ] ESO example deploys cleanly against ardenone-cluster's OpenBao (once v0.x is published)","status":"open","priority":2,"issue_type":"task","created_at":"2026-04-18T21:43:57.059546985Z","created_by":"coding","updated_at":"2026-04-18T21:44:01.551737874Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-8"],"dependencies":[{"issue_id":"miroir-qjt.7","depends_on_id":"miroir-qjt","type":"parent-child","created_at":"2026-04-18T21:43:57.059546985Z","created_by":"coding","metadata":"{}","thread_id":""},{"issue_id":"miroir-qjt.7","depends_on_id":"miroir-qjt.2","type":"blocks","created_at":"2026-04-18T21:44:01.551672128Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon","title":"Phase 0 — Foundation (workspace, crates, config, deps)","description":"## Phase 0 Epic — Foundation\n\nEstablishes the Rust project scaffolding that every subsequent phase builds on. When this phase is done, the repo has a compilable (but non-functional) Cargo workspace with the three crates specified in plan §4 and a fully-typed config struct representing plan §4's YAML schema.\n\n## Why This Phase First\n\nEvery later phase assumes:\n- The crate layout `miroir-core / miroir-proxy / miroir-ctl` exists\n- The `Config` struct and its `validate()` routine can be imported\n- The workspace compiles under a stable Rust toolchain pinned in `rust-toolchain.toml`\n- `cargo test --all` exists and runs (even if empty)\n- CI (Phase 8) targets the same layout\n\nSkipping this phase or deferring \"boring\" bits (deps, lints, musl target) causes expensive backtracking once higher-level work is in flight.\n\n## Scope (plan §4 — Implementation)\n\n- Cargo workspace at repo root\n- `crates/miroir-core` library (routing, merging, topology primitives)\n- `crates/miroir-proxy` HTTP binary (axum server skeleton)\n- `crates/miroir-ctl` CLI binary (clap subcommand skeleton)\n- `rust-toolchain.toml` pinning a stable version compatible with Rust 1.87+ (per CI workflow)\n- Key deps wired: axum, tokio (multi-threaded), reqwest, twox-hash, serde, serde_json, config, rusqlite, prometheus, tracing + tracing-subscriber, clap, uuid\n- `Config` struct mirroring the full YAML schema in plan §4 (even empty defaults for features not yet built)\n- `rustfmt.toml` + `clippy.toml` + `.editorconfig` so style is consistent from commit 1\n- `Cargo.lock` committed (binary crate)\n- `CHANGELOG.md` scaffold (Keep a Changelog format — CI release step extracts sections from this)\n- `LICENSE` (MIT, per §12)\n- `.gitignore`\n\n## Out of Scope\n\n- Actual routing logic (Phase 1)\n- Proxy handlers beyond a `/health` stub (Phase 2)\n- Task registry schema (Phase 3)\n- Anything in §13 (Phase 5)\n\n## Definition of Done\n\n- [ ] `cargo build --all` succeeds\n- [ ] `cargo test --all` succeeds (even with zero tests)\n- [ ] `cargo clippy --all-targets --all-features -- -D warnings` passes\n- [ ] `cargo fmt --all -- --check` passes\n- [ ] `cargo build --release --target x86_64-unknown-linux-musl -p miroir-proxy` succeeds\n- [ ] `Config` round-trips YAML → struct → YAML and matches plan §4 shape\n- [ ] All child beads for this phase are closed","status":"open","priority":0,"issue_type":"epic","created_at":"2026-04-18T21:18:33.116054928Z","created_by":"coding","updated_at":"2026-04-18T21:18:33.116054928Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase","phase-0"]}
{"id":"miroir-qon.1","title":"P0.1 Set up Cargo workspace + toolchain pin","description":"## What\n\nCreate the root Cargo workspace (`Cargo.toml` with `[workspace]` members), pin the Rust toolchain (`rust-toolchain.toml`), and add lint config (`rustfmt.toml`, `clippy.toml`, `.editorconfig`).\n\n## Why\n\nEverything else compiles against this. A pinned toolchain prevents \"works on my machine\" drift across contributors + CI (`rust:1.87-slim` per plan §7). Lint config in the repo from day 1 means we never have to retrofit formatting.\n\n## Details\n\n**Cargo.toml (workspace root):**\n```toml\n[workspace]\nresolver = \"2\"\nmembers = [\"crates/miroir-core\", \"crates/miroir-proxy\", \"crates/miroir-ctl\"]\n\n[workspace.package]\nversion = \"0.1.0\"\nedition = \"2021\"\nlicense = \"MIT\"\nrepository = \"https://github.com/jedarden/miroir\"\nrust-version = \"1.87\"\n```\n\n**rust-toolchain.toml:**\n```toml\n[toolchain]\nchannel = \"1.87\"\ncomponents = [\"rustfmt\", \"clippy\"]\ntargets = [\"x86_64-unknown-linux-musl\"]\n```\n\n**rustfmt.toml:** conservative default; `max_width = 100`, `edition = \"2021\"`.\n\n**clippy.toml:** empty for now; the `-D warnings` enforcement lives in CI (plan §7 `cargo-lint` template).\n\n## Acceptance\n\n- [ ] `cargo build` succeeds on an empty workspace (no members are complete yet but the workspace file parses)\n- [ ] `rustup show` in CI confirms the pinned channel\n- [ ] `cargo fmt --all -- --check` is a no-op (no files to check yet)\n- [ ] `cargo clippy --all-targets -- -D warnings` is a no-op","status":"in_progress","priority":0,"issue_type":"task","assignee":"alpha","created_at":"2026-04-18T21:24:25.694504043Z","created_by":"coding","updated_at":"2026-04-19T00:45:49.271644520Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.1","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.694504043Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.2","title":"P0.2 Scaffold miroir-core crate","description":"## What\n\nCreate `crates/miroir-core/` with the module skeleton from plan §4:\n- `src/lib.rs` — public re-exports\n- `src/router.rs` — rendezvous hash primitives (signatures only; implementation in Phase 1)\n- `src/topology.rs` — `Topology`, `Group`, `Node`, `NodeId`, `NodeStatus` types\n- `src/scatter.rs` — scatter orchestration trait/stubs\n- `src/merger.rs` — result merge trait/stubs\n- `src/task.rs` — task registry trait/stubs\n- `src/config.rs` — `Config` struct (full shape matching plan §4 YAML)\n- `src/error.rs` — `MiroirError` enum + `Result<T>` alias\n\n## Why\n\nThe module boundary is intentional: pure library vs. binaries. `miroir-core` must stay dependency-light (no HTTP server, no CLI crate) so both binaries and downstream users can depend on it cleanly. This is also where the coverage gate (≥ 90%) applies per plan §8 coverage policy.\n\n## Details\n\n- Crate-type: `lib` (default); no `[[bin]]`\n- `Cargo.toml` deps: `serde`, `serde_json`, `twox-hash`, `thiserror`, `tracing` (minimal set — concrete feature-specific deps added as they're needed)\n- Public API starts small — add `pub use` entries to `lib.rs` only as modules are completed\n\n## Acceptance\n\n- [ ] `cargo build -p miroir-core` succeeds with empty stubs\n- [ ] `cargo doc -p miroir-core` produces rustdoc without warnings\n- [ ] `cargo test -p miroir-core` runs (zero tests) successfully","status":"in_progress","priority":0,"issue_type":"task","assignee":"bravo","created_at":"2026-04-18T21:24:25.717048243Z","created_by":"coding","updated_at":"2026-04-19T00:45:52.298859153Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.2","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.717048243Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.3","title":"P0.3 Scaffold miroir-proxy crate","description":"## What\n\nCreate `crates/miroir-proxy/` — the HTTP proxy binary. Module layout from plan §4:\n- `src/main.rs` — startup (load config, init logging, start axum server, install signal handlers)\n- `src/routes/documents.rs`, `search.rs`, `indexes.rs`, `settings.rs`, `tasks.rs`, `health.rs`, `admin.rs` — route handler stubs\n- `src/auth.rs` — bearer-token dispatch per plan §5 (stubbed; real logic in Phase 2)\n- `src/middleware.rs` — tracing/logging + Prometheus middleware stubs\n\n## Why\n\nThis is the thing users install. Separating route modules by concern makes the bearer-token dispatch (plan §5 rules 0–5) and admin-vs-client path split (plan §4 admin API table) obvious from the source tree.\n\n## Details\n\n- `Cargo.toml` deps: `axum`, `tokio` (multi-thread), `reqwest`, `serde`, `serde_json`, `config` (the crate), `tracing`, `tracing-subscriber`, `prometheus`, `miroir-core` (path dep)\n- `main.rs` should already bind `:7700` for the main server and `:9090` for metrics, even if every route returns `501 Not Implemented`\n- Stub `GET /health` to return `{\"status\":\"available\"}` (Meilisearch-compatible; used as K8s liveness)\n\n## Acceptance\n\n- [ ] `cargo build -p miroir-proxy --release --target x86_64-unknown-linux-musl` succeeds\n- [ ] Running the binary binds :7700 and :9090 and `curl http://localhost:7700/health` returns 200\n- [ ] Binary size (release, stripped) < 20 MB — ensures we hit the \"< 15 MB compressed\" target after Docker layer compression","status":"in_progress","priority":0,"issue_type":"task","assignee":"charlie","created_at":"2026-04-18T21:24:25.730677032Z","created_by":"coding","updated_at":"2026-04-19T00:45:55.314605753Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.3","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.730677032Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.4","title":"P0.4 Scaffold miroir-ctl crate","description":"## What\n\nCreate `crates/miroir-ctl/` — the management CLI. Module layout from plan §4:\n- `src/main.rs` — clap root, credential loading (plan §9 priority order)\n- `src/commands/{status,node,rebalance,reshard,verify,task,dump,alias,canary,ttl,cdc,shadow,ui,tenant,explain}.rs` — subcommand stubs\n\n## Why\n\nPlan §11 onboarding shows `miroir-ctl status`, `node add`, `rebalance status --watch`, `task status`, etc. These need to exist from early on so Phase 2+ features write their CLI as they go rather than accumulating a todo list. Also the admin-key loading priority (env → `~/.config/miroir/credentials` → `--admin-key` flag) deserves its own unit-testable module from day 1.\n\n## Details\n\n- `Cargo.toml` deps: `clap` (derive), `reqwest`, `serde`, `serde_json`, `tokio`, `miroir-core`\n- Admin-key loading order per plan §9 `miroir-ctl credential handling`:\n 1. `MIROIR_ADMIN_API_KEY` env\n 2. `~/.config/miroir/credentials` TOML\n 3. `--admin-key` flag (warn about process-list visibility in the help text)\n- Every subcommand returns `Err(\"not yet implemented\")` with a clear \"tracked in bead miroir-*\" message for now\n\n## Acceptance\n\n- [ ] `cargo build -p miroir-ctl --release --target x86_64-unknown-linux-musl` succeeds\n- [ ] `miroir-ctl --help` lists every subcommand enumerated in plan §4\n- [ ] Admin-key loader has a unit test for each of the 3 priority paths","status":"in_progress","priority":0,"issue_type":"task","assignee":"delta","created_at":"2026-04-18T21:24:25.751005786Z","created_by":"coding","updated_at":"2026-04-19T00:45:58.311013984Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.4","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.751005786Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.5","title":"P0.5 Config struct mirroring plan §4 YAML schema","description":"## What\n\nImplement `miroir_core::config::Config` — a `serde`-derived struct tree matching the plan §4 YAML schema exactly, including the §13 advanced-capabilities sub-structs (even if defaults produce `enabled: false`).\n\n## Why\n\nFuture phases can assume a typed `Config` rather than a `HashMap<String,Value>`. Every feature in §13 gets a dedicated struct with its own `enabled` flag + defaults per the plan. Centralizing defaults here makes the \"dev-sized vs. production\" story in plan §6 enforceable by a single `Config::validate()` function.\n\n## Details\n\nCover every block in the plan §4 YAML:\n- `MiroirConfig` — master_key, node_master_key, shards, replication_factor, task_store, admin, replica_groups, nodes[], health, scatter, rebalancer, server\n- `NodeConfig` — id, address, replica_group\n- `TaskStoreConfig` — backend (sqlite|redis), path, url\n- `HealthConfig`, `ScatterConfig`, `RebalancerConfig`, `ServerConfig`\n- `ConnectionPoolConfig`, `TaskRegistryConfig`\n- All §13 blocks: `ReshardingConfig`, `HedgingConfig`, `ReplicaSelectionConfig`, `QueryPlannerConfig`, `SettingsBroadcastConfig`, `SettingsDriftCheckConfig`, `SessionPinningConfig`, `AliasesConfig`, `AntiEntropyConfig`, `DumpImportConfig`, `IdempotencyConfig`, `QueryCoalescingConfig`, `MultiSearchConfig`, `VectorSearchConfig`, `CdcConfig` (+ CdcSinkConfig + CdcBufferConfig), `TtlConfig`, `TenantAffinityConfig`, `ShadowConfig`, `IlmConfig`, `CanaryRunnerConfig`, `ExplainConfig`, `AdminUiConfig`, `SearchUiConfig` (+ auth sub-structs)\n- `PeerDiscoveryConfig`, `LeaderElectionConfig`, `HpaConfig`\n\nPlus:\n- `Config::validate()` cross-field validation (e.g., replicas > 1 requires redis)\n- Layered loading via `config` crate: file → env var overrides → command-line\n- Tests: every example in the plan deserializes without error and re-serializes to equivalent YAML\n\n## Acceptance\n\n- [ ] Full plan §4 `miroir:` block deserializes into the struct without field loss\n- [ ] Every default in the plan is reproduced when the field is absent\n- [ ] `Config::validate()` rejects every combination the Helm `values.schema.json` will reject (dev-defaults in HA mode, scoped_key timing inversion, etc.)\n- [ ] Round-trip property test: YAML → Config → YAML is equivalent under a stable serializer","status":"in_progress","priority":0,"issue_type":"task","assignee":"alpha","created_at":"2026-04-18T21:24:25.775002832Z","created_by":"coding","updated_at":"2026-04-19T00:46:06.502908468Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.5","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.775002832Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.6","title":"P0.6 Repo hygiene: LICENSE, CHANGELOG skeleton, .gitignore, README stub","description":"## What\n\n- `LICENSE` — MIT, per plan §12\n- `CHANGELOG.md` — Keep a Changelog 1.1.0 format skeleton with `[Unreleased]` section\n- `.gitignore` — Rust (`target/`, `Cargo.lock` NOT ignored for binary crates), editor junk (`.vscode/`, `.idea/`)\n- `README.md` is already present — leave untouched for now; Phase 11 fills it in\n\n## Why\n\nPlan §12 explicitly requires MIT. Plan §7 \"CI release step extracts the relevant section automatically\" from CHANGELOG.md using an `awk` parser that expects `## [<version>]` section headers — the format must match from day 1 or the first release will fail.\n\n## Details\n\nSample CHANGELOG skeleton:\n```markdown\n# Changelog\n\nAll notable changes to this project will be documented in this file.\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),\nand this project adheres to [Semantic Versioning](https://semver.org/).\n\n## [Unreleased]\n\n### Added\n### Changed\n### Deprecated\n### Removed\n### Fixed\n### Security\n\n## [0.1.0] - TBD\n\n### Added\n- Initial release.\n```\n\n## Acceptance\n\n- [ ] `LICENSE` matches SPDX `MIT`\n- [ ] `awk \"/^## \\[0.1.0\\]/{found=1; next} found && /^## /{exit} found{print}\" CHANGELOG.md` (the extractor from plan §7) returns non-empty output for a tagged release\n- [ ] `.gitignore` keeps `target/` out and `Cargo.lock` in","status":"in_progress","priority":1,"issue_type":"task","assignee":"bravo","created_at":"2026-04-18T21:24:25.807632846Z","created_by":"coding","updated_at":"2026-04-19T00:46:09.520944305Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.6","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.807632846Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.7","title":"P0.7 CI smoke: fmt/clippy/test on push","description":"## What\n\nStand up a minimal CI path — just enough to run `cargo fmt --check`, `cargo clippy -D warnings`, `cargo test --all` — on every push to `main`. This is the earliest viable version of the full `miroir-ci` Argo Workflow template that Phase 8 ships.\n\n## Why\n\nIf CI only lands in Phase 8, Phases 1–7 accumulate quietly-broken code. Plan §7 makes fmt/clippy/test the first three steps of the pipeline on purpose; shipping those now (on iad-ci via a minimal WorkflowTemplate) catches regressions on every commit.\n\n## Details\n\n- Create a stripped-down `miroir-ci-smoke` WorkflowTemplate in `jedarden/declarative-config → k8s/iad-ci/argo-workflows/` that runs only checkout + lint + test\n- Trigger on push to `main` (initially operators kick manually; webhook automation lands in Phase 8)\n- Image: `rust:1.87-slim` to match the full CI template\n- No musl target yet (that's Phase 8); just `cargo test --all`\n\n## Acceptance\n\n- [ ] Manual submit: `kubectl --kubeconfig=$HOME/.kube/iad-ci.kubeconfig create -f - <<<workflow.yaml` runs the pipeline end-to-end in under 5 min\n- [ ] A breaking commit (intentional failing test) fails the pipeline visibly\n- [ ] The template lives in declarative-config and is synced by ArgoCD","status":"in_progress","priority":1,"issue_type":"task","assignee":"charlie","created_at":"2026-04-18T21:24:25.838313791Z","created_by":"coding","updated_at":"2026-04-19T00:46:12.526928056Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.7","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.838313791Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.1","title":"P0.1 Set up Cargo workspace + toolchain pin","description":"## What\n\nCreate the root Cargo workspace (`Cargo.toml` with `[workspace]` members), pin the Rust toolchain (`rust-toolchain.toml`), and add lint config (`rustfmt.toml`, `clippy.toml`, `.editorconfig`).\n\n## Why\n\nEverything else compiles against this. A pinned toolchain prevents \"works on my machine\" drift across contributors + CI (`rust:1.87-slim` per plan §7). Lint config in the repo from day 1 means we never have to retrofit formatting.\n\n## Details\n\n**Cargo.toml (workspace root):**\n```toml\n[workspace]\nresolver = \"2\"\nmembers = [\"crates/miroir-core\", \"crates/miroir-proxy\", \"crates/miroir-ctl\"]\n\n[workspace.package]\nversion = \"0.1.0\"\nedition = \"2021\"\nlicense = \"MIT\"\nrepository = \"https://github.com/jedarden/miroir\"\nrust-version = \"1.87\"\n```\n\n**rust-toolchain.toml:**\n```toml\n[toolchain]\nchannel = \"1.87\"\ncomponents = [\"rustfmt\", \"clippy\"]\ntargets = [\"x86_64-unknown-linux-musl\"]\n```\n\n**rustfmt.toml:** conservative default; `max_width = 100`, `edition = \"2021\"`.\n\n**clippy.toml:** empty for now; the `-D warnings` enforcement lives in CI (plan §7 `cargo-lint` template).\n\n## Acceptance\n\n- [ ] `cargo build` succeeds on an empty workspace (no members are complete yet but the workspace file parses)\n- [ ] `rustup show` in CI confirms the pinned channel\n- [ ] `cargo fmt --all -- --check` is a no-op (no files to check yet)\n- [ ] `cargo clippy --all-targets -- -D warnings` is a no-op","status":"closed","priority":0,"issue_type":"task","assignee":"alpha","created_at":"2026-04-18T21:24:25.694504043Z","created_by":"coding","updated_at":"2026-04-19T00:53:19.797128870Z","closed_at":"2026-04-19T00:53:19.796677927Z","close_reason":"Updated workspace Cargo.toml to match spec: explicit members list (miroir-core, miroir-proxy, miroir-ctl), edition 2021, MIT license, rust-version 1.87. Existing rust-toolchain.toml, rustfmt.toml, clippy.toml, and .editorconfig already matched requirements. Workspace metadata parses correctly with all three members identified.","source_repo":".","compaction_level":0,"original_size":0,"labels":["failure-count:1","phase-0"],"dependencies":[{"issue_id":"miroir-qon.1","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.694504043Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.2","title":"P0.2 Scaffold miroir-core crate","description":"## What\n\nCreate `crates/miroir-core/` with the module skeleton from plan §4:\n- `src/lib.rs` — public re-exports\n- `src/router.rs` — rendezvous hash primitives (signatures only; implementation in Phase 1)\n- `src/topology.rs` — `Topology`, `Group`, `Node`, `NodeId`, `NodeStatus` types\n- `src/scatter.rs` — scatter orchestration trait/stubs\n- `src/merger.rs` — result merge trait/stubs\n- `src/task.rs` — task registry trait/stubs\n- `src/config.rs` — `Config` struct (full shape matching plan §4 YAML)\n- `src/error.rs` — `MiroirError` enum + `Result<T>` alias\n\n## Why\n\nThe module boundary is intentional: pure library vs. binaries. `miroir-core` must stay dependency-light (no HTTP server, no CLI crate) so both binaries and downstream users can depend on it cleanly. This is also where the coverage gate (≥ 90%) applies per plan §8 coverage policy.\n\n## Details\n\n- Crate-type: `lib` (default); no `[[bin]]`\n- `Cargo.toml` deps: `serde`, `serde_json`, `twox-hash`, `thiserror`, `tracing` (minimal set — concrete feature-specific deps added as they're needed)\n- Public API starts small — add `pub use` entries to `lib.rs` only as modules are completed\n\n## Acceptance\n\n- [ ] `cargo build -p miroir-core` succeeds with empty stubs\n- [ ] `cargo doc -p miroir-core` produces rustdoc without warnings\n- [ ] `cargo test -p miroir-core` runs (zero tests) successfully","status":"closed","priority":0,"issue_type":"task","assignee":"bravo","created_at":"2026-04-18T21:24:25.717048243Z","created_by":"coding","updated_at":"2026-04-19T00:57:59.769651734Z","closed_at":"2026-04-19T00:57:59.769335750Z","close_reason":"done","source_repo":".","compaction_level":0,"original_size":0,"labels":["failure-count:2","phase-0"],"dependencies":[{"issue_id":"miroir-qon.2","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.717048243Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.3","title":"P0.3 Scaffold miroir-proxy crate","description":"## What\n\nCreate `crates/miroir-proxy/` — the HTTP proxy binary. Module layout from plan §4:\n- `src/main.rs` — startup (load config, init logging, start axum server, install signal handlers)\n- `src/routes/documents.rs`, `search.rs`, `indexes.rs`, `settings.rs`, `tasks.rs`, `health.rs`, `admin.rs` — route handler stubs\n- `src/auth.rs` — bearer-token dispatch per plan §5 (stubbed; real logic in Phase 2)\n- `src/middleware.rs` — tracing/logging + Prometheus middleware stubs\n\n## Why\n\nThis is the thing users install. Separating route modules by concern makes the bearer-token dispatch (plan §5 rules 0–5) and admin-vs-client path split (plan §4 admin API table) obvious from the source tree.\n\n## Details\n\n- `Cargo.toml` deps: `axum`, `tokio` (multi-thread), `reqwest`, `serde`, `serde_json`, `config` (the crate), `tracing`, `tracing-subscriber`, `prometheus`, `miroir-core` (path dep)\n- `main.rs` should already bind `:7700` for the main server and `:9090` for metrics, even if every route returns `501 Not Implemented`\n- Stub `GET /health` to return `{\"status\":\"available\"}` (Meilisearch-compatible; used as K8s liveness)\n\n## Acceptance\n\n- [ ] `cargo build -p miroir-proxy --release --target x86_64-unknown-linux-musl` succeeds\n- [ ] Running the binary binds :7700 and :9090 and `curl http://localhost:7700/health` returns 200\n- [ ] Binary size (release, stripped) < 20 MB — ensures we hit the \"< 15 MB compressed\" target after Docker layer compression","status":"closed","priority":0,"issue_type":"task","assignee":"charlie","created_at":"2026-04-18T21:24:25.730677032Z","created_by":"coding","updated_at":"2026-04-19T00:58:12.176910752Z","closed_at":"2026-04-19T00:58:12.176602790Z","close_reason":"done","source_repo":".","compaction_level":0,"original_size":0,"labels":["failure-count:2","phase-0"],"dependencies":[{"issue_id":"miroir-qon.3","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.730677032Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.4","title":"P0.4 Scaffold miroir-ctl crate","description":"## What\n\nCreate `crates/miroir-ctl/` — the management CLI. Module layout from plan §4:\n- `src/main.rs` — clap root, credential loading (plan §9 priority order)\n- `src/commands/{status,node,rebalance,reshard,verify,task,dump,alias,canary,ttl,cdc,shadow,ui,tenant,explain}.rs` — subcommand stubs\n\n## Why\n\nPlan §11 onboarding shows `miroir-ctl status`, `node add`, `rebalance status --watch`, `task status`, etc. These need to exist from early on so Phase 2+ features write their CLI as they go rather than accumulating a todo list. Also the admin-key loading priority (env → `~/.config/miroir/credentials` → `--admin-key` flag) deserves its own unit-testable module from day 1.\n\n## Details\n\n- `Cargo.toml` deps: `clap` (derive), `reqwest`, `serde`, `serde_json`, `tokio`, `miroir-core`\n- Admin-key loading order per plan §9 `miroir-ctl credential handling`:\n 1. `MIROIR_ADMIN_API_KEY` env\n 2. `~/.config/miroir/credentials` TOML\n 3. `--admin-key` flag (warn about process-list visibility in the help text)\n- Every subcommand returns `Err(\"not yet implemented\")` with a clear \"tracked in bead miroir-*\" message for now\n\n## Acceptance\n\n- [ ] `cargo build -p miroir-ctl --release --target x86_64-unknown-linux-musl` succeeds\n- [ ] `miroir-ctl --help` lists every subcommand enumerated in plan §4\n- [ ] Admin-key loader has a unit test for each of the 3 priority paths","status":"closed","priority":0,"issue_type":"task","assignee":"delta","created_at":"2026-04-18T21:24:25.751005786Z","created_by":"coding","updated_at":"2026-04-19T01:01:36.003063126Z","closed_at":"2026-04-19T01:01:36.002955040Z","close_reason":"Scaffolded miroir-ctl crate with all 15 subcommands from plan §4:\n\n- clap root CLI with credential loading (priority: MIROIR_ADMIN_API_KEY env → ~/.config/miroir/credentials.toml → --admin-key flag)\n- All subcommand stubs: status, node, rebalance, reshard, verify, task, dump, alias, canary, ttl, cdc, shadow, ui, tenant, explain\n- Unit tests for credential loader covering all 3 priority paths\n- Each subcommand returns clear 'not yet implemented' message pointing to tracking bead\n\nNote: musl target build not tested due to missing x86_64-linux-musl-gcc toolchain on this system; native release build succeeds.","source_repo":".","compaction_level":0,"original_size":0,"labels":["failure-count:3","phase-0"],"dependencies":[{"issue_id":"miroir-qon.4","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.751005786Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.5","title":"P0.5 Config struct mirroring plan §4 YAML schema","description":"## What\n\nImplement `miroir_core::config::Config` — a `serde`-derived struct tree matching the plan §4 YAML schema exactly, including the §13 advanced-capabilities sub-structs (even if defaults produce `enabled: false`).\n\n## Why\n\nFuture phases can assume a typed `Config` rather than a `HashMap<String,Value>`. Every feature in §13 gets a dedicated struct with its own `enabled` flag + defaults per the plan. Centralizing defaults here makes the \"dev-sized vs. production\" story in plan §6 enforceable by a single `Config::validate()` function.\n\n## Details\n\nCover every block in the plan §4 YAML:\n- `MiroirConfig` — master_key, node_master_key, shards, replication_factor, task_store, admin, replica_groups, nodes[], health, scatter, rebalancer, server\n- `NodeConfig` — id, address, replica_group\n- `TaskStoreConfig` — backend (sqlite|redis), path, url\n- `HealthConfig`, `ScatterConfig`, `RebalancerConfig`, `ServerConfig`\n- `ConnectionPoolConfig`, `TaskRegistryConfig`\n- All §13 blocks: `ReshardingConfig`, `HedgingConfig`, `ReplicaSelectionConfig`, `QueryPlannerConfig`, `SettingsBroadcastConfig`, `SettingsDriftCheckConfig`, `SessionPinningConfig`, `AliasesConfig`, `AntiEntropyConfig`, `DumpImportConfig`, `IdempotencyConfig`, `QueryCoalescingConfig`, `MultiSearchConfig`, `VectorSearchConfig`, `CdcConfig` (+ CdcSinkConfig + CdcBufferConfig), `TtlConfig`, `TenantAffinityConfig`, `ShadowConfig`, `IlmConfig`, `CanaryRunnerConfig`, `ExplainConfig`, `AdminUiConfig`, `SearchUiConfig` (+ auth sub-structs)\n- `PeerDiscoveryConfig`, `LeaderElectionConfig`, `HpaConfig`\n\nPlus:\n- `Config::validate()` cross-field validation (e.g., replicas > 1 requires redis)\n- Layered loading via `config` crate: file → env var overrides → command-line\n- Tests: every example in the plan deserializes without error and re-serializes to equivalent YAML\n\n## Acceptance\n\n- [ ] Full plan §4 `miroir:` block deserializes into the struct without field loss\n- [ ] Every default in the plan is reproduced when the field is absent\n- [ ] `Config::validate()` rejects every combination the Helm `values.schema.json` will reject (dev-defaults in HA mode, scoped_key timing inversion, etc.)\n- [ ] Round-trip property test: YAML → Config → YAML is equivalent under a stable serializer","status":"closed","priority":0,"issue_type":"task","assignee":"alpha","created_at":"2026-04-18T21:24:25.775002832Z","created_by":"coding","updated_at":"2026-04-19T01:52:51.379382557Z","closed_at":"2026-04-19T01:52:51.379316634Z","close_reason":"done","source_repo":".","compaction_level":0,"original_size":0,"labels":["deferred","failure-count:1","phase-0"],"dependencies":[{"issue_id":"miroir-qon.5","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.775002832Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.6","title":"P0.6 Repo hygiene: LICENSE, CHANGELOG skeleton, .gitignore, README stub","description":"## What\n\n- `LICENSE` — MIT, per plan §12\n- `CHANGELOG.md` — Keep a Changelog 1.1.0 format skeleton with `[Unreleased]` section\n- `.gitignore` — Rust (`target/`, `Cargo.lock` NOT ignored for binary crates), editor junk (`.vscode/`, `.idea/`)\n- `README.md` is already present — leave untouched for now; Phase 11 fills it in\n\n## Why\n\nPlan §12 explicitly requires MIT. Plan §7 \"CI release step extracts the relevant section automatically\" from CHANGELOG.md using an `awk` parser that expects `## [<version>]` section headers — the format must match from day 1 or the first release will fail.\n\n## Details\n\nSample CHANGELOG skeleton:\n```markdown\n# Changelog\n\nAll notable changes to this project will be documented in this file.\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),\nand this project adheres to [Semantic Versioning](https://semver.org/).\n\n## [Unreleased]\n\n### Added\n### Changed\n### Deprecated\n### Removed\n### Fixed\n### Security\n\n## [0.1.0] - TBD\n\n### Added\n- Initial release.\n```\n\n## Acceptance\n\n- [ ] `LICENSE` matches SPDX `MIT`\n- [ ] `awk \"/^## \\[0.1.0\\]/{found=1; next} found && /^## /{exit} found{print}\" CHANGELOG.md` (the extractor from plan §7) returns non-empty output for a tagged release\n- [ ] `.gitignore` keeps `target/` out and `Cargo.lock` in","status":"closed","priority":1,"issue_type":"task","assignee":"bravo","created_at":"2026-04-18T21:24:25.807632846Z","created_by":"coding","updated_at":"2026-04-19T00:48:12.804426259Z","closed_at":"2026-04-19T00:48:12.804262088Z","close_reason":"Created repo hygiene files: MIT LICENSE, CHANGELOG.md (Keep a Changelog 1.1.0 skeleton with [0.1.0] section), .gitignore (target/, editor junk; Cargo.lock kept). All acceptance criteria verified. Root commit initialized git repo.\n\n## Retrospective\n- **What worked:** Straightforward file creation — clear specs from plan.\n- **What didn't:** Nothing — acceptance criteria were unambiguous.\n- **Surprise:** Workspace had no git repo yet, so this became the root commit.\n- **Reusable pattern:** Always verify the plan's extraction command against CHANGELOG before committing.","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-0"],"dependencies":[{"issue_id":"miroir-qon.6","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.807632846Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-qon.7","title":"P0.7 CI smoke: fmt/clippy/test on push","description":"## What\n\nStand up a minimal CI path — just enough to run `cargo fmt --check`, `cargo clippy -D warnings`, `cargo test --all` — on every push to `main`. This is the earliest viable version of the full `miroir-ci` Argo Workflow template that Phase 8 ships.\n\n## Why\n\nIf CI only lands in Phase 8, Phases 1–7 accumulate quietly-broken code. Plan §7 makes fmt/clippy/test the first three steps of the pipeline on purpose; shipping those now (on iad-ci via a minimal WorkflowTemplate) catches regressions on every commit.\n\n## Details\n\n- Create a stripped-down `miroir-ci-smoke` WorkflowTemplate in `jedarden/declarative-config → k8s/iad-ci/argo-workflows/` that runs only checkout + lint + test\n- Trigger on push to `main` (initially operators kick manually; webhook automation lands in Phase 8)\n- Image: `rust:1.87-slim` to match the full CI template\n- No musl target yet (that's Phase 8); just `cargo test --all`\n\n## Acceptance\n\n- [ ] Manual submit: `kubectl --kubeconfig=$HOME/.kube/iad-ci.kubeconfig create -f - <<<workflow.yaml` runs the pipeline end-to-end in under 5 min\n- [ ] A breaking commit (intentional failing test) fails the pipeline visibly\n- [ ] The template lives in declarative-config and is synced by ArgoCD","status":"in_progress","priority":1,"issue_type":"task","assignee":"charlie","created_at":"2026-04-18T21:24:25.838313791Z","created_by":"coding","updated_at":"2026-04-19T02:54:01.224829079Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["deferred","failure-count:9","phase-0"],"dependencies":[{"issue_id":"miroir-qon.7","depends_on_id":"miroir-qon","type":"parent-child","created_at":"2026-04-18T21:24:25.838313791Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-r3j","title":"Phase 3 — Task Registry + Persistence (SQLite schema, Redis mirror)","description":"## Phase 3 Epic — Task Registry + Persistence\n\nAdds the 14-table task-store schema from plan §4 and a Redis mirror of the same keyspace so the system can survive pod restarts and (later) run multi-replica. Every §13 advanced capability and §14 HA mode consumes one or more of these tables, so settling the schema here prevents per-feature bespoke persistence.\n\n## Why This Happens Before §13 / §14\n\n- Plan §4 explicitly says \"Every table below is defined here and cross-referenced from the §13 / §14.5 section that consumes it.\"\n- Without `tasks`, any write that returns a `miroir_task_id` is ephemeral — a pod restart would lose every in-flight task (plan §3 task-id reconciliation paragraph).\n- Multi-pod HPA in Phase 6 **requires** Redis (plan §14.4 — Helm schema rejects `replicas > 1` + `taskStore.backend: sqlite`). Getting the Redis keyspace right now is cheaper than retrofitting.\n\n## Scope — the 14 tables and 14 Redis keyspaces (plan §4)\n\n1. `tasks` — Miroir task registry (miroir_id → node_tasks map + status)\n2. `node_settings_version` — per-(index, node) settings freshness (for §13.5 + `X-Miroir-Min-Settings-Version`)\n3. `aliases` — single-target + multi-target (`kind`, `current_uid`, `target_uids`, `version`, `history`)\n4. `sessions` — read-your-writes session pins (§13.6)\n5. `idempotency_cache` — write dedup (§13.10)\n6. `jobs` — work-queued background jobs (§14.5 Mode C)\n7. `leader_lease` — singleton-coordinator lease (§14.5 Mode B; SQLite advisory lock substitute for single-replica)\n8. `canaries` — canary definitions (§13.18)\n9. `canary_runs` — canary run history (§13.18)\n10. `cdc_cursors` — per-(sink, index) CDC cursor (§13.13)\n11. `tenant_map` — API-key → tenant mapping (§13.15 `api_key` mode)\n12. `rollover_policies` — ILM rollover policies (§13.17)\n13. `search_ui_config` — per-index search-UI config (§13.21)\n14. `admin_sessions` — Admin UI session registry (§13.19)\n\n## Redis keyspace mirror (plan §4 \"Redis mode (HA)\")\n\nEvery table above mapped to a hash + `_index` secondary set so list-wide queries are O(cardinality) without `SCAN`. Plus:\n\n- `miroir:ratelimit:searchui:<ip>` (EXPIRE `search_ui.rate_limit.redis_ttl_s`)\n- `miroir:ratelimit:adminlogin:<ip>` + `miroir:ratelimit:adminlogin:backoff:<ip>` (§13.19, required in HA)\n- `miroir:cdc:overflow:<sink>` (1 GiB per sink default)\n- `miroir:search_ui_scoped_key:<index>` + `miroir:search_ui_scoped_key_observed:<pod>:<index>` (§13.21 rotation coordination)\n- `miroir:admin_session:revoked` Pub/Sub channel for instant logout propagation\n\n## Definition of Done\n\n- [ ] `rusqlite`-backed store initializing every table idempotently at startup\n- [ ] Redis-backed store mirrors the same API (trait `TaskStore` or equivalent), chosen at runtime by `task_store.backend`\n- [ ] Migrations/versioning: schema version recorded in a `schema_version` row so future upgrades detect incompatibility loudly\n- [ ] Property tests: `(insert, get)` round-trip + `(upsert, list)` semantics on SQLite backend\n- [ ] Integration test: restart an orchestrator pod mid-task-poll; task status survives (simulate by opening/closing the SQLite handle between operations)\n- [ ] Redis-backend integration test (`testcontainers` or similar) exercising leases, idempotency dedup, and alias history\n- [ ] `miroir:tasks:_index`-style iteration actually used for list endpoints (no `SCAN`)\n- [ ] `taskStore.backend: redis` + `replicas > 1` enforced by Helm `values.schema.json` (verified with `helm lint`)\n- [ ] Plan §14.7 Redis memory accounting validated against a representative load (bucket count × average size)","status":"open","priority":0,"issue_type":"epic","created_at":"2026-04-18T21:19:53.974489140Z","created_by":"coding","updated_at":"2026-04-18T21:23:08.581853353Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase","phase-3"],"dependencies":[{"issue_id":"miroir-r3j","depends_on_id":"miroir-qon","type":"blocks","created_at":"2026-04-18T21:23:08.581818683Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-r3j.1","title":"P3.1 TaskStore trait + SQLite backend (tables 1-7)","description":"## What\n\nDefine the `TaskStore` trait in `miroir-core` and implement the SQLite backend for the first 7 tables in plan §4 \"Task store schema\":\n\n1. `tasks` — Miroir task registry\n2. `node_settings_version`\n3. `aliases` (both single and multi-target)\n4. `sessions` (read-your-writes pins)\n5. `idempotency_cache`\n6. `jobs`\n7. `leader_lease`\n\n## Why Start Here\n\nThese are the always-present tables — needed even in single-pod dev mode. Tables 8–14 (canaries, cdc_cursors, tenant_map, rollover_policies, search_ui_config, admin_sessions) only instantiate when their respective feature flag is on, so they can land alongside the Phase 5 feature they serve.\n\nDefining the trait **in `miroir-core`** (not `miroir-proxy`) lets the crate be consumed by `miroir-ctl` for diagnostics without pulling in the proxy binary.\n\n## Details\n\nEach table's DDL is already in plan §4 (scroll to the table headers). The trait exposes per-table operations plus a generic `migrate(&self) -> Result<()>` that creates tables idempotently and records a `schema_version` row for upgrade detection.\n\n**Non-obvious**:\n- `tasks.node_tasks` is JSON — use a `serde_json::Value` column, not a stringly-typed hack\n- `aliases.history` is a JSON array bounded by `aliases.history_retention`; enforce bound on `UPDATE`\n- `idempotency_cache.body_sha256` is a `BLOB`, not TEXT — 32 raw bytes\n- `jobs.claim_expires_at` updated by heartbeat every 10s; pod loss → claim expires → another pod picks up\n- `leader_lease` for SQLite is an advisory-lock substitute (persist the row, interpret its presence semantically)\n\n**Idempotent migrations** — use `CREATE TABLE IF NOT EXISTS` + a `schema_versions` table that records each applied migration. Future migrations use `INSERT OR IGNORE` + explicit version gates.\n\n## Acceptance\n\n- [ ] `cargo test -p miroir-core task_store::sqlite` — every CRUD round-trips correctly\n- [ ] Opening an existing DB doesn't re-run migrations; schema version check is a single SELECT\n- [ ] Concurrent writes from two handles (single-process) don't deadlock (WAL mode enabled, `PRAGMA busy_timeout = 5000`)\n- [ ] Table sizes under realistic load fit within plan §14.2 \"Task registry cache 100 MB\" budget","status":"open","priority":0,"issue_type":"task","created_at":"2026-04-18T21:30:07.264404312Z","created_by":"coding","updated_at":"2026-04-18T21:30:07.264404312Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-3"],"dependencies":[{"issue_id":"miroir-r3j.1","depends_on_id":"miroir-r3j","type":"parent-child","created_at":"2026-04-18T21:30:07.264404312Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-r3j.2","title":"P3.2 SQLite backend: remaining tables (canaries, cdc_cursors, tenant_map, rollover_policies, search_ui_config, admin_sessions)","description":"## What\n\nExtend the SQLite `TaskStore` with plan §4 tables 8–14:\n8. `canaries` (§13.18)\n9. `canary_runs` (§13.18) — bounded by `canary_runner.run_history_per_canary` (default 100); auto-prune on insert\n10. `cdc_cursors` (§13.13)\n11. `tenant_map` (§13.15 `api_key` mode only)\n12. `rollover_policies` (§13.17)\n13. `search_ui_config` (§13.21)\n14. `admin_sessions` (§13.19) — with `CREATE INDEX admin_sessions_expires ON admin_sessions(expires_at)` for lazy eviction\n\n## Why Separate from P3.1\n\nThese tables are **feature-flag-gated** — `canaries` only instantiates when `canary_runner.enabled`, etc. Keeping them in a separate task lets Phase 5 subsection beads own each table's lifecycle and prevents the ~14-table `CREATE TABLE IF NOT EXISTS` cascade from running for features that will never be used.\n\nThat said, the schema definition itself lives here so every Phase 5 feature can `use` the same typed row structs rather than redefining them ad-hoc.\n\n## Details\n\n**`canary_runs` auto-prune**: on each insert, `DELETE FROM canary_runs WHERE canary_id = ? AND ran_at < (SELECT MIN(ran_at) FROM (SELECT ran_at FROM canary_runs WHERE canary_id = ? ORDER BY ran_at DESC LIMIT N))`. Wrap in a trigger so application code never forgets.\n\n**`admin_sessions.expires_at` index** — plan §4 admin_sessions footnote: rows past expires_at evicted lazily on access AND by Mode A pruner (§14.5). The index makes the scan cheap.\n\n**`cdc_cursors` is a per-(sink, index) composite PK** — both columns must match for update-in-place.\n\n**`tenant_map.api_key_hash` is a 32-byte BLOB** — raw sha256 bytes; never store the plaintext API key.\n\n## Acceptance\n\n- [ ] Every table's typed struct round-trips `insert`/`get` in a unit test\n- [ ] `canary_runs` trigger keeps row count ≤ `run_history_per_canary`\n- [ ] Tables that remain empty when their feature is disabled consume < 16 KB each (SQLite overhead)\n- [ ] Tables are created only when `TaskStore::migrate` is called with the relevant feature flag set (so dev-mode single-pod with all features off creates just 7 tables)","status":"open","priority":0,"issue_type":"task","created_at":"2026-04-18T21:30:07.286925769Z","created_by":"coding","updated_at":"2026-04-18T21:30:11.179830060Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-3"],"dependencies":[{"issue_id":"miroir-r3j.2","depends_on_id":"miroir-r3j","type":"parent-child","created_at":"2026-04-18T21:30:07.286925769Z","created_by":"coding","metadata":"{}","thread_id":""},{"issue_id":"miroir-r3j.2","depends_on_id":"miroir-r3j.1","type":"blocks","created_at":"2026-04-18T21:30:11.179800727Z","created_by":"coding","metadata":"{}","thread_id":""}]}
@ -136,9 +136,9 @@
{"id":"miroir-uyx.5","title":"P11.5 Common issues + troubleshooting","description":"## What\n\nPlan §11 \"Common issues\" as structured troubleshooting docs at `docs/troubleshooting.md`:\n- \"primary key required\" — Miroir requires explicit primary key at index creation\n- \"Search returns fewer results than expected\" — degraded-node cross-reference + `GET /_miroir/topology`\n- \"Task polling stuck at processing\" — per-node task status via `miroir-ctl task status`\n\nPlus others discovered during Phase 9 testing and chaos scenarios.\n\n## Why\n\nEvery production system accumulates a list of \"the 10 things new users hit in their first week.\" Documenting them transparently shortens the mean-time-to-productive-user from hours to minutes.\n\n## Details\n\n**Per-issue structure**:\n```markdown\n## Error: \"primary key required\"\n\n### Symptom\nClient sees: `HTTP 400 { \"code\": \"miroir_primary_key_required\" }`\n\n### Cause\nThe index was created without a primary key. Miroir cannot route without one.\n\n### Fix\n```bash\ncurl -X POST https://miroir/indexes \\\n -H \"Authorization: Bearer $KEY\" \\\n -d '{\"uid\": \"myindex\", \"primaryKey\": \"id\"}'\n```\n\n### Why this differs from Meilisearch\nMeilisearch can infer the primary key from the first document batch. Miroir cannot — it needs to hash the PK *before* any node sees it. Explicit primary_key at index creation is required.\n```\n\n**Diagnostic playbook**: `docs/troubleshooting/diagnostics.md` — first thing to check for any symptom:\n1. `GET /_miroir/topology` — all nodes healthy?\n2. `GET /_miroir/metrics | grep degraded` — any degraded shards?\n3. `kubectl logs miroir-0 --tail=100 | jq 'select(.level==\"ERROR\")'` — recent errors?\n4. `kubectl get pods -n search` — all running?\n\n## Acceptance\n\n- [ ] 3 plan §11 issues documented with the template\n- [ ] At least 5 additional issues discovered in Phase 9 chaos added\n- [ ] Troubleshooting doc cross-linked from README, install guide, each migration guide","status":"open","priority":1,"issue_type":"task","created_at":"2026-04-18T21:48:38.877214633Z","created_by":"coding","updated_at":"2026-04-18T21:48:38.877214633Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-11"],"dependencies":[{"issue_id":"miroir-uyx.5","depends_on_id":"miroir-uyx","type":"parent-child","created_at":"2026-04-18T21:48:38.877214633Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-uyx.6","title":"P11.6 Helm chart publication: GH Pages + OCI push","description":"## What\n\nPlan §12 delivered artifacts for the Helm chart:\n- **Primary**: `https://jedarden.github.io/miroir` (GitHub Pages, `gh-pages` branch)\n- **OCI**: `ghcr.io/jedarden/charts/miroir` (for air-gapped environments)\n\nExtend the Phase 8 Argo Workflow `miroir-ci` template with:\n- On tag: `helm package charts/miroir -d dist/`\n- Push to gh-pages: update `index.yaml` + copy `.tgz` into the branch, commit via `gh-pages` helper\n- OCI push: `helm push dist/miroir-<version>.tgz oci://ghcr.io/jedarden/charts`\n\n## Why\n\nPlan §12: chart users expect `helm repo add` to work. Without publication, operators have to `helm install charts/miroir/` from a git clone — fine for dev, wrong for prod.\n\n## Details\n\n**gh-pages flow**:\n```bash\ngit worktree add gh-pages gh-pages\nhelm package charts/miroir -d gh-pages/\nhelm repo index gh-pages/ --url https://jedarden.github.io/miroir --merge gh-pages/index.yaml\ngit -C gh-pages add -A\ngit -C gh-pages commit -m \"Release chart v<version>\"\ngit -C gh-pages push origin gh-pages\n```\n\n**OCI push** requires GHCR write token (already have in `ghcr-credentials`):\n```bash\necho $GHCR_TOKEN | helm registry login ghcr.io -u <user> --password-stdin\nhelm push miroir-<version>.tgz oci://ghcr.io/jedarden/charts\n```\n\n**Chart-only fixes**: when a chart change doesn't need an app rebuild, bump only chart version (not appVersion). CI must detect \"chart-only\" change (e.g., by diffing `charts/**` vs. `crates/**`) and skip the binary rebuild.\n\n## Acceptance\n\n- [ ] After `git tag v0.1.0 && git push`, `helm repo add miroir https://jedarden.github.io/miroir && helm repo update` discovers v0.1.0\n- [ ] `helm install ... oci://ghcr.io/jedarden/charts/miroir --version 0.1.0` works identically\n- [ ] Chart-only fix: tagging `v0.1.1` after editing only a template file bumps chart version without new app binary","status":"open","priority":2,"issue_type":"task","created_at":"2026-04-18T21:48:38.909893288Z","created_by":"coding","updated_at":"2026-04-18T21:48:38.909893288Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase-11"],"dependencies":[{"issue_id":"miroir-uyx.6","depends_on_id":"miroir-uyx","type":"parent-child","created_at":"2026-04-18T21:48:38.909893288Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2","title":"Phase 12 — Open Problems + Research (§15)","description":"## Phase 12 Epic — Open Problems Tracking\n\nStanding bucket for the plan §15 open problems that are **not** fully resolved by initial implementation. These are research/validation/future-enhancement beads, not blockers for v1.0. This phase does not block the genesis bead's shipping path — it's a parallel track that persists beyond v1.0.\n\n## Why An Epic At All\n\nPlan §15 flags these as \"documented constraints, not blockers. Initial release ships with known limitations.\" Tracking them as beads means they're not forgotten, they have a visible owner, and their resolution status can be surfaced alongside the rest of the work.\n\n## Scope — the 6 Open Problems (plan §15)\n\n1. **Shard migration write safety** — OP#1. **Status: partially addressed.** Dual-write cutover sequencing (Phase 4) + anti-entropy reconciler (§13.8 / Phase 5) catches slipped docs. Remaining work: chaos-test the cutover boundary, document any reproducible window where data could be lost if anti-entropy is disabled.\n\n2. **Task state HA (Raft vs. Redis)** — OP#2. **Status: deferred.** Current: Redis for multi-pod, SQLite for single-pod. Future: lightweight in-process Raft (or equivalent) so Redis is not required in HA. Not v1.x.\n\n3. **Resharding (S change) vs. node scaling (N change)** — OP#3. **Status: addressed by §13.1** (shadow-index dual-hash). Remaining work: empirical validation of the §13.1 \"2× transient storage and write load\" caveat under real corpora; schedule guidance in the CLI for off-peak reshard windows.\n\n4. **Score normalization at scale** — OP#4. **Status: settings-divergence addressed by §13.5 two-phase broadcast + drift reconciler.** Remaining work is purely statistical: validate that `_rankingScore` remains comparable across shards with very different document-count distributions. Requires corpus diversity tests.\n\n5. **Dump import distribution** — OP#5. **Status: addressed by §13.9 streaming routed dump import.** Broadcast mode retained as fallback. Remaining work: identify and enumerate every dump variant `mode: streaming` cannot fully reconstruct; either extend streaming or document the fallback trigger clearly.\n\n6. **arm64 support** — OP#6. **Status: not planned for v0.x.** Wire into CI when K8s ARM node support is actually needed (likely v1.x or later).\n\n## How To Use This Phase\n\n- Each OP becomes a child bead (bug/feature type) under this epic\n- Beads stay open until the status column above says \"fully addressed\"\n- v1.0 release notes should explicitly link to this epic so operators know what's still on the table\n- New open problems discovered during implementation get added here rather than silently accreted elsewhere\n\n## Not In Scope\n\n- Any concrete implementation work already covered by §13.1 / §13.5 / §13.8 / §13.9 — that belongs to Phase 5.","status":"open","priority":2,"issue_type":"epic","created_at":"2026-04-18T21:22:54.403910669Z","created_by":"coding","updated_at":"2026-04-18T21:22:54.403910669Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["phase","phase-12","research"]}
{"id":"miroir-zc2.1","title":"P12.OP1 Shard migration write safety — cutover race window analysis","description":"## What\n\nPlan §15 Open Problem #1: \"Dual-write during migration must not lose documents that arrive exactly at the migration cutover boundary.\"\n\n**Status** per plan: partially addressed. Race window mitigated by §13.8 anti-entropy; any slipped doc caught on next reconciliation pass.\n\n**Remaining work**:\n- Chaos-test the cutover boundary — specifically: docs arriving at the instant of `active` transition (step 7 in plan §2 \"Adding a node\")\n- Document any reproducible window where data could be lost if anti-entropy is disabled\n- If found: extend Phase 4 dual-write to hold the window longer OR require anti-entropy to be on (hard-coded policy)\n\n## Why\n\n\"Plan §15 Open Problem 1 closure\" has been claimed in §13.8 — this bead verifies that claim empirically before we ship v1.0 committing to it.\n\n## Details\n\n**Chaos test design**:\n1. Start 3-node cluster, write 1000 docs\n2. Trigger node addition (`POST /_miroir/nodes`)\n3. During dual-write, rapid-fire new writes with tight (1ms) interval\n4. Tight-loop the transition from step 4 (migration complete) to step 7 (old replica deleted)\n5. Assert: every written doc retrievable AFTER step 7\n\n**Variants**:\n- With anti-entropy enabled (default) — expect 100% retrievable\n- With anti-entropy **disabled** — measure loss rate. If > 0, document + add a schema constraint refusing to enable migrations when anti-entropy is off\n\n## Acceptance\n\n- [ ] Chaos test published; runs on every v1.0-gating CI run\n- [ ] Loss rate measured at < 1 per 1M writes with AE on\n- [ ] Loss rate measured without AE; decision documented in `docs/trade-offs.md`\n- [ ] If `anti_entropy.enabled: false` + migration concurrent → loud warning log + (decided) refuse or warn","status":"in_progress","priority":2,"issue_type":"bug","assignee":"alpha","created_at":"2026-04-18T21:49:47.774525899Z","created_by":"coding","updated_at":"2026-04-19T00:46:16.945939685Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.1","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.774525899Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.2","title":"P12.OP2 Task state HA — evaluate lightweight Raft vs. Redis requirement","description":"## What\n\nPlan §15 Open Problem #2: \"SQLite is single-writer. Running 2 Miroir replicas requires Redis. A future enhancement is a lightweight Raft-based in-process consensus so Redis is not required for HA mode.\"\n\n**Status** per plan: deferred. Current solution (Redis) works; Raft would remove an external dependency.\n\n**Research work**:\n- Survey embedded Raft crates: `openraft`, `raft-rs`, `async-raft`\n- Prototype: `TaskStore` trait impl backed by Raft state machine\n- Measure: latency + throughput vs. Redis; memory footprint per plan §14.2\n- Decide: ship in v1.x or never\n\n## Why\n\nRemoving Redis as a hard dependency shrinks the operational surface (one less thing to monitor, backup, rotate secrets for). But Raft adds complexity — a bad Raft impl can eat data in ways Redis doesn't.\n\nNot blocking v0.x or v1.0 — but worth prototyping before v2.0.\n\n## Details\n\n**Decision gate**: the Raft-backed path must be measurably better than Redis on at least one metric (ops simplicity, latency, or memory) without being worse on any of the others, before shipping.\n\n**Output**: `docs/research/raft-task-store.md` with the decision + benchmark data + reasoning. Keep or discard based on findings.\n\n## Acceptance\n\n- [ ] Research doc published with prototype branch linked\n- [ ] Decision recorded: ship / don't ship / revisit when","status":"in_progress","priority":3,"issue_type":"feature","assignee":"bravo","created_at":"2026-04-18T21:49:47.798646718Z","created_by":"coding","updated_at":"2026-04-19T00:46:19.981354533Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.2","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.798646718Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.3","title":"P12.OP3 Online resharding — validate 2× transient load caveat under real corpora","description":"## What\n\nPlan §15 Open Problem #3: §13.1 online resharding ships as a remediation, NOT a license to under-provision. Plan: \"doubles transient storage and write load; treat §13.1 as a remediation, not a license to under-provision.\"\n\n**Remaining work**:\n- Empirical validation of the 2× storage + write load estimate under real corpora (varied doc sizes, write rates, settings complexity)\n- CLI schedule guidance: `miroir-ctl reshard --schedule-window off-peak` — refuses to start outside a named window unless `--force`\n\n## Why\n\nOperators will over-commit to resharding if the \"2× transient\" caveat turns out to be 3× or worse in practice. Real numbers prevent that.\n\n## Details\n\n**Test matrix**:\n| Doc size | Corpus | Write rate | RG | RF | Measured peak storage |\n|----------|--------|------------|----|----|-----------------------|\n| 1 KB | 10 GB | 100 dps | 2 | 1 | ? |\n| 10 KB | 100 GB | 1000 dps | 2 | 2 | ? |\n| 1 MB (blobs) | 1 TB | 10 dps | 2 | 1 | ? |\n\nPublish results in `docs/benchmarks/resharding-load.md`.\n\n**CLI window guard**: config knob `resharding.allowed_windows: [\"02:00-06:00 UTC\"]`. CLI refuses outside windows without `--force`.\n\n## Acceptance\n\n- [ ] Benchmark doc published with real numbers\n- [ ] CLI window guard implemented; integration test confirms rejection outside window\n- [ ] Benchmark run in Phase 9 performance suite as part of v1.0 validation","status":"open","priority":3,"issue_type":"task","created_at":"2026-04-18T21:49:47.828099118Z","created_by":"coding","updated_at":"2026-04-18T21:49:47.828099118Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.3","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.828099118Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.4","title":"P12.OP4 Score normalization at scale — statistical validation of cross-shard comparability","description":"## What\n\nPlan §15 Open Problem #4: \"`_rankingScore` is comparable across shards only when index settings are identical.\" Settings divergence addressed by §13.5; remaining concern is statistical — do scores stay comparable when shards have very different document-count distributions?\n\n**Research work**:\n- Build a test corpus with intentionally skewed shard populations (one shard 100×, another shard 0.01× the median)\n- Submit identical queries; measure score distribution per shard\n- Assert: top-K merged ordering matches a ground-truth single-index version within some ε\n- If large ε, document + possibly introduce a score normalization pass\n\n## Why\n\nElasticsearch (plan research doc §1) hits this exactly: \"BM25 scoring depends on IDF, computed per shard by default using only that shard's local term statistics.\" Meilisearch uses its own ranking pipeline, but the same issue applies — local rank stats can drift from global on skewed shards.\n\n## Details\n\n**Ground truth**: single-index Meilisearch running the same queries against the same corpus.\n\n**Divergence metric**: Kendall τ between Miroir result ordering and single-index result ordering across 10k random queries.\n\n**If τ < 0.95 on average**: investigate whether a global IDF-style preflight is worth adding (plan research §1 \"`dfs_query_then_fetch`\" pattern).\n\n**Output**: `docs/research/score-normalization-at-scale.md`.\n\n## Acceptance\n\n- [ ] Benchmark corpus + query set published in `tests/benches/score-comparability/`\n- [ ] Results reported with confidence intervals\n- [ ] If τ < 0.95: follow-up bead created for a normalization pass\n- [ ] If τ ≥ 0.95: note-of-no-action in the bead's close comment","status":"open","priority":3,"issue_type":"task","created_at":"2026-04-18T21:49:47.849019120Z","created_by":"coding","updated_at":"2026-04-18T21:49:47.849019120Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.4","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.849019120Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.5","title":"P12.OP5 Dump import variants — enumerate what streaming mode can't handle","description":"## What\n\nPlan §15 Open Problem #5: §13.9 streaming routed dump import addresses the main case; broadcast mode retained as a fallback for dump variants Miroir cannot fully reconstruct via public API.\n\n**Remaining work**:\n- Identify and enumerate every dump variant streaming can't reconstruct\n- Either extend streaming to handle them OR document the fallback trigger clearly in `miroir-ctl dump import --help`\n\n## Why\n\n\"Can't reconstruct\" is vague — operators deserve concrete lists of what works and what doesn't. Without this, the `broadcast` fallback path is a bug waiting to happen.\n\n## Details\n\n**Potential failure modes to investigate**:\n- Dumps from older Meilisearch versions with pre-v1.37 schema\n- Dumps with custom keys (POST /keys) that have indexes list or actions not representable via public API\n- Dumps with snapshot-taken-mid-write where Miroir-injected `_miroir_shard` would conflict with an existing client field\n\n**Deliverable**: `docs/dump-import/compatibility-matrix.md` with columns:\n| Meilisearch version | Dump variant | Streaming works? | Broadcast needed? | Workaround |\n\n## Acceptance\n\n- [ ] Matrix published\n- [ ] Each \"broadcast needed\" row has a workaround or a link to an open enhancement bead\n- [ ] `miroir-ctl dump import` output references the matrix when falling back to broadcast","status":"open","priority":3,"issue_type":"task","created_at":"2026-04-18T21:49:47.884303207Z","created_by":"coding","updated_at":"2026-04-18T21:49:47.884303207Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.5","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.884303207Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.6","title":"P12.OP6 arm64 support (deferred to v1.x+)","description":"## What\n\nPlan §15 Open Problem #6: \"Not planned for v0.x. Added when K8s ARM node support is required.\"\n\n**Future work when prioritized**:\n- Cross-compile `miroir-proxy` and `miroir-ctl` for `aarch64-unknown-linux-musl` in the CI pipeline\n- Docker image manifest list: `ghcr.io/jedarden/miroir:<version>` spans `linux/amd64` + `linux/arm64`\n- Helm chart: no changes (binary is arch-agnostic at the k8s layer)\n- Phase 9 CI: add arm64 test runs\n\n## Why\n\nARM node support is increasingly common (Hetzner Ampere, AWS Graviton, GCP Tau T2A, Rackspace Spot). But Miroir's fleet is currently all amd64 (iad-ci is amd64; ardenone cluster nodes are amd64). No current demand to justify the CI complexity.\n\nKeep this bead open as a placeholder; promote to in-progress when a concrete use case emerges.\n\n## Details\n\n**When ready**: the Argo Workflow `cargo-build` step needs a matrix over targets:\n```yaml\n- name: cargo-build\n container:\n args:\n - |\n rustup target add x86_64-unknown-linux-musl\n rustup target add aarch64-unknown-linux-musl\n apt-get install -qy musl-tools gcc-aarch64-linux-gnu\n cargo build --release --target x86_64-unknown-linux-musl -p miroir-proxy\n cargo build --release --target aarch64-unknown-linux-musl -p miroir-proxy\n ...\n```\n\nKaniko build needs `--customPlatform=linux/amd64,linux/arm64` or equivalent for multi-arch manifests.\n\n## Acceptance\n\n- [ ] Not to be closed until arm64 is a live deliverable\n- [ ] Cross-reference here when the priority flips","status":"open","priority":4,"issue_type":"feature","created_at":"2026-04-18T21:49:47.917666333Z","created_by":"coding","updated_at":"2026-04-18T21:49:47.917666333Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","roadmap"],"dependencies":[{"issue_id":"miroir-zc2.6","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.917666333Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.1","title":"P12.OP1 Shard migration write safety — cutover race window analysis","description":"## What\n\nPlan §15 Open Problem #1: \"Dual-write during migration must not lose documents that arrive exactly at the migration cutover boundary.\"\n\n**Status** per plan: partially addressed. Race window mitigated by §13.8 anti-entropy; any slipped doc caught on next reconciliation pass.\n\n**Remaining work**:\n- Chaos-test the cutover boundary — specifically: docs arriving at the instant of `active` transition (step 7 in plan §2 \"Adding a node\")\n- Document any reproducible window where data could be lost if anti-entropy is disabled\n- If found: extend Phase 4 dual-write to hold the window longer OR require anti-entropy to be on (hard-coded policy)\n\n## Why\n\n\"Plan §15 Open Problem 1 closure\" has been claimed in §13.8 — this bead verifies that claim empirically before we ship v1.0 committing to it.\n\n## Details\n\n**Chaos test design**:\n1. Start 3-node cluster, write 1000 docs\n2. Trigger node addition (`POST /_miroir/nodes`)\n3. During dual-write, rapid-fire new writes with tight (1ms) interval\n4. Tight-loop the transition from step 4 (migration complete) to step 7 (old replica deleted)\n5. Assert: every written doc retrievable AFTER step 7\n\n**Variants**:\n- With anti-entropy enabled (default) — expect 100% retrievable\n- With anti-entropy **disabled** — measure loss rate. If > 0, document + add a schema constraint refusing to enable migrations when anti-entropy is off\n\n## Acceptance\n\n- [ ] Chaos test published; runs on every v1.0-gating CI run\n- [ ] Loss rate measured at < 1 per 1M writes with AE on\n- [ ] Loss rate measured without AE; decision documented in `docs/trade-offs.md`\n- [ ] If `anti_entropy.enabled: false` + migration concurrent → loud warning log + (decided) refuse or warn","status":"closed","priority":2,"issue_type":"bug","assignee":"alpha","created_at":"2026-04-18T21:49:47.774525899Z","created_by":"coding","updated_at":"2026-04-19T02:01:02.057461283Z","closed_at":"2026-04-19T02:01:02.057395870Z","close_reason":"done","source_repo":".","compaction_level":0,"original_size":0,"labels":["deferred","open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.1","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.774525899Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.2","title":"P12.OP2 Task state HA — evaluate lightweight Raft vs. Redis requirement","description":"## What\n\nPlan §15 Open Problem #2: \"SQLite is single-writer. Running 2 Miroir replicas requires Redis. A future enhancement is a lightweight Raft-based in-process consensus so Redis is not required for HA mode.\"\n\n**Status** per plan: deferred. Current solution (Redis) works; Raft would remove an external dependency.\n\n**Research work**:\n- Survey embedded Raft crates: `openraft`, `raft-rs`, `async-raft`\n- Prototype: `TaskStore` trait impl backed by Raft state machine\n- Measure: latency + throughput vs. Redis; memory footprint per plan §14.2\n- Decide: ship in v1.x or never\n\n## Why\n\nRemoving Redis as a hard dependency shrinks the operational surface (one less thing to monitor, backup, rotate secrets for). But Raft adds complexity — a bad Raft impl can eat data in ways Redis doesn't.\n\nNot blocking v0.x or v1.0 — but worth prototyping before v2.0.\n\n## Details\n\n**Decision gate**: the Raft-backed path must be measurably better than Redis on at least one metric (ops simplicity, latency, or memory) without being worse on any of the others, before shipping.\n\n**Output**: `docs/research/raft-task-store.md` with the decision + benchmark data + reasoning. Keep or discard based on findings.\n\n## Acceptance\n\n- [ ] Research doc published with prototype branch linked\n- [ ] Decision recorded: ship / don't ship / revisit when","status":"closed","priority":3,"issue_type":"feature","assignee":"bravo","created_at":"2026-04-18T21:49:47.798646718Z","created_by":"coding","updated_at":"2026-04-19T02:57:16.452177084Z","closed_at":"2026-04-19T02:57:16.452114067Z","close_reason":"P12.OP2 complete. Surveyed openraft/raft-rs/async-raft (recommend openraft if revisited). Built feature-gated Raft state machine prototype at crates/miroir-core/src/raft_proto/ with benchmarks. Decision: do not ship Raft in v0.x/v1.0 -- Redis wins on write latency, throughput, correctness maturity, and operational tooling. Raft only wins on ops simplicity and read latency. Does not pass the decision gate. Revisit before v2.0 when Redis backend is production-stabilized and openraft reaches v1.0. Full analysis in docs/research/raft-task-store.md.","source_repo":".","compaction_level":0,"original_size":0,"labels":["deferred","open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.2","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.798646718Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.3","title":"P12.OP3 Online resharding — validate 2× transient load caveat under real corpora","description":"## What\n\nPlan §15 Open Problem #3: §13.1 online resharding ships as a remediation, NOT a license to under-provision. Plan: \"doubles transient storage and write load; treat §13.1 as a remediation, not a license to under-provision.\"\n\n**Remaining work**:\n- Empirical validation of the 2× storage + write load estimate under real corpora (varied doc sizes, write rates, settings complexity)\n- CLI schedule guidance: `miroir-ctl reshard --schedule-window off-peak` — refuses to start outside a named window unless `--force`\n\n## Why\n\nOperators will over-commit to resharding if the \"2× transient\" caveat turns out to be 3× or worse in practice. Real numbers prevent that.\n\n## Details\n\n**Test matrix**:\n| Doc size | Corpus | Write rate | RG | RF | Measured peak storage |\n|----------|--------|------------|----|----|-----------------------|\n| 1 KB | 10 GB | 100 dps | 2 | 1 | ? |\n| 10 KB | 100 GB | 1000 dps | 2 | 2 | ? |\n| 1 MB (blobs) | 1 TB | 10 dps | 2 | 1 | ? |\n\nPublish results in `docs/benchmarks/resharding-load.md`.\n\n**CLI window guard**: config knob `resharding.allowed_windows: [\"02:00-06:00 UTC\"]`. CLI refuses outside windows without `--force`.\n\n## Acceptance\n\n- [ ] Benchmark doc published with real numbers\n- [ ] CLI window guard implemented; integration test confirms rejection outside window\n- [ ] Benchmark run in Phase 9 performance suite as part of v1.0 validation","status":"closed","priority":3,"issue_type":"task","assignee":"bravo","created_at":"2026-04-18T21:49:47.828099118Z","created_by":"coding","updated_at":"2026-04-19T02:09:48.450456008Z","closed_at":"2026-04-19T02:09:48.450390357Z","close_reason":"P12.OP3 complete. All acceptance criteria verified:\n\n1. Benchmark doc (docs/benchmarks/resharding-load.md): Published with results for all 3 scenarios (1KB/10GB, 10KB/100GB, 1MB/1TB). Storage amplification confirmed at exactly 2.00x and dual-write amplification at exactly 2.00x across all scenarios.\n\n2. CLI schedule window guard: Implemented in miroir-ctl reshard command. Config knob resharding.allowed_windows restricts resharding to named windows. CLI refuses outside windows unless --force given.\n\n3. Integration tests (window_guard.rs): 4 tests all passing. 24 total resharding tests pass.\n\n4. Benchmark binary (reshard_load.rs): Full simulation using actual routing code, validates invariants.","source_repo":".","compaction_level":0,"original_size":0,"labels":["deferred","open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.3","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.828099118Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.4","title":"P12.OP4 Score normalization at scale — statistical validation of cross-shard comparability","description":"## What\n\nPlan §15 Open Problem #4: \"`_rankingScore` is comparable across shards only when index settings are identical.\" Settings divergence addressed by §13.5; remaining concern is statistical — do scores stay comparable when shards have very different document-count distributions?\n\n**Research work**:\n- Build a test corpus with intentionally skewed shard populations (one shard 100×, another shard 0.01× the median)\n- Submit identical queries; measure score distribution per shard\n- Assert: top-K merged ordering matches a ground-truth single-index version within some ε\n- If large ε, document + possibly introduce a score normalization pass\n\n## Why\n\nElasticsearch (plan research doc §1) hits this exactly: \"BM25 scoring depends on IDF, computed per shard by default using only that shard's local term statistics.\" Meilisearch uses its own ranking pipeline, but the same issue applies — local rank stats can drift from global on skewed shards.\n\n## Details\n\n**Ground truth**: single-index Meilisearch running the same queries against the same corpus.\n\n**Divergence metric**: Kendall τ between Miroir result ordering and single-index result ordering across 10k random queries.\n\n**If τ < 0.95 on average**: investigate whether a global IDF-style preflight is worth adding (plan research §1 \"`dfs_query_then_fetch`\" pattern).\n\n**Output**: `docs/research/score-normalization-at-scale.md`.\n\n## Acceptance\n\n- [ ] Benchmark corpus + query set published in `tests/benches/score-comparability/`\n- [ ] Results reported with confidence intervals\n- [ ] If τ < 0.95: follow-up bead created for a normalization pass\n- [ ] If τ ≥ 0.95: note-of-no-action in the bead's close comment","status":"in_progress","priority":3,"issue_type":"task","assignee":"alpha","created_at":"2026-04-18T21:49:47.849019120Z","created_by":"coding","updated_at":"2026-04-19T00:53:29.090661959Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.4","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.849019120Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.5","title":"P12.OP5 Dump import variants — enumerate what streaming mode can't handle","description":"## What\n\nPlan §15 Open Problem #5: §13.9 streaming routed dump import addresses the main case; broadcast mode retained as a fallback for dump variants Miroir cannot fully reconstruct via public API.\n\n**Remaining work**:\n- Identify and enumerate every dump variant streaming can't reconstruct\n- Either extend streaming to handle them OR document the fallback trigger clearly in `miroir-ctl dump import --help`\n\n## Why\n\n\"Can't reconstruct\" is vague — operators deserve concrete lists of what works and what doesn't. Without this, the `broadcast` fallback path is a bug waiting to happen.\n\n## Details\n\n**Potential failure modes to investigate**:\n- Dumps from older Meilisearch versions with pre-v1.37 schema\n- Dumps with custom keys (POST /keys) that have indexes list or actions not representable via public API\n- Dumps with snapshot-taken-mid-write where Miroir-injected `_miroir_shard` would conflict with an existing client field\n\n**Deliverable**: `docs/dump-import/compatibility-matrix.md` with columns:\n| Meilisearch version | Dump variant | Streaming works? | Broadcast needed? | Workaround |\n\n## Acceptance\n\n- [ ] Matrix published\n- [ ] Each \"broadcast needed\" row has a workaround or a link to an open enhancement bead\n- [ ] `miroir-ctl dump import` output references the matrix when falling back to broadcast","status":"closed","priority":3,"issue_type":"task","assignee":"bravo","created_at":"2026-04-18T21:49:47.884303207Z","created_by":"coding","updated_at":"2026-04-19T01:09:27.327131515Z","closed_at":"2026-04-19T01:09:27.327067549Z","close_reason":"Compatibility matrix published at docs/dump-import/compatibility-matrix.md\n\n- Matrix enumerates all dump variants that streaming mode can/cannot reconstruct\n- Each broadcast fallback row has workaround or enhancement bead link\n- CLI output reference section documents fallback message\n- Covers: version compatibility, field conflicts, EE features, snapshots, corrupted dumps","source_repo":".","compaction_level":0,"original_size":0,"labels":["failure-count:1","open-problem","phase-12","research"],"dependencies":[{"issue_id":"miroir-zc2.5","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.884303207Z","created_by":"coding","metadata":"{}","thread_id":""}]}
{"id":"miroir-zc2.6","title":"P12.OP6 arm64 support (deferred to v1.x+)","description":"## What\n\nPlan §15 Open Problem #6: \"Not planned for v0.x. Added when K8s ARM node support is required.\"\n\n**Future work when prioritized**:\n- Cross-compile `miroir-proxy` and `miroir-ctl` for `aarch64-unknown-linux-musl` in the CI pipeline\n- Docker image manifest list: `ghcr.io/jedarden/miroir:<version>` spans `linux/amd64` + `linux/arm64`\n- Helm chart: no changes (binary is arch-agnostic at the k8s layer)\n- Phase 9 CI: add arm64 test runs\n\n## Why\n\nARM node support is increasingly common (Hetzner Ampere, AWS Graviton, GCP Tau T2A, Rackspace Spot). But Miroir's fleet is currently all amd64 (iad-ci is amd64; ardenone cluster nodes are amd64). No current demand to justify the CI complexity.\n\nKeep this bead open as a placeholder; promote to in-progress when a concrete use case emerges.\n\n## Details\n\n**When ready**: the Argo Workflow `cargo-build` step needs a matrix over targets:\n```yaml\n- name: cargo-build\n container:\n args:\n - |\n rustup target add x86_64-unknown-linux-musl\n rustup target add aarch64-unknown-linux-musl\n apt-get install -qy musl-tools gcc-aarch64-linux-gnu\n cargo build --release --target x86_64-unknown-linux-musl -p miroir-proxy\n cargo build --release --target aarch64-unknown-linux-musl -p miroir-proxy\n ...\n```\n\nKaniko build needs `--customPlatform=linux/amd64,linux/arm64` or equivalent for multi-arch manifests.\n\n## Acceptance\n\n- [ ] Not to be closed until arm64 is a live deliverable\n- [ ] Cross-reference here when the priority flips","status":"in_progress","priority":4,"issue_type":"feature","assignee":"charlie","created_at":"2026-04-18T21:49:47.917666333Z","created_by":"coding","updated_at":"2026-04-19T00:58:19.767272778Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["open-problem","phase-12","roadmap"],"dependencies":[{"issue_id":"miroir-zc2.6","depends_on_id":"miroir-zc2","type":"parent-child","created_at":"2026-04-18T21:49:47.917666333Z","created_by":"coding","metadata":"{}","thread_id":""}]}

1
.needle-predispatch-sha
View file

@ -0,0 +1 @@
2b1ea87f3e20825ca10ab4a4bea83e1a30bb7800

2
docs/plan/plan.md
View file

@ -3726,7 +3726,7 @@ This is supported but **not** the recommended production topology — horizontal
These are documented constraints, not blockers. Initial release ships with known limitations.
1. **Shard migration write safety** — Dual-write during migration must not lose documents that arrive exactly at the migration cutover boundary. Requires careful sequencing: new writes go to both old and new nodes until the new node is confirmed complete, then old node stops receiving those shards. Race condition analysis needed before implementation. **Status:** Race window mitigated by the anti-entropy reconciler (§13.8); any document slipped by the cutover is caught and repaired on the next reconciliation pass.
1. **Shard migration write safety** — Dual-write during migration must not lose documents that arrive exactly at the migration cutover boundary. Requires careful sequencing: new writes go to both old and new nodes until the new node is confirmed complete, then old node stops receiving those shards. Race condition analysis needed before implementation. **Status:** Verified empirically via chaos tests (`cutover_race.rs`). Loss rate: 0/1M writes with AE on + delta pass on; 0/50K with delta pass alone. Hard refusal policy blocks `skip_delta_pass + anti_entropy_enabled=false`. See `docs/trade-offs.md` for full results and rationale.
2. **Task state HA** — SQLite is single-writer. Running 2 Miroir replicas requires Redis. A future enhancement is a lightweight Raft-based in-process consensus so Redis is not required for HA mode.