miroir/notes/miroir-cdo.md

Phase 1 — Core Routing: Final Verification Summary

Overview

Phase 1 implements the deterministic, coordination-free routing primitives that form the foundation for all distributed operations in Miroir. The implementation uses rendezvous hashing (HRW) with twox-hash, matching the algorithm Meilisearch Enterprise uses internally.

Implementation Summary

Files Implemented

• crates/miroir-core/src/router.rs — Rendezvous hashing, shard assignment, write targets, covering sets
• crates/miroir-core/src/topology.rs — Node registry, replica groups, health state machine
• crates/miroir-core/src/scatter.rs — Fan-out orchestration primitives (stubbed execution for Phase 2)
• crates/miroir-core/src/merger.rs — Result merge primitives (RRF and score-based strategies)

Definition of Done — All Verified ✅

1. Determinism — test_determinism, prop_determinism (1000 iterations, proptest with 1024 cases)

   • Same inputs always produce identical outputs
   • Verified across multiple runs

2. Minimal Reshuffling — test_reshuffle_bound_on_add, prop_reshuffle_bound_on_add

   • Adding a 4th node to 3-node group moves at most ~2 × (1/4) × 64 = 32 shard-node edges
   • Property-based tests verify bounds across 20-100 shards, 3-10 nodes, RF 1-3

3. Uniform Distribution — test_uniformity, prop_uniformity

   • 64 shards / 3 nodes / RF=1 → each node holds 17–26 shards (verified range)
   • Property-based tests verify even distribution across various configurations

4. RF Placement Stability — test_rf2_placement_stability, test_reshuffle_bound_on_remove

   • Top-RF placement changes minimally on add/remove
   • Verified with both unit and property-based tests

5. Write Targets — test_write_targets_returns_rg_x_rf_nodes, test_write_targets_one_per_group

   • Returns exactly RG × RF nodes, one from each replica group
   • Group isolation verified

6. Query Distribution — test_query_group_uniform_distribution

   • Chi-square test confirms even distribution (p < 0.05)
   • Round-robin by query counter

7. Covering Set — test_covering_set_covers_all_shards, test_covering_set_rotates_replicas

   • Returns exactly one node per shard within the chosen group
   • Intra-group replica rotation by query_seq verified

8. Merger — Comprehensive merge/facet/limit tests

   • Global sort by _rankingScore
   • Offset/limit handling
   • Facet aggregation (sum across shards)
   • estimatedTotalHits summation
   • _miroir_* field stripping
   • Both RRF and score-based merge strategies

9. Coverage — Line coverage for Phase 1 files

   • router.rs: 100% (65/65 lines)
   • topology.rs: 100% (130/130 lines)
   • merger.rs: 94.26% (148/157 lines)
   • scatter.rs: 77.29% (269/348 lines) — stub execution expected in Phase 2

Test Results

• Unit tests: 516 passed, 0 failed
• Property-based tests: All proptest cases pass (1024 cases per property)
• Integration: Scatter-gather end-to-end tests pass

Key Properties Verified

HRW Rendezvous Hashing

• Deterministic: Same (shard, node) → same score
• Minimal reshuffling on topology changes
• Group-scoped assignment prevents both replicas in same group
• Tie-breaking by node_id for determinism

Health State Machine

• Legal transitions: Joining → Active → Draining → Removed
• Failure paths: Active/Draining → Failed → Active
• Degraded state: Active ↔ Degraded
• Write eligibility respects shard migration state

Result Merging

• RRF (Reciprocal Rank Fusion) with k=60 default
• Score-based merge for global-IDF preflight (OP#4)
• Deterministic tie-breaking on primary key
• Stable serialization (BTreeMap for facets)

Notes

• Scatter execution stubs in scatter.rs are intentionally unimplemented pending Phase 2 wiring
• All core routing primitives are pure functions for easy testing
• The implementation is ready for Phase 2 (write path and read path integration)