Miroir

Multi-node Index Replication Orchestrator, Integrated Rebalancing

Miroir is a RAID-like sharding and high-availability layer for Meilisearch Community Edition. It stripes a large index across a fleet of small-RAM Meilisearch nodes with a configurable replication factor, fans out search queries across all shards, and rebalances shard assignments when nodes are added or removed — all while presenting a standard Meilisearch-compatible API to clients.

What is Meilisearch?

Meilisearch is an open-source, typo-tolerant search engine written in Rust. It is fast to set up, easy to integrate via REST API, and ships official SDKs for most languages. Meilisearch Community Edition (CE) is MIT-licensed and free to use; Meilisearch Enterprise adds commercial features including native multi-node sharding.

Miroir targets Meilisearch CE users who need horizontal scale without a commercial license.

The Problem

Meilisearch loads its entire index into memory-mapped LMDB files. A large index that exceeds a single server's available RAM cannot run on that server. The Enterprise Edition's native sharding is gated behind a commercial license. Miroir solves this without it.

How It Works

Client
  │
  ▼
Miroir Orchestrator
  ├── Write path: hash(doc_id) → assign to shard → write to R replicas
  ├── Read path:  scatter query to all shards → gather → merge ranked results
  └── Rebalance: on node add/remove → recompute assignments → migrate minimum shards

Meilisearch Nodes (N instances, each holding a subset of shards)
  node-0   node-1   node-2   ...   node-N

Miroir uses rendezvous hashing (HRW) — the same algorithm used internally by Meilisearch Enterprise — to assign documents to shards. This minimizes reshuffling when the cluster topology changes: only the shards that change owners need to be migrated. All Meilisearch nodes run unmodified Community Edition — no patches, no forks, no custom builds.

The logical shard count S is fixed at index creation time. The node count N is fully elastic: you can add or remove nodes at any time and trigger a rebalance to redistribute shard ownership.

Replication Factor

Analogous to software RAID — configurable per deployment:

RF	Redundancy	Node failures tolerated	Usable capacity
1	None (stripe only)	0	100% of fleet
2	One replica	1 per shard group	50% of fleet
3	Two replicas	2 per shard group	33% of fleet

Miroir also supports replica groups (RG) — independent sets of replicas that can serve read traffic in parallel, scaling read throughput horizontally beyond what a single RF set provides.

Crates

Miroir is a three-crate Rust workspace:

Crate	Description
miroir-core	Core sharding logic: rendezvous hash router, rebalancer, scatter-gather engine, result merger
miroir-proxy	HTTP proxy server that presents the Meilisearch-compatible REST API to clients
miroir-ctl	CLI management tool for cluster operations: node management, rebalancing, status

Static binaries compiled against musl libc; Docker images are built from scratch for minimal attack surface.

Quick Start

Get Miroir running locally in 5 minutes with Docker Compose.

examples/docker-compose-dev.yml

services:
  meili-0:
    image: getmeili/meilisearch:v1.37.0
    environment:
      - MEILI_MASTER_KEY=dev-node-key
  meili-1:
    image: getmeili/meilisearch:v1.37.0
    environment:
      - MEILI_MASTER_KEY=dev-node-key
  meili-2:
    image: getmeili/meilisearch:v1.37.0
    environment:
      - MEILI_MASTER_KEY=dev-node-key
  miroir:
    build:
      context: ..
      dockerfile: Dockerfile
    image: miroir-dev:latest
    environment:
      - MIROIR_MASTER_KEY=dev-key
      - MIROIR_NODE_MASTER_KEY=dev-node-key
    ports:
      - "7700:7700"
    volumes:
      - ./examples/dev-config.yaml:/etc/miroir/config.yaml:ro

examples/dev-config.yaml

master_key: "dev-key"
node_master_key: "dev-node-key"
shards: 16
replication_factor: 1
replica_groups: 1
nodes:
  - id: "meili-0"
    address: "http://meili-0:7700"
    replica_group: 0
  - id: "meili-1"
    address: "http://meili-1:7700"
    replica_group: 0
  - id: "meili-2"
    address: "http://meili-2:7700"
    replica_group: 0
task_store:
  backend: sqlite
  path: /data/miroir-tasks.db
server:
  port: 7700

Running

# Clone the repository
git clone https://github.com/jedarden/miroir.git
cd miroir

# Start the development stack (3 Meilisearch nodes + 1 Miroir orchestrator)
docker compose -f examples/docker-compose-dev.yml up -d

# Verify health
curl http://localhost:7700/health
# Expected: {"status":"available"}

# Index documents (standard Meilisearch API — no changes needed)
curl -X POST http://localhost:7700/indexes/movies/documents \
  -H "Authorization: Bearer dev-key" \
  -H "Content-Type: application/json" \
  -d '[{"id":1,"title":"The Matrix"},{"id":2,"title":"Inception"}]'

# Search
curl -X POST http://localhost:7700/indexes/movies/search \
  -H "Authorization: Bearer dev-key" \
  -H "Content-Type: application/json" \
  -d '{"q":"matrix"}'

# Teardown (removes containers and volumes)
docker compose -f examples/docker-compose-dev.yml down -v

See examples/README.md for more details on the development stack, configuration options, and troubleshooting.

miroir-ctl

miroir-ctl is the CLI management tool for operating a Miroir cluster.

# Cluster status — node health, shard distribution, replication state
miroir-ctl status

# List nodes
miroir-ctl nodes list

# Add a node to the cluster (does not trigger rebalance automatically)
miroir-ctl nodes add --id meili-3 --address http://meili-3:7700 --group 0

# Remove a node — migrates its shards away before removal
miroir-ctl nodes remove meili-1

# Trigger a rebalance (after adding/removing nodes)
miroir-ctl rebalance

# Watch rebalance progress
miroir-ctl rebalance status --watch

SDK Configuration

Miroir is a drop-in replacement at the network layer. Migrating from a single Meilisearch node requires only changing the endpoint URL. All index operations, document CRUD, and search queries work without modification.

Python

# Before — single-node Meilisearch
client = meilisearch.Client('https://old-meili.example.com', 'api-key')

# After — Miroir
client = meilisearch.Client('https://search.example.com', 'miroir-master-key')

TypeScript / JavaScript

// Before — single-node Meilisearch
const client = new MeiliSearch({
  host: 'https://old-meili.example.com',
  apiKey: 'api-key'
})

// After — Miroir
const client = new MeiliSearch({
  host: 'https://search.example.com',
  apiKey: 'miroir-master-key'
})

Go

// Before — single-node Meilisearch
client := meilisearch.NewClient(meilisearch.ClientConfig{
    Host:   "https://old-meili.example.com",
    APIKey: "api-key",
})

// After — Miroir
client := meilisearch.NewClient(meilisearch.ClientConfig{
    Host:   "https://search.example.com",
    APIKey: "miroir-master-key",
})

That's it — no other code changes required. Miroir presents the same Meilisearch-compatible API surface to all official SDKs.

Feature Matrix

Miroir implements 21 advanced capabilities (plan §13) that sit entirely within the orchestrator layer. Every Meilisearch node runs unmodified Community Edition — no patches, no forks, no custom builds.

Capability	Description	Default
§13.1 Online resharding	Change shard count without reindex via shadow index	on
§13.2 Hedged requests	Tail-latency mitigation via duplicate requests to alternate replicas	on
§13.3 Adaptive replica selection	EWMA-based routing to lowest-latency nodes	on
§13.4 Shard-aware query planner	Narrow fan-out for PK-constrained searches	on
§13.5 Two-phase settings broadcast	Atomic settings changes with verification	on
§13.6 Read-your-writes	Session pinning for immediate consistency	on
§13.7 Atomic index aliases	Blue-green reindexing and multi-target aliases	on
§13.8 Anti-entropy reconciler	Continuous shard repair and drift detection	on
§13.9 Streaming dump import	Route documents during import (no broadcast)	on
§13.10 Idempotency keys	Request deduplication and query coalescing	on
§13.11 Multi-search	Batch API for multiple queries in one round-trip	on
§13.12 Vector + hybrid search	Over-fetch with RRF/convex merging for correct global ranking	on
§13.13 CDC stream	Change data capture to webhook/NATS/Kafka/internal queue	on
§13.14 Document TTL	Automatic expiration with background sweeper	on
§13.15 Tenant affinity	Route tenant queries to dedicated replica groups	on
§13.16 Traffic shadow	Async request tee to shadow cluster with diff analysis	on
§13.17 ILM	Rolling time-series indexes with rollover policies	on
§13.18 Canary queries	Synthetic queries with golden assertions for relevance testing	on
§13.19 Admin Web UI	Embedded SPA for topology, config, query debugging, operations	on
§13.20 Query explain	Debug routing decisions and warnings without executing	on
§13.21 End-user Search UI	Embedded instant-search SPA with facets, keyboard nav, i18n	on

See docs/plan/plan.md#13-advanced-capabilities for detailed design of each capability.

Production Deployment

For production deployments, see the Deployment Sizing Guide to determine orchestrator pod count and task store configuration based on your corpus size and query throughput.

When to use

• Multi-pod with Redis — Recommended for production. Horizontal scaling with 2+ orchestrator pods delivers fault tolerance (zero-downtime rollouts, pod-loss survival) and scales query throughput via HPA. See Deployment Sizing Guide.

• Single oversized pod — Supported for dev clusters, very small deployments, or constrained environments. A single pod at 4 vCPU / 8 GB is validated but loses HA benefits (no zero-downtime rollouts, no pod-loss survival). See Single-Pod Mode.

• Large index sharding — When a single Meilisearch node cannot fit your corpus in RAM, Miroir stripes it across multiple nodes with configurable replication factor.

Additional production resources:

• Production Deployment Guide — Operational considerations, monitoring, and troubleshooting
• Per-Feature Scaling Behavior — Which features need Redis, work queues, or nothing
• Versioning Policy — Backward compatibility commitments and upgrade guidance

Stability

Miroir is currently in active development (v0.x). The API, configuration schema, and CLI flags may change between minor versions. Starting with v1.0, the project provides backward-compatibility commitments for:

• The Meilisearch-compatible API layer
• The miroir-ctl CLI interface
• The config file schema
• The Helm chart values

See docs/versioning-policy.md for the full versioning policy, including what constitutes a breaking change and the deprecation process.

Documentation

• Design Plan — Complete architecture, protocol, and capability specifications
• CHANGELOG.md — Release notes and version history
• API Compatibility — Meilisearch API coverage and extensions
• Helm Chart — Production deployment on Kubernetes
• Deployment Guides — Production setup, sizing, and operational considerations
• Migration Runbook — Paths from single-node Meilisearch to Miroir
• Troubleshooting Guide — Common issues and diagnostic playbook

Community

• Issues — Bug reports and feature requests
• Discussions — Q&A and design discussions
• Contributing — Development workflow and code submission guidelines

License

MIT License — see LICENSE for details.