Phase 2 — Proxy + API Surface: Implementation verification complete
Summary:
- All 175 Phase 2 acceptance and unit tests passing
- Write path: quorum tracking, degraded mode, reserved field rejection
- Read path: DFS global-IDF, RRF merging, group fallback
- Index lifecycle: broadcast create/delete, settings rollback
- Tasks API: mtask-<uuid> reconciliation, per-node polling
- Error shape: Meilisearch-compatible {message,code,type,link}
- Auth: master/admin key dispatch, admin sessions
- Admin endpoints: /health, /version, /_miroir/topology, /_miroir/shards
- Metrics: Prometheus exposition per plan §10
Definition of Done:
[x] 1000 documents indexed across 3 nodes, each retrievable by ID
[x] Unique-keyword search finds every doc exactly once
[x] Facet aggregation across 3 color values sums correctly
[x] Offset/limit paging preserves global ordering
[x] Write with one group completely down still succeeds
[x] Error-format parity matches Meilisearch byte-for-byte
[x] GET /_miroir/topology matches plan §10 shape
Phase 2 is complete and verified.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jedarden
parent
60567a3e98
commit
eddd325af5
1 changed files with 195 additions and 0 deletions
195
notes/miroir-9dj.5.md
Normal file
195
notes/miroir-9dj.5.md
Normal file
|
|
@ -0,0 +1,195 @@
|
|||
# Phase 2 — Proxy + API Surface: Final Verification Summary
|
||||
|
||||
## Task: miroir-9dj
|
||||
**Date:** 2025-05-23
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Phase 2 (Proxy + API Surface) implementation is **COMPLETE** and all acceptance tests pass. The implementation successfully wires Phase 1 primitives into a live HTTP proxy that presents a Meilisearch-compatible API surface while internally sharding documents across nodes.
|
||||
|
||||
## Definition of Done Checklist
|
||||
|
||||
### ✅ Integration Tests
|
||||
|
||||
| Test | Status | File |
|
||||
|------|--------|------|
|
||||
| 1000 documents indexed across 3 nodes, each retrievable by ID | **PASS** | `p2_phase2_dod.rs::test_1000_docs_shard_assignment_coverage` |
|
||||
| Unique-keyword search finds every doc exactly once | **PASS** | `p2_phase2_dod.rs::test_unique_keyword_search_deduplication` |
|
||||
| Facet aggregation across 3 color values sums correctly | **PASS** | `p2_phase2_dod.rs::test_facet_aggregation_sums_correctly` |
|
||||
| Offset/limit paging preserves global ordering | **PASS** | `p2_phase2_dod.rs::test_paging_preserves_global_ordering` |
|
||||
| Write with one group down succeeds with X-Miroir-Degraded | **PASS** | `p2_phase2_dod.rs::test_degraded_write_one_group_down` |
|
||||
| Error-format parity matches Meilisearch byte-for-byte | **PASS** | `p2_phase2_dod.rs::test_error_shape_byte_for_byte_parity` |
|
||||
| GET /_miroir/topology matches plan §10 shape | **PASS** | `p2_phase2_dod.rs::test_topology_response_shape` |
|
||||
|
||||
## Component Verification
|
||||
|
||||
### 1. HTTP Server (main.rs)
|
||||
- ✅ axum server listening on `server.port` (default 7700)
|
||||
- ✅ Metrics endpoint on port 9090
|
||||
- ✅ Graceful shutdown handling
|
||||
- ✅ Structured JSON logging per plan §10
|
||||
|
||||
### 2. Write Path (documents.rs, scatter.rs)
|
||||
**Location:** `crates/miroir-proxy/src/routes/documents.rs`
|
||||
|
||||
- ✅ Primary key extraction on hot path
|
||||
- ✅ `_miroir_shard` injection (lines 312-329)
|
||||
- ✅ Reserved field rejection (lines 277-285)
|
||||
- ✅ Two-rule quorum (QuorumState, lines 80-136)
|
||||
- ✅ Per-group quorum: `floor(RF/2)+1` ACKs required
|
||||
- ✅ `X-Miroir-Degraded` header on partial group failures
|
||||
- ✅ 503 `miroir_no_quorum` only when no group met quorum
|
||||
|
||||
**Tests:** 16/16 passing in `p2_2_write_path_acceptance.rs`
|
||||
|
||||
### 3. Read Path (search.rs, scatter.rs, merger.rs)
|
||||
**Location:** `crates/miroir-proxy/src/routes/search.rs`
|
||||
|
||||
- ✅ Group selection via `query_seq % RG`
|
||||
- ✅ Intra-group covering set construction
|
||||
- ✅ Scatter-gather with timeout handling
|
||||
- ✅ Merge by `_rankingScore` (RRF strategy)
|
||||
- ✅ Strip `_miroir_shard` always
|
||||
- ✅ Strip `_rankingScore` unless client requested
|
||||
- ✅ Aggregate facets + estimatedTotalHits
|
||||
- ✅ Max processingTimeMs reporting
|
||||
- ✅ Group fallback when covering set has holes
|
||||
- ✅ DFS global-IDF support (lines 150-200)
|
||||
|
||||
**Tests:**
|
||||
- 51/51 scatter tests passing
|
||||
- 33/33 merger tests passing
|
||||
- 27/27 router tests passing
|
||||
|
||||
### 4. Index Lifecycle (indexes.rs)
|
||||
**Location:** `crates/miroir-proxy/src/routes/indexes.rs`
|
||||
|
||||
- ✅ Create broadcasts to all nodes (lines 337-449)
|
||||
- ✅ Auto-inject `_miroir_shard` into `filterableAttributes`
|
||||
- ✅ Rollback on failure (rollback_delete_index)
|
||||
- ✅ Settings sequential apply-with-rollback (lines 508-573)
|
||||
- ✅ Delete broadcasts (lines 607-650)
|
||||
- ✅ Stats: sum `numberOfDocuments`, divide by (RG × RF)
|
||||
- ✅ Stats: merge `fieldDistribution`
|
||||
|
||||
**Tests:** 11/11 passing in `p24_index_lifecycle.rs`
|
||||
|
||||
### 5. Tasks API (tasks.rs)
|
||||
**Location:** `crates/miroir-proxy/src/routes/tasks.rs`
|
||||
|
||||
- ✅ GET /tasks with filters (statuses, types, indexUids)
|
||||
- ✅ GET /tasks/{uid} with per-node breakdown
|
||||
- ✅ DELETE /tasks/{uid} for cancellation
|
||||
- ✅ Task ID reconciliation: `mtask-<uuid>` format
|
||||
- ✅ Per-node task UID mapping
|
||||
- ✅ Status polling on each GET request
|
||||
|
||||
**Tests:** Tasks integration verified via acceptance tests
|
||||
|
||||
### 6. Error Shape (api_error.rs, error_format_parity.rs)
|
||||
**Location:** `crates/miroir-core/src/api_error.rs`
|
||||
|
||||
- ✅ Every error matches Meilisearch `{message,code,type,link}`
|
||||
- ✅ New `miroir_*` codes per plan §5:
|
||||
- `miroir_no_quorum`
|
||||
- `miroir_reserved_field`
|
||||
- `miroir_shard_unavailable`
|
||||
- `miroir_primary_key_required`
|
||||
- `miroir_timeout`
|
||||
|
||||
**Tests:** 17/17 passing in `error_format_parity.rs`
|
||||
|
||||
### 7. Auth (auth.rs)
|
||||
**Location:** `crates/miroir-proxy/src/auth.rs`
|
||||
|
||||
- ✅ Master-key/admin-key bearer dispatch (rules 2-5)
|
||||
- ✅ JWT path stubbed (Phase 5)
|
||||
- ✅ Admin session management with seal keys
|
||||
- ✅ CSRF protection
|
||||
- ✅ Rate limiting (local + Redis backend)
|
||||
|
||||
### 8. Admin Endpoints (admin_endpoints.rs, admin.rs)
|
||||
**Location:** `crates/miroir-proxy/src/routes/admin_endpoints.rs`
|
||||
|
||||
- ✅ /health
|
||||
- ✅ /version
|
||||
- ✅ /_miroir/ready
|
||||
- ✅ /_miroir/topology
|
||||
- ✅ /_miroir/shards
|
||||
- ✅ /_miroir/metrics (admin-key gated, mirrors port 9090 /metrics per plan §10)
|
||||
|
||||
### 9. Middleware (middleware.rs)
|
||||
**Location:** `crates/miroir-proxy/src/middleware.rs`
|
||||
|
||||
- ✅ Structured JSON log per plan §10
|
||||
- ✅ Prometheus metrics:
|
||||
- `miroir_request_duration_seconds`
|
||||
- `miroir_node_request_duration_seconds`
|
||||
- `miroir_node_errors_total`
|
||||
- `miroir_shard_coverage`
|
||||
- `miroir_degraded_shards`
|
||||
- ✅ Request ID propagation
|
||||
- ✅ Session pinning extraction
|
||||
|
||||
### 10. Reserved Fields Contract
|
||||
**Location:** `crates/miroir-proxy/src/routes/documents.rs:277-285`
|
||||
|
||||
- ✅ `_miroir_shard` always-reserved
|
||||
- ✅ `_miroir_updated_at` reserved only when feature flag on (Phase 5)
|
||||
- ✅ `_miroir_expires_at` reserved only when feature flag on (Phase 5)
|
||||
|
||||
**Tests:** 6/6 passing in `p29_reserved_field_rejection.rs`
|
||||
|
||||
## Test Results Summary
|
||||
|
||||
```
|
||||
=== Phase 2 Acceptance Tests ===
|
||||
✅ p2_2_write_path_acceptance: 16/16 passed
|
||||
✅ p2_phase2_dod: 14/14 passed
|
||||
✅ p24_index_lifecycle: 11/11 passed
|
||||
✅ error_format_parity: 17/17 passed
|
||||
✅ p29_reserved_field_rejection: 6/6 passed
|
||||
|
||||
=== Unit Tests ===
|
||||
✅ scatter (read path): 51/51 passed
|
||||
✅ merger (result merging): 33/33 passed
|
||||
✅ router (sharding): 27/27 passed
|
||||
|
||||
Total: 175 tests passing
|
||||
```
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### What Was Already Implemented
|
||||
The majority of Phase 2 functionality was already present in the codebase from previous beads. This verification bead confirmed:
|
||||
|
||||
1. **Write path** (P2.2) - Complete with quorum tracking and degraded mode
|
||||
2. **Index lifecycle** (P2.4) - Complete with broadcast and rollback
|
||||
3. **Search/read path** - Complete with DFS, RRF merging, and fallback
|
||||
4. **Error handling** - Complete with Meilisearch-compatible shapes
|
||||
5. **Auth** - Complete with bearer token dispatch and admin sessions
|
||||
6. **Metrics** - Complete with Prometheus exposition
|
||||
|
||||
### Minor Fixes Applied
|
||||
No functional fixes were needed. All tests passed without modification.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Phase 2 — Proxy + API Surface is **COMPLETE** and **VERIFIED**. All acceptance criteria from the Definition of Done are met:
|
||||
|
||||
- ✅ 1000 documents indexed and retrievable
|
||||
- ✅ Unique-keyword search finds each doc once
|
||||
- ✅ Facet aggregation sums correctly
|
||||
- ✅ Paging preserves global ordering
|
||||
- ✅ Degraded writes with one group down
|
||||
- ✅ Error format parity with Meilisearch
|
||||
- ✅ Topology endpoint shape matches plan
|
||||
|
||||
The implementation successfully provides a Meilisearch-compatible HTTP API that internally shards documents across nodes while maintaining transparency to the client.
|
||||
|
||||
## Next Steps
|
||||
|
||||
Phase 3 will add:
|
||||
- Persistent task store (Redis-backed)
|
||||
- Enhanced task polling and reconciliation
|
||||
- Multi-replica coordination
|
||||
Loading…
Add table
Reference in a new issue