jedarden/miroir
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

P6.10 Wire §14.8 resource-aware config defaults into Rust + values.yaml

Browse source 
Complete acceptance criteria:
- Each §14.8 key present in crates/miroir-core/src/config/ with documented default
- charts/miroir/values.yaml exposes the same keys with identical defaults
- values.schema.json accepts documented ranges; cross-field validation in _helpers.tpl
- K8s resources block matches §14.8 (500m/2000m CPU, 1Gi/3584Mi mem)
- Unit test: section_14_8_defaults_match compares Config::default() to §14.8 reference
- Drift guard: doc-test at top of MiroirConfig struct validates defaults

All defaults sized for 2 vCPU / 3.75 GB envelope per plan §14.8.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jedarden 2026-05-20 07:34:33 -04:00
parent b9e92e18e2
commit d8d81a12a8
5 changed files with 354 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

70
charts/miroir/templates/_helpers.tpl
View file

@ -133,26 +133,57 @@ rebalancer:
server:
port: 7700
bind: "0.0.0.0"
max_body_bytes: 104857600
max_concurrent_requests: 500
request_timeout_ms: 30000
max_body_bytes: {{ .Values.miroir.server.max_body_bytes | default 104857600 }}
max_concurrent_requests: {{ .Values.miroir.server.max_concurrent_requests | default 500 }}
request_timeout_ms: {{ .Values.miroir.server.request_timeout_ms | default 30000 }}
connection_pool_per_node:
max_idle: 32
max_total: 128
idle_timeout_s: 60
max_idle: {{ .Values.miroir.connection_pool_per_node.max_idle | default 32 }}
max_total: {{ .Values.miroir.connection_pool_per_node.max_total | default 128 }}
idle_timeout_s: {{ .Values.miroir.connection_pool_per_node.idle_timeout_s | default 60 }}
task_registry:
cache_size: 10000
redis_pool_max: 50
ttl_seconds: 604800
prune_interval_s: 300
prune_batch_size: 10000
cache_size: {{ .Values.miroir.task_registry.cache_size | default 10000 }}
redis_pool_max: {{ .Values.miroir.task_registry.redis_pool_max | default 50 }}
ttl_seconds: {{ .Values.miroir.task_registry.ttl_seconds | default 604800 }}
prune_interval_s: {{ .Values.miroir.task_registry.prune_interval_s | default 300 }}
prune_batch_size: {{ .Values.miroir.task_registry.prune_batch_size | default 10000 }}
idempotency:
enabled: {{ .Values.miroir.idempotency.enabled | default true }}
max_cached_keys: {{ .Values.miroir.idempotency.max_cached_keys | default 1000000 }}
ttl_seconds: {{ .Values.miroir.idempotency.ttl_seconds | default 86400 }}
session_pinning:
enabled: {{ .Values.miroir.session_pinning.enabled | default true }}
ttl_seconds: {{ .Values.miroir.session_pinning.ttl_seconds | default 900 }}
max_sessions: {{ .Values.miroir.session_pinning.max_sessions | default 100000 }}
wait_strategy: {{ .Values.miroir.session_pinning.wait_strategy | default "block" }}
max_wait_ms: {{ .Values.miroir.session_pinning.max_wait_ms | default 5000 }}
query_coalescing:
enabled: {{ .Values.miroir.query_coalescing.enabled | default true }}
window_ms: {{ .Values.miroir.query_coalescing.window_ms | default 50 }}
max_subscribers: {{ .Values.miroir.query_coalescing.max_subscribers | default 1000 }}
max_pending_queries: {{ .Values.miroir.query_coalescing.max_pending_queries | default 10000 }}
anti_entropy:
enabled: {{ .Values.miroir.anti_entropy.enabled | default true }}
schedule: {{ .Values.miroir.anti_entropy.schedule | default "every 6h" }}
shards_per_pass: {{ .Values.miroir.anti_entropy.shards_per_pass | default 0 }}
max_read_concurrency: {{ .Values.miroir.anti_entropy.max_read_concurrency | default 2 }}
fingerprint_batch_size: {{ .Values.miroir.anti_entropy.fingerprint_batch_size | default 1000 }}
auto_repair: {{ .Values.miroir.anti_entropy.auto_repair | default true }}
updated_at_field: {{ .Values.miroir.anti_entropy.updated_at_field | default "_miroir_updated_at" }}
resharding:
enabled: {{ .Values.miroir.resharding.enabled | default true }}
backfill_concurrency: {{ .Values.miroir.resharding.backfill_concurrency | default 4 }}
backfill_batch_size: {{ .Values.miroir.resharding.backfill_batch_size | default 1000 }}
throttle_docs_per_sec: {{ .Values.miroir.resharding.throttle_docs_per_sec | default 0 }}
verify_before_swap: {{ .Values.miroir.resharding.verify_before_swap | default true }}
retain_old_index_hours: {{ .Values.miroir.resharding.retain_old_index_hours | default 48 }}
allowed_windows: {{ .Values.miroir.resharding.allowed_windows | default list | toJson }}
peer_discovery:
service_name: {{ .Release.Name }}-headless
refresh_interval_s: 15
service_name: {{ .Values.miroir.peer_discovery.service_name | default .Release.Name }}-headless
refresh_interval_s: {{ .Values.miroir.peer_discovery.refresh_interval_s | default 15 }}
leader_election:
enabled: true
lease_ttl_s: 10
renew_interval_s: 3
enabled: {{ .Values.miroir.leader_election.enabled | default true }}
lease_ttl_s: {{ .Values.miroir.leader_election.lease_ttl_s | default 10 }}
renew_interval_s: {{ .Values.miroir.leader_election.renew_interval_s | default 3 }}
hpa:
enabled: {{ .Values.hpa.enabled | default false }}
tracing:
@ -237,4 +268,11 @@ Rendered as an empty ConfigMap; fails template rendering on invalid config.
{{- end -}}
{{- end -}}
{{- end -}}
{{- if .Values.miroir.leader_election -}}
{{- if and (hasKey .Values.miroir.leader_election "lease_ttl_s") (hasKey .Values.miroir.leader_election "renew_interval_s") -}}
{{- if le (int .Values.miroir.leader_election.lease_ttl_s) (int .Values.miroir.leader_election.renew_interval_s) -}}
{{- fail (printf "leader_election.lease_ttl_s (%d) must be greater than leader_election.renew_interval_s (%d); otherwise the lease expires before it can be renewed" (int .Values.miroir.leader_election.lease_ttl_s) (int .Values.miroir.leader_election.renew_interval_s)) -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}

95
charts/miroir/values.schema.json
View file

@ -62,6 +62,98 @@
}
}
}
},
"server": {
"type": "object",
"properties": {
"max_body_bytes": { "type": "integer", "minimum": 0 },
"max_concurrent_requests": { "type": "integer", "minimum": 1 },
"request_timeout_ms": { "type": "integer", "minimum": 0 }
}
},
"connection_pool_per_node": {
"type": "object",
"properties": {
"max_idle": { "type": "integer", "minimum": 0 },
"max_total": { "type": "integer", "minimum": 1 },
"idle_timeout_s": { "type": "integer", "minimum": 1 }
}
},
"task_registry": {
"type": "object",
"properties": {
"cache_size": { "type": "integer", "minimum": 0 },
"redis_pool_max": { "type": "integer", "minimum": 1 },
"ttl_seconds": { "type": "integer", "minimum": 1 },
"prune_interval_s": { "type": "integer", "minimum": 1 },
"prune_batch_size": { "type": "integer", "minimum": 1 }
}
},
"idempotency": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" },
"max_cached_keys": { "type": "integer", "minimum": 0 },
"ttl_seconds": { "type": "integer", "minimum": 1 }
}
},
"session_pinning": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" },
"ttl_seconds": { "type": "integer", "minimum": 1 },
"max_sessions": { "type": "integer", "minimum": 0 },
"wait_strategy": { "type": "string", "enum": ["block", "route_pin"] },
"max_wait_ms": { "type": "integer", "minimum": 0 }
}
},
"query_coalescing": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" },
"window_ms": { "type": "integer", "minimum": 0 },
"max_subscribers": { "type": "integer", "minimum": 1 },
"max_pending_queries": { "type": "integer", "minimum": 1 }
}
},
"anti_entropy": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" },
"schedule": { "type": "string" },
"shards_per_pass": { "type": "integer", "minimum": 0 },
"max_read_concurrency": { "type": "integer", "minimum": 1 },
"fingerprint_batch_size": { "type": "integer", "minimum": 1 },
"auto_repair": { "type": "boolean" },
"updated_at_field": { "type": "string" }
}
},
"resharding": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" },
"backfill_concurrency": { "type": "integer", "minimum": 1 },
"backfill_batch_size": { "type": "integer", "minimum": 1 },
"throttle_docs_per_sec": { "type": "integer", "minimum": 0 },
"verify_before_swap": { "type": "boolean" },
"retain_old_index_hours": { "type": "integer", "minimum": 0 },
"allowed_windows": { "type": "array", "items": { "type": "string" } }
}
},
"peer_discovery": {
"type": "object",
"properties": {
"service_name": { "type": "string" },
"refresh_interval_s": { "type": "integer", "minimum": 1 }
}
},
"leader_election": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" },
"lease_ttl_s": { "type": "integer", "minimum": 1 },
"renew_interval_s": { "type": "integer", "minimum": 1 }
}
}
}
},
@ -415,6 +507,9 @@
},
{
"description": "Rule 5: scoped_key_rotate_before_expiry_days must be strictly less than scoped_key_max_age_days (enforced at render time via _helpers.tpl since JSON Schema draft-7 cannot compare sibling properties)"
},
{
"description": "Rule 6: leader_election.lease_ttl_s must be greater than leader_election.renew_interval_s (enforced at render time via _helpers.tpl since JSON Schema draft-7 cannot compare sibling properties)"
}
]
}

62
charts/miroir/values.yaml
View file

@ -46,11 +46,11 @@ miroir:
podLabels: {}
resources:
limits:
cpu: 1000m
memory: 1Gi
cpu: 2000m
memory: 3584Mi # 3.5 GiB (leaves headroom under 3.75 GB envelope)
requests:
cpu: 250m
memory: 512Mi
cpu: 500m
memory: 1Gi
nodeSelector: {}
tolerations: []
affinity: {}
@ -64,6 +64,60 @@ miroir:
pvc_size: 10Gi
pvc_storage_class: ""
memory_bytes: 67108864
# §14.8 Resource-aware configuration defaults
# All defaults sized for 2 vCPU / 3.75 GB envelope
server:
max_body_bytes: 104857600 # 100 MiB per request
max_concurrent_requests: 500
request_timeout_ms: 30000
connection_pool_per_node:
max_idle: 32
max_total: 128
idle_timeout_s: 60
task_registry:
cache_size: 10000
redis_pool_max: 50
ttl_seconds: 604800 # 7 days
prune_interval_s: 300
prune_batch_size: 10000
idempotency:
enabled: true
max_cached_keys: 1000000 # ~100 MB
ttl_seconds: 86400
session_pinning:
enabled: true
ttl_seconds: 900
max_sessions: 100000 # ~50 MB
wait_strategy: block # block | route_pin
max_wait_ms: 5000
query_coalescing:
enabled: true
window_ms: 50
max_subscribers: 1000
max_pending_queries: 10000
anti_entropy:
enabled: true
schedule: every 6h
shards_per_pass: 0 # 0 = all shards
max_read_concurrency: 2
fingerprint_batch_size: 1000
auto_repair: true
updated_at_field: _miroir_updated_at
resharding:
enabled: true
backfill_concurrency: 4
backfill_batch_size: 1000
throttle_docs_per_sec: 0 # 0 = no throttle
verify_before_swap: true
retain_old_index_hours: 48
allowed_windows: [] # e.g., ["22:00-04:00 UTC"]
peer_discovery:
service_name: miroir-headless
refresh_interval_s: 15
leader_election:
enabled: true # auto-true when replicas > 1
lease_ttl_s: 10
renew_interval_s: 3
search_ui:
enabled: true

83
crates/miroir-core/src/config.rs
View file

@ -11,6 +11,38 @@ pub use advanced::{SearchUiConfig, CspOverridesConfig};
use serde::{Deserialize, Serialize};
/// Top-level configuration matching plan §4 YAML schema under `miroir:`.
///
/// # Drift Guard (§14.8)
///
/// The default values for resource-sensitive knobs are sized for the
/// 2 vCPU / 3.75 GB envelope. This doc-test ensures the Rust defaults
/// match the §14.8 reference fixture:
///
/// ```
/// use miroir_core::config::MiroirConfig;
/// let cfg = MiroirConfig::default();
/// assert_eq!(cfg.server.max_body_bytes, 104_857_600);
/// assert_eq!(cfg.server.max_concurrent_requests, 500);
/// assert_eq!(cfg.server.request_timeout_ms, 30_000);
/// assert_eq!(cfg.connection_pool_per_node.max_idle, 32);
/// assert_eq!(cfg.connection_pool_per_node.max_total, 128);
/// assert_eq!(cfg.connection_pool_per_node.idle_timeout_s, 60);
/// assert_eq!(cfg.task_registry.cache_size, 10_000);
/// assert_eq!(cfg.task_registry.redis_pool_max, 50);
/// assert_eq!(cfg.idempotency.max_cached_keys, 1_000_000);
/// assert_eq!(cfg.idempotency.ttl_seconds, 86_400);
/// assert_eq!(cfg.session_pinning.max_sessions, 100_000);
/// assert_eq!(cfg.query_coalescing.max_subscribers, 1_000);
/// assert_eq!(cfg.query_coalescing.max_pending_queries, 10_000);
/// assert_eq!(cfg.anti_entropy.max_read_concurrency, 2);
/// assert_eq!(cfg.anti_entropy.fingerprint_batch_size, 1_000);
/// assert_eq!(cfg.resharding.backfill_concurrency, 4);
/// assert_eq!(cfg.resharding.backfill_batch_size, 1_000);
/// assert_eq!(cfg.peer_discovery.service_name, "miroir-headless");
/// assert_eq!(cfg.peer_discovery.refresh_interval_s, 15);
/// assert_eq!(cfg.leader_election.lease_ttl_s, 10);
/// assert_eq!(cfg.leader_election.renew_interval_s, 3);
/// ```
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct MiroirConfig {
@ -643,4 +675,55 @@ task_store:
let cfg = MiroirConfig::load_from(std::path::Path::new("/nonexistent/miroir.yaml"));
assert!(cfg.is_err());
}
/// Drift guard: ensure Config::default() matches §14.8 reference defaults.
///
/// This test serializes the default config and compares key §14.8 knobs
/// against the reference fixture. If this fails, either the Rust defaults
/// have drifted from §14.8, or the plan needs updating.
#[test]
fn section_14_8_defaults_match() {
let cfg = MiroirConfig::default();
// §14.8 server defaults
assert_eq!(cfg.server.max_body_bytes, 104_857_600, "server.max_body_bytes");
assert_eq!(cfg.server.max_concurrent_requests, 500, "server.max_concurrent_requests");
assert_eq!(cfg.server.request_timeout_ms, 30_000, "server.request_timeout_ms");
// §14.8 connection_pool_per_node defaults
assert_eq!(cfg.connection_pool_per_node.max_idle, 32, "connection_pool_per_node.max_idle");
assert_eq!(cfg.connection_pool_per_node.max_total, 128, "connection_pool_per_node.max_total");
assert_eq!(cfg.connection_pool_per_node.idle_timeout_s, 60, "connection_pool_per_node.idle_timeout_s");
// §14.8 task_registry defaults
assert_eq!(cfg.task_registry.cache_size, 10_000, "task_registry.cache_size");
assert_eq!(cfg.task_registry.redis_pool_max, 50, "task_registry.redis_pool_max");
// §14.8 idempotency defaults
assert_eq!(cfg.idempotency.max_cached_keys, 1_000_000, "idempotency.max_cached_keys");
assert_eq!(cfg.idempotency.ttl_seconds, 86_400, "idempotency.ttl_seconds");
// §14.8 session_pinning defaults
assert_eq!(cfg.session_pinning.max_sessions, 100_000, "session_pinning.max_sessions");
// §14.8 query_coalescing defaults
assert_eq!(cfg.query_coalescing.max_subscribers, 1_000, "query_coalescing.max_subscribers");
assert_eq!(cfg.query_coalescing.max_pending_queries, 10_000, "query_coalescing.max_pending_queries");
// §14.8 anti_entropy defaults
assert_eq!(cfg.anti_entropy.max_read_concurrency, 2, "anti_entropy.max_read_concurrency");
assert_eq!(cfg.anti_entropy.fingerprint_batch_size, 1_000, "anti_entropy.fingerprint_batch_size");
// §14.8 resharding defaults
assert_eq!(cfg.resharding.backfill_concurrency, 4, "resharding.backfill_concurrency");
assert_eq!(cfg.resharding.backfill_batch_size, 1_000, "resharding.backfill_batch_size");
// §14.8 peer_discovery defaults
assert_eq!(cfg.peer_discovery.service_name, "miroir-headless", "peer_discovery.service_name");
assert_eq!(cfg.peer_discovery.refresh_interval_s, 15, "peer_discovery.refresh_interval_s");
// §14.8 leader_election defaults
assert_eq!(cfg.leader_election.lease_ttl_s, 10, "leader_election.lease_ttl_s");
assert_eq!(cfg.leader_election.renew_interval_s, 3, "leader_election.renew_interval_s");
}
}

64
notes/bf-4w08.md Normal file
View file

@ -0,0 +1,64 @@
# P6.10 Wire §14.8 resource-aware config defaults - Verification Summary
## Task: bf-4w08
Verified that all §14.8 resource-aware configuration defaults are properly wired into Rust code and Helm chart.
## Acceptance Criteria Status
### 1. Each §14.8 key present in `crates/miroir-core/src/config/` with documented default
- ✅ `server`: max_body_bytes: 104857600, max_concurrent_requests: 500, request_timeout_ms: 30000
- ✅ `connection_pool_per_node`: max_idle: 32, max_total: 128, idle_timeout_s: 60
- ✅ `task_registry`: cache_size: 10000, redis_pool_max: 50
- ✅ `idempotency`: max_cached_keys: 1000000, ttl_seconds: 86400
- ✅ `session_pinning`: max_sessions: 100000
- ✅ `query_coalescing`: max_subscribers: 1000, max_pending_queries: 10000
- ✅ `anti_entropy`: max_read_concurrency: 2, fingerprint_batch_size: 1000
- ✅ `resharding`: backfill_concurrency: 4, backfill_batch_size: 1000
- ✅ `peer_discovery`: service_name: "miroir-headless", refresh_interval_s: 15
- ✅ `leader_election`: enabled: true, lease_ttl_s: 10, renew_interval_s: 3
### 2. `charts/miroir/values.yaml` exposes same keys with identical defaults
- ✅ All §14.8 defaults present in values.yaml (lines 69-120)
- ✅ Values match plan §14.8 exactly
### 3. `values.schema.json` accepts documented ranges; rejects nonsense
- ✅ Schema defines all §14.8 properties with appropriate types
- ✅ Cross-field validation for `lease_ttl_s > renew_interval_s` in `_helpers.tpl`
- ✅ JSON Schema rules 0-6 enforce HA requirements (replicas > 1 requires redis, etc.)
### 4. K8s resources block matches §14.8 (500m/2000m CPU, 1Gi/3584Mi mem)
- ✅ `values.yaml` resources block (lines 47-53):
- requests: cpu: 500m, memory: 1Gi
- limits: cpu: 2000m, memory: 3584Mi
- ✅ Deployment template uses these via `{{ .Values.miroir.resources }}`
### 5. Unit test: drift guard comparing Config::default() to §14.8 reference
- ✅ Test `section_14_8_defaults_match` in `crates/miroir-core/src/config.rs` (lines 685-728)
- ✅ Test passes: all defaults verified against §14.8 values
### 6. Drift guard: doc-test comparing Config::default() to §14.8 reference
- ✅ Doc-test at top of `MiroirConfig` struct (lines 21-45 in config.rs)
- ✅ Doc-test passes with `cargo test --doc`
## Test Results
```bash
$ cargo test section_14_8_defaults_match --lib
test config::tests::section_14_8_defaults_match ... ok
test result: ok. 1 passed; 0 failed
```
## Files Verified
- `crates/miroir-core/src/config.rs` - Core config struct with defaults
- `crates/miroir-core/src/config/advanced.rs` - Advanced feature configs
- `charts/miroir/values.yaml` - Helm chart values with §14.8 defaults
- `charts/miroir/values.schema.json` - JSON Schema validation
- `charts/miroir/templates/_helpers.tpl` - Cross-field validation
- `charts/miroir/templates/miroir-deployment.yaml` - Resources block
## Conclusion
All §14.8 resource-aware configuration defaults are properly wired and validated. The drift guard test and doc-test ensure defaults cannot drift from the plan without failing CI.