From 94811d4e2348d0f9f3b24e531cb5c6fbabc18410 Mon Sep 17 00:00:00 2001 From: jedarden Date: Wed, 3 Jun 2026 23:01:26 -0400 Subject: [PATCH] notes(bf-1bvca): update with current status - migration complete, blocked by cluster CPU --- notes/bf-1bvca.md | 43 +++++++++++++++++++------------------------ 1 file changed, 19 insertions(+), 24 deletions(-) diff --git a/notes/bf-1bvca.md b/notes/bf-1bvca.md index f9dc7c8..0f0b8b2 100644 --- a/notes/bf-1bvca.md +++ b/notes/bf-1bvca.md @@ -22,48 +22,43 @@ The `combat_turns` column was already present in `~/declarative-config/k8s/apexa ### 2. Rollout Annotation Bumped -The deployment annotation was bumped to force a restart of the schema-init pod: +The deployment annotation was bumped multiple times to force restart of the schema-init pod: ```yaml -checksum/schema: "v4-combat-turns-migration-2026-06-03-b" +checksum/schema: "v5-combat-turns-migration-2026-06-03-i" ``` -### 3. Commits Already Pushed - -All changes were committed and pushed to origin/main: - -``` -137b7c5 fix(acb-schema): bump annotation to force combat_turns migration redeploy -51e4a36 fix(schema): bump annotation to force schema-init restart for combat_turns column -f74367f feat(apexalgo-iad): bump schema-init rollout annotation for combat_turns migration -00e1f5c feat(apexalgo-iad): add acb-schema-init deployment with combat_turns migration -``` +Latest commit: `9908a18 fix(acb-schema-init): bump rollout annotation for combat_turns migration (v5-combat-turns-migration-2026-06-03-i)` ## Status ✅ **Migration SQL is in place** (line 305 of acb-schema-init.yml) -✅ **Changes committed and pushed** to declarative-config -✅ **Rollout annotation bumped** to v4-combat-turns-migration-2026-06-03-b -✅ **ArgoCD has synced** the deployment spec (verified via kubectl) +✅ **Changes committed and pushed** to declarative-config (9908a18) +✅ **Rollout annotation bumped** to v5-combat-turns-migration-2026-06-03-i +🔄 **ArgoCD syncing** (ai-code-battle-ns-apexalgo-iad app: OutOfSync, Operation: Running) ⏸️ **BLOCKED: apexalgo-iad cluster has insufficient CPU resources** -## Cluster Resource Issue +## Cluster Resource Issue (BLOCKING VERIFICATION) All pods in ai-code-battle namespace are stuck in Pending state: ``` -acb-api-7c46c9d5b6-jfl9w 0/1 Pending (Insufficient cpu) -acb-evolver-85549b574d-pqbjd 0/1 Pending (Insufficient cpu) +acb-api-5646489f75-l4zmq 0/1 Pending (Insufficient cpu) +acb-evolver-7654d8b866-psvk5 0/1 Pending (Insufficient cpu) acb-index-builder-6669fdbc95-nxwhf 0/1 Pending (Insufficient cpu) -acb-schema-init-74c6894ddb-9g67p 0/1 Pending (Insufficient cpu) +acb-schema-init-5c68bc7bbc-4497t 1/1 Running (low CPU req: 10m) +acb-worker-bf5bfdb98-g9jnn 0/1 Pending (Insufficient cpu) ``` -Error message: `0/3 nodes are available: 3 Insufficient cpu. preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod.` +Error message: `0/3 nodes are available: 3 Insufficient cpu. preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod` -The schema-init pod cannot schedule to apply the migration. Once cluster resources are available: -1. New schema-init pod will schedule and apply the migration -2. Index-builder will no longer crash on "combat_turns does not exist" +The index-builder pod cannot schedule to verify the fix. The schema-init pod IS running because it has very low CPU requests (10m). + +Once cluster resources are available: +1. ArgoCD will complete sync of schema-init deployment +2. New schema-init pod (with updated annotation) will apply migration +3. Index-builder will schedule and no longer crash on "combat_turns does not exist" ## Migration Code Complete -The migration code is **complete and deployed** - only cluster resources are blocking verification. +The migration code is **complete and pushed** - only cluster CPU resources are blocking verification of the fix.