From e06091fb649122053b3b05c37de5d65dc59e20f1 Mon Sep 17 00:00:00 2001 From: jedarden Date: Sun, 31 May 2026 11:59:29 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20remove=20R2=20warm=20cache=20tier=20fro?= =?UTF-8?q?m=20plan=20=E2=80=94=20B2-only=20storage?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/plan/plan.md | 337 ++++++++++++++++++---------------------------- 1 file changed, 132 insertions(+), 205 deletions(-) diff --git a/docs/plan/plan.md b/docs/plan/plan.md index e4feb55..0d4b358 100644 --- a/docs/plan/plan.md +++ b/docs/plan/plan.md @@ -23,11 +23,7 @@ All compute runs in the **apexalgo-iad** Kubernetes cluster (Rackspace Spot), which acts as a **match factory**: it runs battles, generates replays, and periodically publishes the updated site to Pages. -Replay files use **tiered storage**: Backblaze B2 is the permanent cold -archive (all replays, forever), and Cloudflare R2 is a warm cache for recent -replays (free tier, ≤10 GB). The browser fetches replays from R2; old replays -that have aged out of R2 fall back to B2 (free egress via Cloudflare Bandwidth -Alliance). +Replay files are stored in and served directly from **Backblaze B2** (via Cloudflare CDN / Bandwidth Alliance). B2 is the single storage layer — all replays, thumbnails, bot cards, match metadata, and evolution status files live in B2. Free egress via Cloudflare Bandwidth Alliance (zero egress fees). ### Cloudflare (Static Tier) @@ -36,19 +32,13 @@ Alliance). indexes, series, seasons, evolution data, blog posts. Updated every ~90 minutes by the K8s index builder via `wrangler pages deploy`. Global CDN, zero-config TLS, instant cache invalidation on deploy. -- **Cloudflare R2** (warm replay cache): Stores recent replay files and - per-match metadata — the subset of data too numerous for Pages' 20K file - limit and too hot for B2-only serving. Capped at ≤10 GB to stay within - the free tier. The index builder manages the R2 lifecycle: promotes recent - replays from B2, prunes old ones when approaching the cap. -### Backblaze B2 (Cold Archive) +### Backblaze B2 (Storage) -- **B2 bucket**: Permanent archive for **all** replay files and match - metadata. Match workers upload directly to B2 after each match. B2 is the - source of truth for replay data — R2 is just a CDN cache in front of it. - Free egress to browsers via Cloudflare Bandwidth Alliance (B2 + Cloudflare - = zero egress fees). S3-compatible API. +- **B2 bucket**: Permanent storage for **all** replay files, match metadata, + thumbnails, bot cards, and evolution status files. Match workers upload + directly to B2 after each match. Served to browsers via Cloudflare CDN + (Bandwidth Alliance = zero egress fees). S3-compatible API. ### apexalgo-iad (Compute Tier) @@ -72,9 +62,7 @@ All backend compute runs in the `ai-code-battle` namespace: every 4h. - **Index Builder Deployment**: Sleep-loop (15 min cycle). Reads PostgreSQL, generates all JSON index files, deploys them to Cloudflare Pages via - `wrangler pages deploy`. Manages R2 warm cache (copies recent replays - from B2, prunes old replays to stay within free tier). Self-restarts - every 4h. + `wrangler pages deploy`. Self-restarts every 4h. ### Go API (deferred) @@ -128,20 +116,7 @@ Pages project (ai-code-battle.pages.dev): └── {map_id}.json ``` -**Cloudflare R2** (warm replay cache, free tier ≤10 GB): -``` -R2 bucket: -├── replays/ -│ └── {match_id}.json.gz (recent replay files, promoted from B2) -├── matches/ -│ └── {match_id}.json (recent per-match metadata) -├── thumbnails/ -│ └── {match_id}.png (auto-generated match thumbnails) -└── cards/ - └── {bot_id}.png (bot profile card images) -``` - -**Backblaze B2** (cold archive, permanent): +**Backblaze B2** (storage — all replay data, served via Cloudflare CDN): ``` B2 bucket: ├── replays/ @@ -150,8 +125,10 @@ B2 bucket: │ └── {match_id}.json (ALL per-match metadata) ├── thumbnails/ │ └── {match_id}.png (ALL thumbnails) -└── cards/ - └── {bot_id}.png (ALL bot card images) +├── cards/ +│ └── {bot_id}.png (ALL bot card images) +└── evolution/ + └── live.json (evolver status, updated each cycle) ``` **Data loading pattern in the SPA:** @@ -159,24 +136,18 @@ B2 bucket: ```js // SPA shell + index data from Cloudflare Pages (same origin) const PAGES = '' // relative — same origin as the SPA -// Replays from R2 warm cache (recent), B2 cold archive (old) -const R2 = 'https://r2.aicodebattle.com' // or R2 public URL const B2 = 'https://b2.aicodebattle.com' // B2 via Cloudflare CDN // Leaderboard, bot profiles, match indexes — all from Pages (same origin): const lb = await fetch(`${PAGES}/data/leaderboard.json`).then(r => r.json()) -// Replay viewer — try R2 warm cache first, fall back to B2 cold archive: +// Replay viewer — fetches directly from B2: async function fetchReplay(matchId) { - const r2 = await fetch(`${R2}/replays/${matchId}.json.gz`) - if (r2.ok) return r2 - return fetch(`${B2}/replays/${matchId}.json.gz`) // cold fallback + return fetch(`${B2}/replays/${matchId}.json.gz`) } -// Match metadata — same R2-then-B2 pattern: -const meta = await fetch(`${R2}/matches/${matchId}.json`) - .then(r => r.ok ? r : fetch(`${B2}/matches/${matchId}.json`)) - .then(r => r.json()) +// Match metadata — fetches directly from B2: +const meta = await fetch(`${B2}/matches/${matchId}.json`).then(r => r.json()) ``` **Cache behavior:** @@ -184,64 +155,54 @@ const meta = await fetch(`${R2}/matches/${matchId}.json`) - **Pages assets**: Cloudflare Pages handles caching automatically. Deploys via `wrangler pages deploy` invalidate the cache globally. Index data is at most ~90 minutes stale (the index builder's cycle time). -- **R2 objects** (warm cache): Served with appropriate `Cache-Control` headers: +- **B2 objects**: Served via Cloudflare CDN with appropriate `Cache-Control` headers: - `replays/*.json.gz`: `immutable, max-age=31536000` (content-addressed) - `matches/*.json`: `immutable, max-age=31536000` (content-addressed) - `thumbnails/`, `cards/`: `max-age=86400` (regenerated rarely) -- **B2 objects** (cold archive): Same cache headers. B2 egress through - Cloudflare Bandwidth Alliance = zero egress fees. Cloudflare CDN caches - B2 responses, so frequently accessed cold replays still perform well. + - `evolution/live.json`: `max-age=10` (updated each evolver cycle) + B2 egress through Cloudflare Bandwidth Alliance = zero egress fees. Cloudflare CDN + caches B2 responses so frequently accessed replays perform well globally. -**Tiered storage lifecycle:** +**Data flow:** 1. Match worker completes a match → uploads `replays/{match_id}.json.gz` - and `matches/{match_id}.json` to **B2** (cold archive, permanent) + and `matches/{match_id}.json` to **B2** 2. Worker writes match result to **PostgreSQL** (scores, ratings, metadata) 3. Index builder (every ~15 min) reads new results from PostgreSQL, rebuilds all JSON index files, deploys to **Pages** via `wrangler pages deploy` -4. Index builder promotes recent replays from B2 to **R2** (warm cache) -5. Index builder prunes oldest replays from R2 when approaching 10 GB cap -6. Browser loads SPA + indexes from Pages, fetches replays from R2 (warm) - with B2 fallback (cold) +4. Browser loads SPA + indexes from Pages, fetches replays and match data + directly from **B2** (via Cloudflare CDN) **Storage budget:** -- **R2 (warm cache)**: ≤10 GB free tier. At ~50 KB/replay (gzipped), holds - ~200K replays. At 60 matches/hour, that's ~139 days of warm replays. - Class A writes: index builder promotes ~1,440 replays/day = ~43K/month - (well within 1M/month free tier). -- **B2 (cold archive)**: First 10 GB free, $0.006/GB/month after. At 60 - matches/hour, ~2.2 GB/month. Year one: ~26 GB = ~$0.10/month. +- **B2**: First 10 GB free, $0.006/GB/month after. At 60 matches/hour, + ~2.2 GB/month for replays. Year one: ~26 GB ≈ $0.10/month. Free egress via Cloudflare Bandwidth Alliance. - **Pages**: 20K file limit per deployment. Only SPA + JSON indexes — well - within limits (replays are on R2/B2, not Pages). + within limits (replays and match data are on B2, not Pages). ``` ┌────────── Cloudflare ──────────────────────────────────┐ │ │ -│ ┌─────────────────────┐ ┌─────────────────────────┐ │ -│ │ Cloudflare Pages │ │ Cloudflare R2 │ │ -│ │ (static site) │ │ (warm replay cache) │ │ -│ │ │ │ │ │ -│ │ SPA shell (HTML/ │ │ replays/*.json.gz │ │ -│ │ JS/CSS) │ │ matches/*.json │ │ -│ │ data/*.json indexes │ │ thumbnails/*.png │ │ -│ │ maps/*.json │ │ cards/*.png │ │ -│ │ docs/, img/ │ │ ≤10 GB free tier │ │ -│ └─────────────────────┘ └─────────────────────────┘ │ -│ ▲ wrangler deploy ▲ S3 PUT (promote) │ -└────────┼─────────────────────────┼───────────────────────┘ - │ │ -┌────────┼─────────────────────────┼───────────────────────┐ -│ │ apexalgo-iad cluster — ai-code-battle ns │ -│ │ │ │ -│ ┌─────┴──────────────────┐ │ │ -│ │ Index Builder Dep. │ │ │ -│ │ Reads PostgreSQL, │──────┘ │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ Cloudflare Pages (static site) │ │ +│ │ │ │ +│ │ SPA shell (HTML/JS/CSS) │ │ +│ │ data/*.json indexes │ │ +│ │ maps/*.json │ │ +│ │ docs/, img/ │ │ +│ └─────────────────────────────────────────────────┘ │ +│ ▲ wrangler deploy │ +└────────┼──────────────────────────────────────────────────┘ + │ +┌────────┼──────────────────────────────────────────────────┐ +│ │ apexalgo-iad cluster — ai-code-battle ns │ +│ │ │ +│ ┌─────┴──────────────────┐ │ +│ │ Index Builder Dep. │ │ +│ │ Reads PostgreSQL, │ │ │ │ generates JSON indexes,│ │ -│ │ deploys to Pages, │ │ -│ │ promotes replays to R2,│ │ -│ │ prunes R2 warm cache │ │ +│ │ deploys to Pages │ │ │ └─────────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────┐ │ @@ -299,15 +260,14 @@ const meta = await fetch(`${R2}/matches/${matchId}.json`) | Component | Where | Role | |-----------|-------|------| | **Cloudflare Pages** | Cloudflare | Static site: SPA (HTML/JS/CSS) and pre-computed JSON index files. Updated every ~90 min by the index builder via `wrangler pages deploy`. Global CDN with automatic cache invalidation. | -| **Cloudflare R2** | Cloudflare | Warm replay cache: recent replays, match metadata, thumbnails, bot cards. Free tier ≤10 GB. Managed by the index builder (promote from B2, prune when approaching cap). | -| **Backblaze B2** | Backblaze | Cold archive: ALL replays and match data, permanently. Workers upload directly after each match. Free egress via Cloudflare Bandwidth Alliance. | +| **Backblaze B2** | Backblaze | Storage: ALL replays, match metadata, thumbnails, bot cards, and evolution status. Workers upload directly after each match. Served via Cloudflare CDN (Bandwidth Alliance = zero egress fees). | | **Matchmaker** | Deployment (ai-code-battle ns) | Internal scheduler: computes pairings, enqueues jobs to Valkey, health checks bots, reaps stale jobs. No external exposure. | | **PostgreSQL** | CNPG cluster (cnpg ns, `cnpg-apexalgo`) | Relational database — bot registry, match queue, ratings, results, series, seasons. Source of truth for structured data. | | **Valkey** | Cluster service | Job queue (`acb:jobs:pending`), ephemeral caching. | | **Match Workers** | Deployment (ai-code-battle ns) | Stateless match execution — BRPOP from Valkey, run simulation, upload replay to B2, write result to PostgreSQL. | | **Bot Containers** | Deployments + Services (ai-code-battle ns) | Strategy bots (x6) + evolved bots (0-50) — HTTP servers called by workers during matches via cluster-internal Service DNS. | | **Evolver** | Deployment (ai-code-battle ns) | Evolution pipeline — reads lineage/meta from PostgreSQL, generates candidates, writes evolution data to PostgreSQL. | -| **Index Builder** | Deployment (ai-code-battle ns) | Sleep-loop (15 min cycle). Reads PostgreSQL, generates JSON indexes, deploys to Pages. Promotes recent replays from B2 to R2 warm cache. Prunes R2 to stay within free tier. Self-restarts every 4h. | +| **Index Builder** | Deployment (ai-code-battle ns) | Sleep-loop (15 min cycle). Reads PostgreSQL, generates JSON indexes, deploys to Pages. Self-restarts every 4h. | | **Go API** | Deferred | Social features (predictions, comments, voting) and third-party bot registration. Not required for v1. | | **ArgoCD** | Cluster (argocd ns) | GitOps: syncs all K8s manifests from git. All deployments are declarative. | | **Argo Workflows** | Cluster (argo ns) | CI pipelines: builds container images, pushes to Forgejo registry, builds static site. | @@ -1174,26 +1134,18 @@ encoding — only recording events that changed from the previous turn. ### 7.2 Storage -Replays use **tiered storage**: Backblaze B2 is the permanent cold archive -(all replays, forever), and Cloudflare R2 is a warm cache for recent replays -(free tier, ≤10 GB). Pre-computed JSON index files are deployed to -**Cloudflare Pages** by the index builder. No PersistentVolumes are used for -web-facing data. +All replay files, match metadata, thumbnails, bot cards, and evolution status +files are stored in **Backblaze B2** and served via Cloudflare CDN (Bandwidth +Alliance). Pre-computed JSON index files are deployed to **Cloudflare Pages** +by the index builder. No PersistentVolumes are used for web-facing data. -**B2 data layout** (cold archive — all data, permanently): +**B2 data layout** (all data — served via Cloudflare CDN): ``` replays/{match_id}.json.gz # ALL replay files matches/{match_id}.json # ALL per-match metadata thumbnails/{match_id}.png # ALL match thumbnails cards/{bot_id}.png # ALL bot profile card images -``` - -**R2 data layout** (warm cache — recent subset, ≤10 GB): -``` -replays/{match_id}.json.gz # recent replay files (promoted from B2) -matches/{match_id}.json # recent per-match metadata -thumbnails/{match_id}.png # recent match thumbnails -cards/{bot_id}.png # bot profile card images +evolution/live.json # evolver status (updated each cycle) ``` **Pages data layout** (static site): @@ -1215,36 +1167,23 @@ maps/{map_id}.json # map definitions **How data flows:** 1. Match worker completes a match → uploads `replays/{match_id}.json.gz` - and `matches/{match_id}.json` to **B2** (cold archive, via S3-compatible - API) + and `matches/{match_id}.json` to **B2** (via S3-compatible API) 2. Worker writes match result to **PostgreSQL** (scores, ratings, metadata) 3. Index builder (every ~15 min) reads new results from PostgreSQL, rebuilds all JSON index files, deploys to **Pages** via `wrangler pages deploy` -4. Index builder promotes recent replays from **B2 to R2** (warm cache) -5. Index builder prunes oldest replays from **R2** when approaching 10 GB -6. Browser loads SPA + indexes from Pages, fetches replays from R2 (warm) - with B2 fallback (cold) +4. Browser loads SPA + indexes from Pages, fetches replays and match data + directly from **B2** (via Cloudflare CDN) -**Tiered retention:** -- **B2 (cold)**: All replays retained permanently. No pruning. The canonical - archive. -- **R2 (warm)**: Recent replays, capped at ≤10 GB. The index builder prunes - by age when approaching the cap — oldest replays are removed from R2 - (they remain in B2). -- **Exemptions from R2 pruning**: Replays referenced by playlists ("Closest - Finishes", "Biggest Upsets", etc.), rivalry pages, series, or season - archives are kept warm longer. The pruning job checks the exemption list - from PostgreSQL before removing from R2. +**Retention:** +- **B2**: All replays retained permanently. No pruning. The canonical + store. - **PostgreSQL**: Match metadata retained indefinitely (rows are small). - Index files are append-with-rotation: `index.json` holds the last 1000; older pages at `index-{page}.json`. **Storage costs:** -- **R2 (warm cache)**: ≤10 GB free tier. At ~50 KB/replay, holds ~200K warm - replays (~139 days at 60 matches/hour). Class A writes: ~43K/month - (index builder promoting replays). Well within free tier limits. -- **B2 (cold archive)**: First 10 GB free, $0.006/GB/month after. At 60 - matches/hour, ~2.2 GB/month. Year one: ~26 GB ≈ $0.10/month. Free +- **B2**: First 10 GB free, $0.006/GB/month after. At 60 matches/hour, + ~2.2 GB/month for replays. Year one: ~26 GB ≈ $0.10/month. Free egress via Cloudflare Bandwidth Alliance. - **Pages**: No per-file storage costs. 20K file limit per deployment — only SPA + JSON indexes, well within limits. @@ -1255,14 +1194,14 @@ The replay viewer is a client-side TypeScript application rendered on HTML5 Canvas. **Rendering pipeline:** -1. Fetch `replay.json.gz` from R2 warm cache (falling back to B2 cold - archive); browser handles gzip decompression via `Accept-Encoding` +1. Fetch `replay.json.gz` from B2 (via Cloudflare CDN); browser handles + gzip decompression via `Accept-Encoding` 2. Parse and index: build per-turn game state by replaying events from turn 0 3. Render the current turn to canvas 4. User controls advance/rewind the turn index No API invocations — the viewer is a static page (served from Pages) loading -a replay file from R2 (warm cache) or B2 (cold archive). +a replay file from B2 (via Cloudflare CDN). **Visual design:** @@ -1299,11 +1238,11 @@ replay viewer is the landing page for any match. No login required to watch. The web platform spans **Cloudflare** (static site and data files) and the **apexalgo-iad** Kubernetes cluster (API, compute, databases). **Cloudflare -Pages** serves the SPA globally. **Cloudflare R2** stores replays and -per-match data. The **Go API Deployment** on K8s handles registration, job -coordination, and scheduling logic. **CNPG PostgreSQL** stores all relational -data. **Valkey** provides the job queue and caching. No PersistentVolumes -are used for web-facing data. +Pages** serves the SPA globally. **Backblaze B2** (via Cloudflare CDN) stores +replays and all per-match data. The **Go API Deployment** on K8s handles +registration, job coordination, and scheduling logic. **CNPG PostgreSQL** stores +all relational data. **Valkey** provides the job queue and caching. No +PersistentVolumes are used for web-facing data. ### 8.1 Cloudflare Pages (Static Site) @@ -1316,9 +1255,9 @@ single Pages project. The index builder on K8s updates the data files every / → Landing page, featured replays, leaderboard summary /leaderboard → Full leaderboard (fetches data/leaderboard.json from Pages) /matches → Match history (fetches data/matches/index.json from Pages) -/replay/{match_id} → Replay viewer (fetches replays/{match_id}.json.gz from R2) +/replay/{match_id} → Replay viewer (fetches replays/{match_id}.json.gz from B2) /bot/{bot_id} → Bot profile (fetches data/bots/{bot_id}.json from Pages) -/evolution → Evolution dashboard (fetches data/evolution/*.json from Pages + evolution/live.json from R2) +/evolution → Evolution dashboard (fetches data/evolution/*.json from Pages + evolution/live.json from B2) /register → Bot registration form (submits to Go API on K8s) /docs → Protocol spec, starter kit links, getting started ``` @@ -1333,8 +1272,8 @@ No build-time data fetching -- all data loaded at runtime. ```js // SPA shell + index data from Pages (same origin) const leaderboard = await fetch('/data/leaderboard.json').then(r => r.json()) -// Replays from R2 (cross-origin, CORS enabled on R2 bucket) -const replay = await fetch(`https://r2.aicodebattle.com/replays/${matchId}.json.gz`) +// Replays from B2 (cross-origin, CORS enabled on B2 bucket) +const replay = await fetch(`https://b2.aicodebattle.com/replays/${matchId}.json.gz`) // Dynamic operations from K8s API const result = await fetch('https://api.aicodebattle.com/api/register', { method: 'POST', body: ... }) ``` @@ -1343,7 +1282,7 @@ Index JSON files are rebuilt and deployed to Pages every ~15 minutes by the index builder (with actual Pages deploys batched every ~90 minutes to stay well within Cloudflare's deploy limits). Visitors see index data that is at most ~90 minutes old. Replays and per-match metadata are uploaded to -R2 in real time by match workers and available immediately. +B2 in real time by match workers and available immediately. ### 8.2 Go API Service @@ -1592,16 +1531,10 @@ start → build indexes → sleep 15m → build indexes → sleep 15m → ... build) and deploys to Cloudflare Pages via `wrangler pages deploy`. Only the `data/` directory changes between deploys; the SPA shell updates only when a new site build is triggered. -4. **Prune on R2:** On one cycle per week (checked via day-of-week), runs - replay pruning — lists objects in R2's `replays/` prefix older than - 90 days, queries PostgreSQL for exempt match IDs, deletes non-exempt - objects via S3 DeleteObjects API. **Environment:** The Deployment Pod has PostgreSQL credentials and a **Cloudflare API token** (for `wrangler pages deploy`) stored as -SealedSecrets. It also has R2 credentials (S3-compatible access key) for -the weekly pruning cycle. No PersistentVolumes needed — all output goes to -Cloudflare. +SealedSecrets. No PersistentVolumes needed — all output goes to Cloudflare. ### 8.5 Bot Registration @@ -1687,8 +1620,8 @@ time range, human-only vs all). Auto-refresh every 60 seconds. Public - Each entry: match_id, participants, scores, date, link to replay **Match detail** (`/replay/{match_id}`): -- Fetches `matches/{match_id}.json` from R2 for metadata -- Fetches `replays/{match_id}.json.gz` from R2 for the replay +- Fetches `matches/{match_id}.json` from B2 for metadata +- Fetches `replays/{match_id}.json.gz` from B2 for the replay - Embedded replay viewer (auto-plays) - Score breakdown, participants, match duration @@ -1700,21 +1633,21 @@ time range, human-only vs all). Auto-refresh every 60 seconds. Public Compute runs in the **apexalgo-iad** Kubernetes cluster (Rackspace Spot) in a dedicated `ai-code-battle` namespace. The public-facing product is a -**Cloudflare Pages** static site. Replay files use tiered storage: -**Cloudflare R2** (warm cache, free tier ≤10 GB) and **Backblaze B2** (cold -archive, permanent). The cluster is existing infrastructure shared with other -workloads — it already provides PostgreSQL (CNPG), Valkey, Traefik ingress, -cert-manager, ArgoCD, Argo Workflows, Argo Events, Forgejo (git + container -registry), SATA (Cinder CSI) storage, and Sealed Secrets. +**Cloudflare Pages** static site. All replay files and match data are stored +in **Backblaze B2** and served via Cloudflare CDN (Bandwidth Alliance). +The cluster is existing infrastructure shared with other workloads — it +already provides PostgreSQL (CNPG), Valkey, Traefik ingress, cert-manager, +ArgoCD, Argo Workflows, Argo Events, Forgejo (git + container registry), +SATA (Cinder CSI) storage, and Sealed Secrets. Key principles: - **Static-first architecture** — the public product is a static site on Pages. All data visitors see is pre-computed JSON. K8s is the factory that generates data and publishes it to Pages. -- **Tiered storage** — B2 is the permanent archive for all replays. R2 is - a warm CDN cache for recent replays, capped at the free tier. The index - builder manages the R2 lifecycle. +- **B2 as single storage layer** — B2 stores all replays, match metadata, + thumbnails, bot cards, and evolution status. Served directly via Cloudflare + CDN (Bandwidth Alliance = zero egress fees). No warm cache tier needed. - **GitOps via ArgoCD** — all K8s manifests are committed to git and synced by ArgoCD. Never apply manifests directly with `kubectl`. - **Argo Workflows for CI** — container image builds and static site builds @@ -1744,8 +1677,6 @@ Cross-namespace dependencies: - **Cloudflare Pages project**: `ai-code-battle` (`ai-code-battle.pages.dev`) — hosts the static SPA and data indexes. Deployed by the index builder via `wrangler pages deploy`. -- **Cloudflare R2 bucket**: Warm cache for recent replays, thumbnails, bot - cards. Free tier (≤10 GB). Managed by the index builder. - **DNS** (when custom domain is desired): `aicodebattle.com` CNAME to Pages. **Backblaze B2 infrastructure requirements:** @@ -1763,7 +1694,7 @@ declarative-config/k8s/apexalgo-iad/ai-code-battle/ ├── acb-schema-init.yml (ConfigMap + Deployment for schema migration) ├── acb-matchmaker-deployment.yml (matchmaker: pairings, job enqueue, health, reaper) ├── acb-worker-deployment.yml (match workers: run matches, upload to B2) -├── acb-index-builder-deployment.yml (index builder: generate JSON, deploy to Pages, manage R2) +├── acb-index-builder-deployment.yml (index builder: generate JSON, deploy to Pages) ├── acb-evolver-deployment.yml (LLM evolution pipeline) ├── acb-strategy-random-deployment.yml (RandomBot — Python) ├── acb-strategy-random-service.yml @@ -1777,7 +1708,7 @@ declarative-config/k8s/apexalgo-iad/ai-code-battle/ ├── acb-strategy-swarm-service.yml ├── acb-strategy-hunter-deployment.yml (HunterBot — Java) ├── acb-strategy-hunter-service.yml -├── acb-secret.yml.template (template: B2/R2/Cloudflare/bot secrets) +├── acb-secret.yml.template (template: B2/Cloudflare/bot secrets) └── acb-sealedsecret.yml (sealed version of above) ``` @@ -1796,7 +1727,7 @@ container registry (`forgejo.ardenone.com/ai-code-battle/`). | `acb-matchmaker` | Go binary on Alpine | Matchmaking, health checks, stale job reaping | Deployment (1 replica) | | `acb-worker` | Go binary on Alpine | Match execution, B2 upload | Deployment (2-10 replicas) | | `acb-evolver` | Go binary on Alpine | Evolution pipeline | Deployment (1 replica) | -| `acb-index-builder` | Go binary on Alpine (includes `wrangler` CLI) | Reads PostgreSQL, generates JSON indexes, deploys to Pages, manages R2 warm cache (promote from B2, prune) | Deployment (sleep-loop, 15 min cycle, Pages deploy every ~90 min, self-restarts every 4h) | +| `acb-index-builder` | Go binary on Alpine (includes `wrangler` CLI) | Reads PostgreSQL, generates JSON indexes, deploys to Pages | Deployment (sleep-loop, 15 min cycle, Pages deploy every ~90 min, self-restarts every 4h) | | `acb-strategy-random` | Python 3.13 slim | RandomBot | Deployment (1 replica) | | `acb-strategy-gatherer` | Go on Alpine | GathererBot | Deployment (1 replica) | | `acb-strategy-rusher` | Rust on Alpine | RusherBot | Deployment (1 replica) | @@ -1826,8 +1757,7 @@ Match workers coordinate via **Valkey** (job queue) and **PostgreSQL** 7. Worker writes result directly to **PostgreSQL** (scores, winner, turn count, condition) and updates ratings (Glicko-2) 8. Index builder (next ~15-min cycle) reads new results from PostgreSQL, - rebuilds all index JSON files, deploys to Pages, promotes recent - replays from B2 to R2 warm cache + rebuilds all index JSON files, deploys to Pages **Stale job recovery:** - Reaper ticker (in matchmaker) checks PostgreSQL every 5 minutes for @@ -1867,11 +1797,10 @@ Match workers coordinate via **Valkey** (job queue) and **PostgreSQL** **External traffic:** - `ai-code-battle.pages.dev` (or custom domain) → Cloudflare Pages (static SPA + data indexes) -- R2 public URL → Cloudflare R2 (warm replay cache) -- B2 public URL (via Cloudflare CDN) → Backblaze B2 (cold replay archive) +- B2 public URL (via Cloudflare CDN) → Backblaze B2 (replay/match data storage) - No K8s services are exposed externally in v1. The Go API IngressRoute at `api.aicodebattle.com` is planned for when social features are added. -- TLS: Pages and R2 handle TLS automatically. B2 via Cloudflare gets TLS +- TLS: Pages handles TLS automatically. B2 via Cloudflare CDN gets TLS from the CDN layer. **Cluster-internal traffic:** @@ -1884,8 +1813,7 @@ Match workers coordinate via **Valkey** (job queue) and **PostgreSQL** - Workers -> B2: HTTPS (S3-compatible API, for replay upload) - Index Builder -> PostgreSQL: same - Index Builder -> Cloudflare Pages: HTTPS (wrangler CLI) -- Index Builder -> R2: HTTPS (S3-compatible API, for warm cache management) -- Index Builder -> B2: HTTPS (S3-compatible API, for promoting replays to R2) +- Index Builder -> B2: HTTPS (S3-compatible API, read stats for index building) - Evolver -> PostgreSQL: same - All cluster-internal traffic is plaintext (trusted network) @@ -1896,9 +1824,9 @@ Match workers coordinate via **Valkey** (job queue) and **PostgreSQL** - PostgreSQL credentials in SealedSecrets (encrypted in git, decrypted in-cluster) - Valkey access is cluster-internal only (no external exposure) -- Pages, R2, and B2 serve public-read data only — no secrets stored there +- Pages and B2 serve public-read data only — no secrets stored there - B2 write credentials (SealedSecret) are scoped to the acb bucket -- Cloudflare API token (SealedSecret) is scoped to Pages deploy + R2 only +- Cloudflare API token (SealedSecret) is scoped to Pages deploy only - NetworkPolicy can restrict egress from bot pods to prevent data exfiltration (future hardening) @@ -1986,7 +1914,6 @@ appropriate Argo Workflow(s). | Worker queue depth | Valkey LLEN on `acb:jobs:pending` | >50 pending for >30 min | | Bot health failures | Matchmaker health checker ticker | >50% failing | | Stale jobs | Matchmaker reaper ticker count | >10 stale in a cycle | -| R2 usage | Index builder tracks warm cache size | >8 GB (approaching 10 GB free tier cap) | | B2 usage | B2 dashboard / API metrics | Informational only (no cap) | Alerts via matchmaker -> webhook to Discord/Slack. Cluster-level monitoring @@ -2485,7 +2412,7 @@ ai-code-battle/ │ ├── acb-mapgen/ # CLI: generate symmetric maps │ ├── acb-worker/ # Container: match execution worker │ ├── acb-evolver/ # Container: LLM evolution pipeline -│ ├── acb-index-builder/ # Container: PostgreSQL -> JSON -> Pages + R2 pruning +│ ├── acb-index-builder/ # Container: PostgreSQL -> JSON -> Pages │ # replay pruning is handled by acb-index-builder (weekly cycle) │ ├── cmd/acb-api/ # Go API service (replaces Cloudflare Worker) @@ -2549,7 +2476,7 @@ ai-code-battle/ │ │ │ ├── blog-post.ts # Markdown renderer for blog content │ │ │ └── skeleton.ts # Per-page skeleton screens │ │ ├── lib/ -│ │ │ ├── data.ts # Data fetching from Pages (same origin) + R2 (cross-origin), caching +│ │ │ ├── data.ts # Data fetching from Pages (same origin) + B2 (cross-origin), caching │ │ │ ├── preload.ts # Hover preload + route cache │ │ │ ├── disclosure.ts # Progressive feature revelation (XP) │ │ │ ├── accessibility.ts # Color palettes, keyboard shortcuts @@ -2638,7 +2565,7 @@ ai-code-battle/ | `acb-api` | `cmd/acb-api/` | Go on Alpine | API + scheduling | Deployment (1 replica) | | `acb-worker` | `cmd/acb-worker/` | Go on Alpine | Match execution | Deployment (2-10 replicas) | | `acb-evolver` | `cmd/acb-evolver/` | Go on Alpine | LLM evolution pipeline | Deployment (1 replica, self-restarts every 4h) | -| `acb-index-builder` | `cmd/acb-index-builder/` | Go on Alpine (includes wrangler) | PostgreSQL -> JSON -> Cloudflare Pages + weekly R2 replay pruning | Deployment (sleep-loop) | +| `acb-index-builder` | `cmd/acb-index-builder/` | Go on Alpine (includes wrangler) | PostgreSQL -> JSON -> Cloudflare Pages | Deployment (sleep-loop) | | `acb-strategy-random` | `bots/random/` | Python 3.13 slim | RandomBot | Deployment (1 replica) | | `acb-strategy-gatherer` | `bots/gatherer/` | Go on Alpine | GathererBot | Deployment (1 replica) | | `acb-strategy-rusher` | `bots/rusher/` | Rust on Alpine | RusherBot | Deployment (1 replica) | @@ -2717,6 +2644,7 @@ Index builder Deployment (every ~15 min build, ~90 min deploy): ├── Query PostgreSQL directly ├── Generate JSON index files └── Deploy to Cloudflare Pages via wrangler pages deploy + ``` ### 11.4 Shared Libraries & Code Reuse @@ -2797,7 +2725,7 @@ match with all visual elements rendering correctly. `rating_history` tables in the `acb` database - Index builder Deployment (`acb-index-builder`): reads PostgreSQL directly, generates index JSON files every ~15 min, deploys to Cloudflare Pages - every ~90 min, manages R2 warm cache (promote from B2, prune old) + every ~90 min - Match worker Deployment (`acb-worker`): BRPOPs jobs from Valkey, runs matches, uploads replays to B2, writes results + Glicko-2 ratings to PostgreSQL @@ -2813,13 +2741,12 @@ System recovers from worker pod failure via the stale job reaper. **Deliverables:** - Cloudflare Pages static SPA (`ai-code-battle.pages.dev`): leaderboard, match history, bot profiles, replay viewer, docs/getting-started page -- SPA fetches replay files from R2 warm cache with B2 cold archive fallback +- SPA fetches replay files directly from B2 (via Cloudflare CDN) - Index builder deploying leaderboard, bot profiles, playlists to Pages - Match workers uploading replays and per-match metadata to B2 -- Index builder promoting recent replays from B2 to R2 warm cache **Exit criteria:** anyone can browse matches, view leaderboards, and watch -replays — SPA from Pages, recent replays from R2, old replays from B2. +replays — SPA from Pages, replay/match data from B2. All data is static and pre-computed. ### Phase 6: Deployment & Production @@ -2829,20 +2756,19 @@ All data is static and pre-computed. namespace, Deployments, Services, SealedSecrets (flat directory structure) - Cloudflare Pages project (`ai-code-battle`, already exists at `ai-code-battle.pages.dev`) -- Cloudflare R2 bucket (warm replay cache, free tier) -- Backblaze B2 bucket (cold replay archive) +- Backblaze B2 bucket (replay/match data storage) - ArgoCD Application syncing the manifests directory - Argo Events sensor: GitHub webhook triggers on push to `ai-code-battle` repo - Argo Workflows: image build (Kaniko -> Forgejo registry), site build (npm build -> artifact for Pages deploy) -- SealedSecrets for PostgreSQL, Valkey, B2, R2, Cloudflare API token +- SealedSecrets for PostgreSQL, Valkey, B2, Cloudflare API token (most already provisioned in the namespace) - Monitoring: matchmaker metrics endpoint + Discord/Slack alerting webhooks -**Exit criteria:** platform is publicly accessible — SPA from Pages, recent -replays from R2, old replays from B2. All K8s manifests are GitOps-managed -by ArgoCD, CI pipelines rebuild images and site on git push, matches run -autonomously, and the leaderboard updates every ~90 minutes. +**Exit criteria:** platform is publicly accessible — SPA from Pages, replay/match +data from B2. All K8s manifests are GitOps-managed by ArgoCD, CI pipelines +rebuild images and site on git push, matches run autonomously, and the +leaderboard updates every ~90 minutes. ### Phase 7: LLM-Driven Evolution @@ -2924,12 +2850,12 @@ event timelines, and shareable bot profile cards. **Deliverables:** - Weekly meta report: auto-generated blog post deployed to Pages, rendered on `/blog` with LLM-enhanced narrative sections -- Public match data: documented static JSON file paths on Pages and R2, +- Public match data: documented static JSON file paths on Pages and B2, OpenAPI-style documentation at `/docs/api`, versioned replay format spec - Accessibility suite: Tol color-blind palette + shape-per-player, keyboard shortcuts for replay viewer, high contrast mode, reduced motion, screen reader transcript, focus indicators -- Live evolution observatory: evolver writes `live.json` to R2 +- Live evolution observatory: evolver writes `live.json` to B2 every cycle, observatory page polls and renders live feed + lineage tree + meta shift chart - Narrative engine: weekly story arc detection, LLM-generated 200-word @@ -4290,17 +4216,18 @@ auto-generated. ### 15.2 Public Match Data (Static JSON) All platform data is pre-computed and stored as static JSON files, split -between **Cloudflare Pages** (indexes) and **Cloudflare R2** (replays and -per-match data). Index files are rebuilt every ~15 min by the index builder -and deployed to Pages every ~90 min. Replays and per-match data are uploaded -to R2 in real time by match workers. The "API" is simply **documented file -paths** -- no dynamic endpoints, no query parameters, no rate limiting needed. +between **Cloudflare Pages** (indexes) and **Backblaze B2** (replays and +per-match data, served via Cloudflare CDN). Index files are rebuilt every ~15 min +by the index builder and deployed to Pages every ~90 min. Replays and per-match +data are uploaded to B2 in real time by match workers. The "API" is simply +**documented file paths** -- no dynamic endpoints, no query parameters, no +rate limiting needed. **Documented data paths:** ``` PAGES = https://aicodebattle.com (Cloudflare Pages) -R2 = https://r2.aicodebattle.com (Cloudflare R2) +B2 = https://b2.aicodebattle.com (Backblaze B2 via Cloudflare CDN) API = https://api.aicodebattle.com (K8s Go API, dynamic only) --- Index files on Pages (deployed every ~90 min by index builder) --- @@ -4348,22 +4275,22 @@ Maps: GET {PAGES}/maps/index.json GET {PAGES}/maps/{map_id}.json ---- Real-time data on R2 (written by workers/evolver) --- +--- Data on B2 (written by workers/evolver, served via Cloudflare CDN) --- Individual match metadata: - GET {R2}/matches/{match_id}.json + GET {B2}/matches/{match_id}.json Replays: - GET {R2}/replays/{match_id}.json.gz + GET {B2}/replays/{match_id}.json.gz Evolution (live feed): - GET {R2}/evolution/live.json + GET {B2}/evolution/live.json Thumbnails: - GET {R2}/thumbnails/{match_id}.png + GET {B2}/thumbnails/{match_id}.png Bot cards: - GET {R2}/cards/{bot_id}.png + GET {B2}/cards/{bot_id}.png ``` **Replay format specification:** @@ -4387,7 +4314,7 @@ no rate limiting -- it's just static files served by Cloudflare. **Why static JSON, not a dynamic API:** -All this data already exists as static files on Pages and R2. The index +All this data already exists as static files on Pages and B2. The index builder Deployment already produces leaderboard.json, bot profiles, match indexes, playlists, etc. Adding a dynamic API layer would add complexity for data that's already pre-computed and publicly readable. Cloudflare @@ -4395,7 +4322,7 @@ serves these globally with automatic CDN caching. Third-party tools just `fetch()` the URLs. If they need to poll for updates, they check the `updated_at` field in each JSON file. Index files -on Pages refresh every ~90 minutes. R2 cache headers guide freshness for +on Pages refresh every ~90 minutes. B2 cache headers guide freshness for replays (immutable) and the evolution live feed (10s). ### 15.3 Accessibility Suite @@ -4509,18 +4436,18 @@ rejected, and promoted. **Data flow:** -The evolver Deployment writes a status file to R2 at each stage of every +The evolver Deployment writes a status file to B2 at each stage of every evolution cycle: ``` -Upload to R2: evolution/live.json -Served as: https://r2.aicodebattle.com/evolution/live.json +Upload to B2: evolution/live.json +Served as: https://b2.aicodebattle.com/evolution/live.json ``` Updated at every state transition: generation start, validation complete, each evaluation match result, promotion decision. At ~15 -minutes per cycle with ~5 state transitions, that's ~20 R2 writes -per hour. R2 serves the file with `Cache-Control: max-age=10`. +minutes per cycle with ~5 state transitions, that's ~20 B2 writes +per hour. B2 serves the file via Cloudflare CDN with `Cache-Control: max-age=10`. **`live.json` schema:** @@ -4654,7 +4581,7 @@ Each entry shows the candidate ID, island, result, and reason. Both visualizations are built from `data/evolution/lineage.json` and `data/evolution/meta.json` (served from Pages, produced by the index builder Deployment). The live feed overlay is the only component that polls -`evolution/live.json` (written by the evolver to R2). +`evolution/live.json` (written by the evolver to B2). ### 15.5 Narrative Engine (Chronicles) @@ -4990,7 +4917,7 @@ downloads replay viewer or WASM code. **Data fetching:** -All data files are served by Cloudflare (Pages for indexes, R2 for replays) with appropriate cache behavior. +All data files are served by Cloudflare (Pages for indexes, B2 via Cloudflare CDN for replays and match data) with appropriate cache behavior. Subsequent visits hit the edge cache. The SPA uses `stale-while-revalidate` pattern: show cached data immediately, fetch fresh data in the background, update the UI when it arrives. The leaderboard never shows a loading