acb-index-builder has been in CrashLoopBackOff for 45 days with silent crashes
after "Copied web assets to output directory". Investigation revealed O(n²) N+1
query loops causing unbounded memory growth and OOMKill.
Changes:
- fetchSeries: batch games query (1000 queries → 1 query) with LIMIT 10000
- fetchChampionshipBracket: batch games query (500 queries → 1 query) with LIMIT 64
- fetchSeasonSnapshots: reduce LIMIT from 10000 to 500
- fetchLineage: reduce LIMIT from 10000 to 1000
- Add strings import for strings.Join in batch queries
These changes prevent the pod from being OOMKilled during fetchAllData() which
runs after copyWebAssets() in the build cycle.
Co-Authored-By: Claude <noreply@anthropic.com>
- Reduce fetchBots LIMIT from 10000 to 2000
- Reduce fetchRatingHistory LIMIT from 10000 to 5000
- Reduce fetchFeedback LIMIT from 5000 to 1000
- Fix O(n²) participant name lookup in generateBotProfiles by using botNameMap
- Add panic recovery in runBuildCycle to log panics via slog before crashing
- Add R2/B2 client helper functions in s3.go
This fixes acb-index-builder CrashLoopBackOff caused by OOMKill after
web asset copy. The pod was silently crashing during fetchAllData()
due to unbounded query results consuming all memory.
Co-Authored-By: Claude <noreply@anthropic.com>
The bot match stats query was introduced in b35a2aa to fix an N+1 query
problem, but it was unbounded and could return an unlimited number of rows.
With many bots in the database, this query could consume excessive memory
and cause OOMKill, resulting in silent crashes after 'Copied web assets'.
Add LIMIT 20000 to prevent unbounded result sets while supporting large
bot populations (the main bots query already limits to 10000 bots).
This fix continues the pattern of adding LIMITs to prevent OOMKill crashes
in acb-index-builder.
Fixes bead bf-2ws: acb-index-builder CrashLoopBackOff investigation
The previous implementation called getBotMatchStats for each bot in a loop,
causing 10,000+ separate database queries when there are many bots. This N+1
query problem caused the pod to exceed memory limits and get OOMKilled,
resulting in CrashLoopBackOff.
Replaced with a single batch query that fetches match stats for all bots at
once, then maps the results to each bot. This reduces database round trips
from O(n) to O(1).
Fixes bead bf-2ws: acb-index-builder CrashLoopBackOff (silent crash after web asset copy)
The fetchSeriesGames function was querying all games for a series without a limit.
With up to 1000 series being fetched, and potentially many games per series,
this could return an unbounded number of rows and cause OOMKill.
A typical series has 3-7 games (best-of-5 or best-of-7), so LIMIT 100 is
more than sufficient to handle edge cases while preventing memory exhaustion.
Fixes acb-index-builder CrashLoopBackOff caused by OOMKill after web asset copy.
- Add LIMIT 10000 to fetchSeasonSnapshots (season_snapshots per season)
- Add LIMIT 500 to fetchChampionshipBracket (series per season bracket)
These queries were called in a loop for each season without LIMITs,
causing acb-index-builder to be OOMKilled with 512Mi memory limit.
Fixes OOMKill after web asset copy in build cycle.
The fetchOpenPredictions function had an unbounded query building a pair
frequency map for rivalry detection. With thousands of bots and matches,
this could return tens of thousands of rows and cause OOMKill.
- Add ORDER BY COUNT(*) DESC to prioritize most common pairings
- Add LIMIT 1000 - sufficient to detect rivalries (pairs with >= 3 matches)
This fixes the 45-day CrashLoopBackOff with 4700+ restarts.
Co-Authored-By: Claude <noreply@anthropic.com>
- Add LIMIT 100 to island populations query (fetchEvolutionMeta)
- Add LIMIT 10000 to lineage programs query (fetchLineage)
These queries had no row limits, causing OOMKill when the programs table
grew large. The pod crashed silently after "Copied web assets" because
Go panics and OOMKills exit without logging to slog.
Fixes acb-index-builder CrashLoopBackOff (4700+ restarts, 45 days).
- Add LIMIT 1000 to fetchChampionshipBracket (was unbounded)
- Reduce fetchSeries from LIMIT 5000 to LIMIT 1000
- Reduce fetchLineage from LIMIT 50000 to LIMIT 10000
- Reduce fetchFeedback from LIMIT 5000 to LIMIT 1000
- Reduce fetchRatingHistory from LIMIT 10000 to LIMIT 5000
The acb-index-builder pod has been in CrashLoopBackOff with OOMKill
(exit code 137) for 45 days with 4713 restarts. These unbounded queries
were loading too much data into memory, causing the kernel to kill the
process before any logs could be written.
Co-Authored-By: Claude <noreply@anthropic.com>
Added LIMIT clauses to 4 unbounded queries that were causing
acb-index-builder to crash with OOMKill after copying web assets:
- fetchPredictorStats: LIMIT 100 (was loading all predictor stats)
- fetchMaps: LIMIT 500 (was loading all maps)
- fetchSeasonSnapshots: LIMIT 1000 (was loading all season snapshots)
- fetchSeasons: LIMIT 100 (was loading all seasons)
These queries had ORDER BY but no LIMIT, causing them to load
massive datasets into memory on each build cycle, leading to
container OOM after the web asset copy phase.
Fixes bead bf-2ws
- Add LIMIT 50000 to fetchLineage (evolution programs table)
- Add LIMIT 10000 to fetchBots
- Add LIMIT 5000 to fetchSeries
These queries had no bounds and could grow arbitrarily large,
causing acb-index-builder to OOM during build cycles.
The lineage table in particular grows unbounded with evolution.
Fixes CrashLoopBackOff that has persisted for 45 days.
- Add matches_today and active_bots fields to LiveData Totals (evolver)
- Query matches table for COUNT(*) WHERE completed_at >= today
- Query bots table for COUNT(*) WHERE status = 'active'
- Add fields to index builder EvolutionMeta struct
- Update homepage to render "X matches today · Y bots active · Gen #Z evolving"
- Add CSS styling for .home-live-stats section
Closes: bf-4m8mo
Index builder:
- Add slog import for structured logging
- Improve fetchEvolutionMeta to return empty meta instead of error when programs table is empty
- Add logging to show evolution system status (running vs not initialized)
- Add logging in generateEvolutionMeta to show when evolution data is written
Evolver:
- Add automatic schema initialization and population seeding in RunEvolutionLoop
- Programs table is now automatically seeded with 6 initial strategy bots on startup
- Log seeding status to indicate whether programs table was already initialized
These changes ensure the evolution system properly initializes when deployed
and provides better visibility into its status via structured logging.
Closes: bf-4zde
- Add island_populations: program count per island
- Add best_ratings: top 10 evolved bots with bot_id, name, rating, island, language
- Add total_promoted and promotion_rate: all-time promotion statistics
- Queries programs table and joins with bots table for current ratings
Closes: bf-1cxv
Plan §15.4 Live Evolution Observatory requires data/evolution/meta.json
and data/evolution/lineage.json files, which the web platform expects
but the index-builder was not generating.
- Add EvolutionMeta type (generation, promoted_today, top_10_count, updated_at)
- Add LineageNode type (full program lineage with parent relationships)
- Add fetchEvolutionMeta() to query evolver database for stats
- Add fetchLineage() to query evolver programs table for lineage tree
- Add generateEvolutionMeta() to write data/evolution/meta.json
- Add generateLineage() to write data/evolution/lineage.json
- Wire generation into generateAllIndexes()
- Add files to R2 upload list
The implementation gracefully handles missing evolver database by
returning empty/placeholder data, allowing the index-builder to run
without evolution data while still producing valid JSON files.
Closes: bf-6cp0
Adds combat_turns metric (distinct turns where ≥1 bot died from enemy
focus-fire, excluding self-collisions). Worker computes it after each
match; index builder sorts matches/index.json and the new most-combat
playlist descending by it, and bumps interest score for combat-heavy
matches so they surface in highlights.
Also switches homepage featured replay default view from influence to
standard so the actual bot-on-bot combat is visible.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add ARIA live region announcement during auto-playback using detailed transcript text
- Transcript panel shows turn-by-turn summaries with current turn highlighting
- T key toggles transcript panel (collapsible UI)
- Panel content is selectable/copyable text for screen reader users
- Fix build errors in clip-maker.ts (remove unused lastExportBlob references)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add ReplayPlayer to type imports in replay-viewer.ts
- Add explicit type annotation for entry parameter in replay.ts transcript map
- Fixes TypeScript compilation errors for §15.3 screen reader transcript feature
Adds generateMapsIndex to acb-index-builder, writing:
- maps/index.json — active/probation/classic maps grouped by player count
- maps/{map_id}.json — full map definition with walls, cores, energy_nodes
Queries maps.map_json column for geometry; adds RawJSON field to MapData
and updates fetchMaps to select it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement auto-curated playlists in the index builder: 12 playlist types
(closest finishes, upsets, comebacks, marathons, rivalry classics, etc.)
with weekly highlight curation. Add DB persistence, R2 pruning exemptions,
frontend pages, and AI commentary enrichment pipeline.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
matches.winner is an INTEGER (player slot), not a bot_id VARCHAR.
Fix two queries that compared mp.bot_id = m.winner (type mismatch)
to use mp.player_slot = m.winner.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
COALESCE(parent_ids, '[]'::json) fails because the column is JSONB
and PostgreSQL won't coerce json to jsonb. Change to '[]'::jsonb.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Created narrative.go with story arc detection per plan §15.5
- Arc types: Rise, Fall, Rivalry, upset, evolution, comeback
- LLMClient for OpenAI-compatible API narrative generation
- generateLLMChronicles() using narrative engine
- Updated blog.go with LLM integration
- Template-based fallback when LLM unavailable
- Added tests in narrative_test.go
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per plan §11.1, the index builder reads PostgreSQL and generates all JSON
index files for Cloudflare Pages deployment:
- main.go: Build cycle orchestration with configurable timeout, self-restart
- config.go: Environment-based configuration with sensible defaults
- db.go: PostgreSQL data fetching for bots, matches, series, seasons, predictions
- generator.go: JSON index generation (leaderboard, bots, matches, playlists)
- deploy.go: Cloudflare Pages deployment via wrangler, R2 warm cache pruning
- Dockerfile: Multi-stage build with Go + Node.js + wrangler CLI
- main_test.go: Tests for config, index generation, playlists
Index builder runs on 15-minute cycles, deploys to Pages every ~90 minutes,
and prunes R2 warm cache weekly to stay within 10GB free tier.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
