From ca77eec03a37d9c18b58cbdb58f7af66f880c271 Mon Sep 17 00:00:00 2001 From: jedarden Date: Mon, 6 Jul 2026 00:58:40 -0400 Subject: [PATCH] docs(bf-13s0): document blob data structure definition and usage patterns - Located primary blob structure in state.js - Identified all blob fields across the codebase - Documented 3D and 2D rendering pipelines - Listed key files and data flow - Provided modification points for future changes --- notes/bf-13s0.md | 131 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 notes/bf-13s0.md diff --git a/notes/bf-13s0.md b/notes/bf-13s0.md new file mode 100644 index 0000000..ee989b2 --- /dev/null +++ b/notes/bf-13s0.md @@ -0,0 +1,131 @@ +# Blob Data Structure Definition - Research Findings + +## Overview +This document summarizes the blob/person data structure used in the 3D dashboard frontend for the Spaxel system. + +## Primary Definition + +**Location:** `/home/coding/spaxel/dashboard/js/state.js` (line 19) + +```javascript +blobs: {}, // Map of blob_id -> { id, x, y, z, confidence, vx, vy, vz, posture, person, ble_device, trails } +``` + +## Complete Blob Structure Fields + +Based on analysis across the codebase, blob objects contain the following fields: + +### Core Position/Velocity Fields +- `id` - Unique blob identifier +- `x` - X coordinate position (meters) +- `y` - Y coordinate position (height in meters, typically 0 for floor level) +- `z` - Z coordinate position (meters) +- `vx` - Velocity X component (m/s) +- `vy` - Velocity Y component (m/s, typically 0) +- `vz` - Velocity Z component (m/s) + +### Identity/Classification Fields +- `confidence` - Detection confidence (0.0 to 1.0, default 0.5) +- `person` - Legacy person identifier/label +- `personName` - New person name field (higher priority than `person`) +- `person_label` - Alternative person label field +- `person_id` - Person ID reference +- `person_color` - Assigned color for person visualization +- `posture` - Detected posture (e.g., 'standing', 'walking') +- `ble_device` - Associated BLE device reference +- `assignedColor` - Direct color assignment for visualization +- `identityResolved` - Boolean flag indicating if identity is resolved + +### Trail/History Fields +- `trail` - Array of historical positions for trail visualization +- `trails` - Alternative trail storage field + +## Key Files Using Blob Data + +### Primary Definition & State Management +- **state.js** - Central blob structure definition and state management + - `updateBlob(id, updates)` - Update blob in state + - `removeBlob(id)` - Remove blob from state + - `blobs` state object stores all active blobs + +### 3D Visualization +- **viz3d.js** - Main 3D rendering engine + - `applyLocUpdate(blobs)` - Update blob positions in 3D scene + - `_createBlobObj(id)` - Create 3D humanoid mesh for blob + - Used fields: `id`, `x`, `z`, `vx`, `vz`, `trail` + - Renders blob as humanoid mesh with animation + - Supports blob interaction (click, hover, context menu) + +### 2D Top-Down Visualization +- **ambient_renderer.js** - 2D ambient/bird's-eye view + - Used fields: `id`, `x`, `y`, `z`, `confidence`, `personName`, `person_label`, `person`, `assignedColor`, `identityResolved` + - Renders blobs as colored circles with size based on confidence + - Priority for person name: `personName` → `person_label` → `person` + +### WebSocket Data Handling +- **app.js** - Main application message handler + - `handleSnapshot(msg)` - Processes snapshot messages containing `msg.blobs` + - Forwards blobs to `Viz3D.handleLocUpdate({ type: 'loc_update', blobs: msg.blobs })` + +- **websocket.js** - WebSocket connection management + - Stores last snapshot for blob extrapolation on disconnect + - `setLastSnapshot(msg)` - Saves snapshot with blob data + +### Quick Actions & Interaction +- **quick-actions.js** - User interaction handlers + - Used fields: `id`, `personName`, `person`, `assignedColor` + - Person identification functions + - Blob following/filtering features + +### Other Files Using Blobs +- **simple-mode.js** - Simplified view mode blob updates +- **home-cards.js** - Home page card displays using blob data +- **explainability.js** - AI explainability features for blob detections +- **crowdflow.js** - Crowd flow visualization +- **tooltip.js** - Hover tooltips showing blob info +- **fresnel.js** - Fresnel zone visualization + +## Data Flow + +1. **Backend → Frontend (WebSocket)** + - Backend sends snapshot message: `{ type: 'snapshot', blobs: [...] }` + - `app.js` receives and calls `handleSnapshot()` + +2. **State Update** + - `handleSnapshot()` calls `Viz3D.handleLocUpdate({ blobs: msg.blobs })` + - `viz3d.js` processes each blob and updates 3D scene + - State management via `SpaxelState.updateBlob()` (optional) + +3. **Visualization** + - `viz3d.js` renders blobs as 3D humanoid meshes + - `ambient_renderer.js` renders blobs as 2D circles in top-down view + - Both use blob position (`x`, `y`, `z`) and identity fields + +## TypeScript Interface Location + +**No TypeScript interfaces currently exist.** The dashboard uses JavaScript with runtime type checking via JSDoc comments. + +**Recommended file for future TypeScript interfaces:** +- `/home/coding/spaxel/dashboard/js/types.d.ts` (would need to be created) +- Or add JSDoc `@typedef` to `state.js` + +## Modification Points + +To modify the blob data structure: + +1. **Primary definition:** `state.js` line 19 - Update comment with new fields +2. **3D rendering:** `viz3d.js` `applyLocUpdate()` function (line 765) - Process new fields +3. **2D rendering:** `ambient_renderer.js` blob rendering section (line 625+) - Use new fields +4. **WebSocket handling:** `app.js` `handleSnapshot()` - No changes needed if structure is additive +5. **Interaction:** `quick-actions.js` - Update if adding identity-related fields + +## Related Backend Structure + +The frontend blob structure mirrors backend blob/person detection data. Changes should be coordinated with: +- Backend fusion engine +- WebSocket message protocol +- Snapshot serialization format + +## Summary + +The blob data structure is centralized in `state.js` and used extensively across the dashboard for 3D visualization, 2D ambient rendering, and user interactions. The primary definition serves as documentation, but actual field usage varies across files. New fields should be added to the definition comment and then integrated into rendering pipelines as needed.