From d12745ac937061290510a57be399dbaa331b9fb9 Mon Sep 17 00:00:00 2001 From: jedarden Date: Mon, 6 Jul 2026 01:01:17 -0400 Subject: [PATCH] docs(bf-13s0): document blob data structure definition and usage patterns - Located primary blob structure definition in backend Go code - Identified all frontend files that create or use blob data - Documented current blob structure fields (position, velocity, trail, identity) - Specified which TypeScript interface/type files to modify (none - implicit JS structure) - Created comprehensive reference for blob data structure modifications --- notes/bf-13s0.md | 254 +++++++++++++++++++++++++++++------------------ 1 file changed, 157 insertions(+), 97 deletions(-) diff --git a/notes/bf-13s0.md b/notes/bf-13s0.md index ee989b2..b9571a6 100644 --- a/notes/bf-13s0.md +++ b/notes/bf-13s0.md @@ -1,131 +1,191 @@ -# Blob Data Structure Definition - Research Findings +# Blob Data Structure Documentation ## Overview -This document summarizes the blob/person data structure used in the 3D dashboard frontend for the Spaxel system. +This document describes the blob/person data structure used in the Spaxel 3D dashboard frontend. -## Primary Definition +## Backend Blob Structure (Go) -**Location:** `/home/coding/spaxel/dashboard/js/state.js` (line 19) +**Location:** `/home/coding/spaxel/mothership/internal/tracking/tracker.go` (lines 20-40) -```javascript -blobs: {}, // Map of blob_id -> { id, x, y, z, confidence, vx, vy, vz, posture, person, ble_device, trails } +```go +type Blob struct { + ID int + X float64 // metres, room X + Z float64 // metres, room Z + VX float64 // m/s + VZ float64 // m/s + Weight float64 // localisation confidence [0..1] + LastSeen time.Time + Trail [][2]float64 // recent positions (newest last) + ukf *UKF + + // Identity fields (populated by BLE-to-blob matching) + PersonID string `json:"person_id,omitempty"` + PersonLabel string `json:"person_label,omitempty"` + PersonColor string `json:"person_color,omitempty"` + IdentityConfidence float64 `json:"identity_confidence,omitempty"` + IdentitySource string `json:"identity_source,omitempty"` + IdentityLastSeen time.Time `json:"-"` + Posture Posture `json:"posture,omitempty"` +} ``` -## Complete Blob Structure Fields +### Posture Enum +```go +type Posture string -Based on analysis across the codebase, blob objects contain the following fields: +const ( + PostureUnknown Posture = "unknown" + PostureStanding Posture = "standing" + PostureWalking = "walking" + PostureSeated = "seated" + PostureLying Posture = "lying" +) +``` -### Core Position/Velocity Fields -- `id` - Unique blob identifier -- `x` - X coordinate position (meters) -- `y` - Y coordinate position (height in meters, typically 0 for floor level) -- `z` - Z coordinate position (meters) -- `vx` - Velocity X component (m/s) -- `vy` - Velocity Y component (m/s, typically 0) -- `vz` - Velocity Z component (m/s) +## Frontend Blob Structure (JavaScript) -### Identity/Classification Fields -- `confidence` - Detection confidence (0.0 to 1.0, default 0.5) -- `person` - Legacy person identifier/label -- `personName` - New person name field (higher priority than `person`) -- `person_label` - Alternative person label field -- `person_id` - Person ID reference -- `person_color` - Assigned color for person visualization -- `posture` - Detected posture (e.g., 'standing', 'walking') -- `ble_device` - Associated BLE device reference -- `assignedColor` - Direct color assignment for visualization -- `identityResolved` - Boolean flag indicating if identity is resolved +**Location:** Implicitly defined through usage in `/home/coding/spaxel/dashboard/js/viz3d.js` -### Trail/History Fields -- `trail` - Array of historical positions for trail visualization -- `trails` - Alternative trail storage field +The frontend blob object has the following properties: -## Key Files Using Blob Data +### Core Position & Motion +- `id` (number) - Blob unique identifier +- `x` (number) - X position in meters (room coordinate system) +- `z` (number) - Z position in meters (room coordinate system) +- `y` (number) - Y/height position (optional, typically 0 for floor level) +- `vx` (number) - X velocity in m/s (optional, defaults to 0) +- `vz` (number) - Z velocity in m/s (optional, defaults to 0) +- `vy` (number) - Y velocity (optional, used in replay mode) -### Primary Definition & State Management -- **state.js** - Central blob structure definition and state management - - `updateBlob(id, updates)` - Update blob in state - - `removeBlob(id)` - Remove blob from state - - `blobs` state object stores all active blobs +### Trail & History +- `trail` (Array of [number, number]) - Array of [x, z] position tuples representing recent positions (max 60 points) -### 3D Visualization -- **viz3d.js** - Main 3D rendering engine - - `applyLocUpdate(blobs)` - Update blob positions in 3D scene - - `_createBlobObj(id)` - Create 3D humanoid mesh for blob - - Used fields: `id`, `x`, `z`, `vx`, `vz`, `trail` - - Renders blob as humanoid mesh with animation - - Supports blob interaction (click, hover, context menu) +### Identity Fields (from BLE matching) +- `person_id` (string) - UUID from BLE registry +- `person_label` (string) - Display name for the person +- `person_color` (string) - Hex color code for UI (e.g., "#4fc3f7") +- `person_name` (string) - Alternative person name field (used in quick-actions.js) -### 2D Top-Down Visualization -- **ambient_renderer.js** - 2D ambient/bird's-eye view - - Used fields: `id`, `x`, `y`, `z`, `confidence`, `personName`, `person_label`, `person`, `assignedColor`, `identityResolved` - - Renders blobs as colored circles with size based on confidence - - Priority for person name: `personName` → `person_label` → `person` +### Quality & Classification +- `weight` (number) - Localization confidence [0..1] +- `posture` (string) - Estimated body posture: "standing", "walking", "seated", "lying", "unknown" -### WebSocket Data Handling -- **app.js** - Main application message handler - - `handleSnapshot(msg)` - Processes snapshot messages containing `msg.blobs` - - Forwards blobs to `Viz3D.handleLocUpdate({ type: 'loc_update', blobs: msg.blobs })` +## Files That Use Blob Data -- **websocket.js** - WebSocket connection management - - Stores last snapshot for blob extrapolation on disconnect - - `setLastSnapshot(msg)` - Saves snapshot with blob data +### Primary Files -### Quick Actions & Interaction -- **quick-actions.js** - User interaction handlers - - Used fields: `id`, `personName`, `person`, `assignedColor` - - Person identification functions - - Blob following/filtering features +1. **`/home/coding/spaxel/dashboard/js/viz3d.js`** + - `applyLocUpdate(blobs)` (line 765) - Main function that processes blob updates + - Creates 3D humanoid representations for each blob + - Updates positions, velocities, and animations + - Manages blob lifecycle (creation/deletion) + - Functions: + - `_createBlobObj(id)` - Creates blob visualization + - `_updateTrail(obj, trailData)` - Updates position trail + - `_updatePillar(obj, x, z, height)` - Updates vertical reference pillar + - `updateReplayBlobs(blobs, timestamp_ms)` - Handles replay blob data -### Other Files Using Blobs -- **simple-mode.js** - Simplified view mode blob updates -- **home-cards.js** - Home page card displays using blob data -- **explainability.js** - AI explainability features for blob detections -- **crowdflow.js** - Crowd flow visualization -- **tooltip.js** - Hover tooltips showing blob info -- **fresnel.js** - Fresnel zone visualization +2. **`/home/coding/spaxel/dashboard/js/ambient_renderer.js`** + - `updateState(state)` (line 138) - Receives blob data for ambient mode + - Interpolates blob positions for smooth 2D rendering + - Properties accessed: `blob.id`, `blob.x`, `blob.y`, `blob.z` -## Data Flow +3. **`/home/coding/spaxel/dashboard/js/quick-actions.js`** + - Spatial quick actions for blob interactions + - Actions include: identify person, follow, show history, mark incorrect, explain, mark unknown + - Properties accessed: `blob.id`, `blob.personName`, `blob.person` -1. **Backend → Frontend (WebSocket)** - - Backend sends snapshot message: `{ type: 'snapshot', blobs: [...] }` - - `app.js` receives and calls `handleSnapshot()` +4. **`/home/coding/spaxel/dashboard/js/app.js`** + - WebSocket message handler for blob updates + - Forwards blob data to Viz3D and ambient renderer -2. **State Update** - - `handleSnapshot()` calls `Viz3D.handleLocUpdate({ blobs: msg.blobs })` - - `viz3d.js` processes each blob and updates 3D scene - - State management via `SpaxelState.updateBlob()` (optional) +### Supporting Files -3. **Visualization** - - `viz3d.js` renders blobs as 3D humanoid meshes - - `ambient_renderer.js` renders blobs as 2D circles in top-down view - - Both use blob position (`x`, `y`, `z`) and identity fields +5. **`/home/coding/spaxel/dashboard/js/explainability.js`** + - Blob explanation feature ("Why is this here?") + - Uses blob_id for fetching explanations -## TypeScript Interface Location +6. **`/home/coding/spaxel/dashboard/js/feedback.js`** + - Blob detection feedback collection + - Event type: `BLOB_DETECTION` -**No TypeScript interfaces currently exist.** The dashboard uses JavaScript with runtime type checking via JSDoc comments. +## Blob Data Flow -**Recommended file for future TypeScript interfaces:** -- `/home/coding/spaxel/dashboard/js/types.d.ts` (would need to be created) -- Or add JSDoc `@typedef` to `state.js` +``` +Backend (Go) mothership/tracker.go + ↓ WebSocket +Frontend app.js (WebSocket handler) + ↓ +┌────────────────┬─────────────────┐ +│ viz3d.js │ ambient_renderer │ +│ (3D rendering) │ (2D ambient) │ +└────────────────┴─────────────────┘ + ↓ + quick-actions.js (user interactions) +``` -## Modification Points +## Usage Pattern Example -To modify the blob data structure: +From `viz3d.js` line 768-790: +```javascript +blobs.forEach(b => { + seen.add(b.id); + let obj = _blobs3D.get(b.id); + if (!obj) { + obj = _createBlobObj(b.id); + obj.createdAt = now; + _blobs3D.set(b.id, obj); + } -1. **Primary definition:** `state.js` line 19 - Update comment with new fields -2. **3D rendering:** `viz3d.js` `applyLocUpdate()` function (line 765) - Process new fields -3. **2D rendering:** `ambient_renderer.js` blob rendering section (line 625+) - Use new fields -4. **WebSocket handling:** `app.js` `handleSnapshot()` - No changes needed if structure is additive -5. **Interaction:** `quick-actions.js` - Update if adding identity-related fields + obj.group.position.set(b.x, 0, b.z); + obj.lastPosition = { x: b.x, z: b.z }; + obj.lastVelocity = { vx: b.vx || 0, vz: b.vz || 0 }; -## Related Backend Structure + const speed = Math.sqrt(b.vx*b.vx + b.vz*b.vz); + _setPosture(obj.humanoid, speed > 0.25 ? 'walking' : 'standing'); + if (speed > 0.25) { + obj.humanoid.actions.walking.timeScale = Math.min(speed * 1.8, 2.5); + obj.group.rotation.y = Math.atan2(b.vx, b.vz); + } -The frontend blob structure mirrors backend blob/person detection data. Changes should be coordinated with: -- Backend fusion engine -- WebSocket message protocol -- Snapshot serialization format + _updateTrail(obj, b.trail); + if (_room) _updatePillar(obj, b.x, b.z, _room.height); +}); +``` + +## Key Modification Points + +To modify the blob data structure, these are the primary files to update: + +1. **Backend:** `/home/coding/spaxel/mothership/internal/tracking/tracker.go` (lines 20-40) + - Modify the `Blob` struct definition + - Ensure JSON tags are correct for serialization + +2. **Frontend:** `/home/coding/spaxel/dashboard/js/viz3d.js` + - Line 765-795: `applyLocUpdate()` - Update property access patterns + - Line 3623-3638: `updateReplayBlobs()` - Update replay blob structure mapping + - Line 699-733: `_createBlobObj()` - Update blob visualization initialization + +3. **Frontend:** `/home/coding/spaxel/dashboard/js/quick-actions.js` + - Update blob property access in action functions + +4. **Frontend:** `/home/coding/spaxel/dashboard/js/ambient_renderer.js` + - Line 138-178: `updateState()` - Update property access for ambient mode + +## Trail Structure + +The `trail` field contains an array of `[x, z]` coordinate tuples: +- Maximum length: 60 points (TrailMaxLen constant in Go backend) +- Newest positions are last in the array +- Used for rendering the path history behind each blob +- Y-coordinate is fixed at floor level (0.02m) for trail rendering ## Summary -The blob data structure is centralized in `state.js` and used extensively across the dashboard for 3D visualization, 2D ambient rendering, and user interactions. The primary definition serves as documentation, but actual field usage varies across files. New fields should be added to the definition comment and then integrated into rendering pipelines as needed. +The blob data structure is defined on the backend in Go and implicitly on the frontend through JavaScript object property access. The primary frontend files that consume and manipulate blob data are: +- `viz3d.js` (3D visualization) +- `ambient_renderer.js` (2D ambient mode) +- `quick-actions.js` (spatial actions) + +For modifications to the blob structure, update the backend Go struct first, then update property access patterns in all three frontend files.