🧬 Neural Lab Guide

A local AI research platform for DGX Spark — brain simulation, RL training, model surgery, and world building.

🏗️ Architecture Overview

How It Runs

Neural Lab is a Python Flask + Socket.IO server running as a systemd user service on the DGX Spark. The UI is a single-page HTML/JS app served by the Clawboard hub.

┌──────────────────────────────────────────────────────────┐
│  Your Phone / Browser                                     │
│  http://100.109.173.109:8090/31-neural-lab/               │
│  (HTML + JS + Three.js + Socket.IO client)                │
└────────────┬─────────────────────────────────┬────────────┘
             │ REST API                        │ WebSocket
             ▼                                 ▼
┌──────────────────────────────────────────────────────────┐
│  Neural Lab Backend (port 8103)                           │
│  orchestrator.py — Flask + Flask-SocketIO                 │
│  ├── SwarmOrchestrator (brain agents, message bus)        │
│  ├── SimulationEngine (pymunk physics)                    │
│  ├── NeuralLabPlatform (registry, assets, extensions)     │
│  ├── RL Training (stable-baselines3 + PyTorch)            │
│  └── Model Workshop (safetensors inspection)              │
│  Service: neural-lab.service                              │
│  Python: /home/pmello/.openclaw/neural-lab-env/           │
└────────────┬─────────────────────────────────┬────────────┘
             │ HTTP (Ollama API)               │ File I/O
             ▼                                 ▼
┌─────────────────────────┐    ┌────────────────────────────┐
│  Ollama (port 11434)     │    │  Local Files                │
│  qwen3.5:0.8b (agents)  │    │  ~/.openclaw/neural-lab/    │
│  qwen3.5:2b (prefrontal)│    │  ├── snapshots/             │
│  qwen3.5:4b (vision)    │    │  ├── memories-*.json        │
│                          │    │  ├── rl-models/             │
│  llama.cpp (port 18080)  │    │  ├── worlds/                │
│  122B-A10B (narrator)    │    │  ├── extensions/            │
└─────────────────────────┘    │  └── logs/                  │
                                └────────────────────────────┘

Connection Flow

1. Hub server (port 8090) serves the static HTML page
2. Page loads and connects Socket.IO to port 8103 (the backend)
3. UI polls /api/state every 1-2 seconds for agent/brain state
4. Real-time events (messages, training progress) come via WebSocket
5. All brain agents call Ollama (0.8b/2b) for thinking
6. Camera/mic from your phone → backend API → Visual/Auditory Cortex agents
7. Multiple devices connect to the same backend (shared brain)

🚀 Quick Start

1. Start the Brain

Click 🧠 Start Brain. This launches 17 agents across 5 brain regions:

2. Chat with the Brain

Use the 💬 Chat tab or the 🧠 Narrator widget. Your messages flow:

You → Wernicke's (comprehension) → Prefrontal (thinking) → Broca's (response) → You

The Narrator is separate — it observes the brain and explains what's happening.

3. Run a Simulation

1. Go to 🏟️ Sim tab
2. Select an environment (Hide & Seek, Foraging, Maze, Open Field)
3. Click ▶ Start
4. Watch agents move in the 3D arena
5. Enable Brain-controlled to let the LLM drive agent decisions

4. Train with RL

1. Expand 🧪 RL Training at the bottom of the Sim tab
2. Pick algorithm (PPO, A2C, DQN), environment, and timesteps
3. Click 🚀 Train
4. Watch the reward curve update in real-time
5. Trained models auto-save to ~/.openclaw/neural-lab/rl-models/

5. Inspect Models

1. Go to 🔬 Workshop tab — models auto-scan
2. Click any model to see layers, shapes, weight statistics
3. Click 🤖 AI Explain for a plain-language architecture summary
4. Click 📋 Duplicate for Editing to make a safe copy

6. Build Worlds

1. In 🏟️ Sim tab, expand 🏗️ World Builder
2. Select a tool (Wall, Box, Food, Flag, Ramp)
3. Click on the 3D arena to place objects
4. Save your world for later, or load environment templates from the 🧩 Platform tab

🧩 Platform & Extensions

Everything in Neural Lab is a plugin. The 🧩 Platform tab shows all registered components:

• 20 Object Prefabs — walls, boxes, food, traps, portals, water, mud, shelters
• 6 Physics Materials — concrete, ice, rubber, wood, metal, sand
• 5 Environment Templates — arena, maze, CTF, obstacle course, foraging
• 5 Sensors — vision cone, proximity, audio, compass, energy
• 3 Training Algorithms — PPO, A2C, DQN
• 4 Reward Functions — foraging, exploration, survival, team

Writing Extensions

Drop a .py file into ~/.openclaw/neural-lab/extensions/:

# my_extension.py
def register(registry):
    from platform import PluginEntry, PluginType
    registry.register(PluginEntry(
        id='env-my-world',
        name='My Custom World',
        type=PluginType.ENVIRONMENT,
        description='A custom environment',
        tags=['custom'],
    ))

💻 CLI Tools

# Model Workshop
workshop scan              # List all 15+ models
workshop inspect gpt2      # See every layer & weight stats
workshop layers gpt2 attn  # Filter to attention layers
workshop explain whisper    # AI explains the architecture
workshop duplicate gpt2 my-exp  # Safe copy for editing
workshop python            # Interactive Python + safetensors

📡 API Reference

🔧 Service Management

# Check status
systemctl --user status neural-lab.service

# Restart after code changes
systemctl --user restart neural-lab.service

# View logs
journalctl --user -u neural-lab.service -f

# The service uses a Python venv with all dependencies:
# /home/pmello/.openclaw/neural-lab-env/
#   pymunk, torch, stable-baselines3, gymnasium, pettingzoo,
#   safetensors, flask, flask-socketio, requests

📁 File Structure

~/.openclaw/workspace-main/tools/neural-lab/
├── orchestrator.py    # Main server (~2700 lines)
├── simulation.py      # Physics engine (pymunk)
├── rl_env.py          # Gymnasium RL environment wrapper
├── model_workshop.py  # Model inspection/surgery
├── platform.py        # Plugin registry, assets, extensions
├── ARCHITECTURE.md    # Full technical specification
└── RESEARCH.md        # Research directions

~/.openclaw/hub/31-neural-lab/
├── index.html         # Main UI (~3300 lines, 10 tabs)
├── guide.html         # This guide
└── lib/three.min.js   # Three.js (local copy)

~/.openclaw/neural-lab/
├── snapshots/         # Brain state snapshots
├── memories-*.json    # Agent memories per session
├── rl-models/         # Trained RL models (.zip)
├── worlds/            # Saved world layouts
├── extensions/        # Custom plugins (.py)
└── logs/              # Session logs

~/models/workshop/     # Model library (safetensors)
├── bert-tiny-4m/      # 4M params — fast experiments
├── gpt2-124m/         # 124M — classic transformer
├── whisper-tiny-39m/  # Audio transcription model
├── clip-vit-base-150m/# Vision+text multimodal
├── tinyllama-1.1b/    # Llama architecture
└── ... (15 models total)

⚡ Hardware Requirements

Neural Lab v2.2 — Built on DGX Spark with ⚡ by the ARC platform