WHIM

01

Application Overview

Whim is a comprehensive desktop and mobile ecosystem built in Python (Tkinter) that unifies voice recording, AI-powered conversation, messaging, IoT automation, screen sharing, and document editing into a single dark-themed application. It runs on Linux (CARRARA Mint) and connects to Samsung Android devices via a reverse SSH tunnel through a public VPS, LAN, or USB/ADB.

Whim desktop application showing the Sessions tab with all 14 module tabs visible across the header.

Key Highlights

02

Connectivity & Networking

Whim uses a multi-layered connectivity architecture to bridge the desktop application with mobile devices, messaging platforms, and IoT infrastructure.

System Architecture & Network Topology (PlantUML)

Connection Channels

Header Bar Controls

The application header provides real-time connectivity controls:

• WS URL — WebSocket endpoint for the OpenClaw gateway (default: ws://127.0.0.1:18789)
• Token — Authentication token for gateway access (masked input)
• Approvals — Toggle operator approval scope for sensitive actions
• Journal Ingest — Shows the LAN IP and port for the upload server with Start/Stop controls
• Tunnel Status Dot — Green/red indicator showing reverse SSH tunnel health (auto-updates every 10s)
• Whim Status Dot — Green/red indicator showing Whim.m server reachability on localhost:8089
• LLM Model Dropdown — Selects the active Ollama model for Whim.AI (llama3.1:8b-16k, deepseek-r1:32b, etc.). Refresh button fetches available models from Ollama.
• Audio Capture button — Opens a floating always-on-top tool for capturing system audio (HDMI, speakers) as lightweight audio files. See section below.

Two-Row Tab Layout

All 17 module tabs are split across two rows for better visibility and click targets. Row 1 contains the primary workflow tabs (Chat through TRV Cipher). Row 2 contains utility and configuration tabs (Library through Settings). The active tab is highlighted in green with a border accent. Tabs are button-based with hover effects.

03

Desktop Tabs & Features

Whim organizes its functionality into 14 dedicated tabs, each serving a specific domain within the ecosystem.

Desktop Tab Screenshots

Chat Tab

Library Tab

AVR Lab Tab

TRV Cipher Tab

04

Whim.AI — Local LLM Console

The Whim.AI console showing the AI chat panel, presets, observability metrics, and tool command reference.

Whim.AI is the central intelligence hub. It connects to a local Ollama instance for streaming chat completions with full observability.

Layout

• Left Column (40%) — AI chat log with line-number gutter, multi-line input (up to 200 lines), capture buttons, and drag-and-drop file import zone
• Right Column (60%) — Presets panel, observability dashboard, output templates, and scrollable Tools & Commands reference

Presets

Observability Dashboard

Real-time performance telemetry including:

• Token/s — Live tokens per second throughput
• Latency — End-to-end response time
• Prompt Eval — Prompt evaluation latency
• GPU VRAM / Utilization — nvidia-smi integration for GPU monitoring
• CPU / RAM — System resource monitoring
• Context Meter — Visual bar showing used vs. available context window
• Tool Trace — Per-turn trace of tool calls, token counts, and timing

Capture Tools

• Quick Note — Save AI response as a note to ~/Journal
• Journal Entry — Export entire conversation to ~/Journal
• Action Items — Extract task lists and save as Markdown
• Export ODT — Full conversation export as ODT (LibreOffice) or fallback TXT to ~/TRANSCRIPT
• Save to TableReads — Save selected audio to ~/TableReads

Output Templates

One-click templates for structured content: Weekly Recap, Meeting Summary, Script Draft, Debug Report.

Tools & Commands Reference

The right panel lists all available commands organized into categories:

05

Whim.m — Mobile App

Whim.m v3.4 is a full-featured mobile companion app served as a native APK or Progressive Web App (PWA) from either the desktop Whim app or a standalone Python server. It provides voice recording, AI chat (via Ollama proxy), wake word voice commands, a cross-device file library, and inter-device messaging — all organized into five dedicated bottom navigation tabs: REC, LIBRARY, CHAT, WAKE, and DEVICES. A persistent "Listening for Hey Whim" banner runs across the top of every tab, giving always-on wake word access regardless of which tab is active. The app uses a hybrid connection strategy: VPS tunnel by default with Tailscale as an opt-in fallback, switchable from a dropdown in the mobile UI or from the desktop Control Panel.

v3.4 — Five-Tab Navigation

Lenovo Tablet (TB311FU)

REC tab (Lenovo Tablet) — Whim.m v3.4 with record button, EXPORT TO WHIM, SENT TO WHIM file list, "Listening for Hey Whim" banner, and five-tab bottom navigation (REC, LIBRARY, CHAT, WAKE)

LIBRARY tab (Lenovo Tablet) — Shared files across devices with "Pick from Gallery / Screenshots" and "Upload any file to library", persistent wake word banner

CHAT tab (Lenovo Tablet) — Device Chat for inter-device messaging with "Message all devices..." input, attachment button, and Send

WAKE tab (Lenovo Tablet) — Wake Word "Hey Whim" with live waveform visualization, voice profile indicator, VOICE CHAT input, and microphone listening state

Whim.ai welcome screen (Lenovo Tablet) — "Welcome. Ask me anything." with Send button, powered by Llama + OpenClaw

Whim.ai chat (Lenovo Tablet) — AI conversation interface with keyboard open, "Ask anything..." prompt, and on-screen keyboard

Samsung Galaxy S22

Whim.m splash screen (S22) — "Connecting to Whim..." with animated logo and Tailscale endpoint

REC tab (S22) — Voice recorder with timer, EXPORT TO WHIM, file list with playback buttons, and status bar indicators

Whim.ai welcome screen (S22) — "Welcome. Ask me anything." with Screen Share and location FABs

Whim.ai live chat (S22) — AI conversation with desktop app query, crab avatar, and detailed response

Tailscale mesh network — showing michaels-s22 and carraramint devices connected under the Whim Tailscale network

Features

• Five-Tab Navigation — Bottom navigation bar with REC, LIBRARY, CHAT, WAKE, and DEVICES tabs for organized access to all features
• Persistent Wake Word Banner — "Listening for Hey Whim" banner visible across all tabs, providing always-on voice activation without switching to the WAKE tab
• Voice Recorder (REC) — Tap-to-record with real-time timer display, WebM/Opus encoding, and per-file playback buttons
• Export to Whim — One-tap upload of recordings to the desktop ~/Journal directory with progress bar
• File Picker — Choose existing audio files (.m4a, .aac, .ogg, .opus, .flac, .wav, .mp3, .3gp, .amr) for upload
• Sent File List — SENT TO WHIM section showing all uploaded files with size and inline playback
• Health Status Bar — Top bar with dot indicators for tunnel, server, mic, ollama, and TS (Tailscale) connectivity status
• Hybrid Connection — Connection mode dropdown (top-right) switches between VPS Tunnel (default), Tailscale (fallback), or Auto-detect. Persists across sessions via server config.
• Auto-Reconnect — When connection drops, retries with exponential backoff (3s base, 30s max) with visible "Connection lost — reconnecting..." banner. All HTTP requests use retry wrappers.
• File Library (LIBRARY) — Share files between devices; pick from gallery/screenshots or upload any file to the shared library
• Device Chat (CHAT) — Real-time messaging between phones and tablets with "Message all devices..." input and file attachment support
• Wake Word (WAKE) — "Hey Whim" wake word activates voice input; listens continuously and responds with MillyAl cloned voice via XTTS
• Voice Commands — Wake word supports app control via whim-cmd JSON protocol:
  • open_maps — Open Organic Maps for navigation
  • open_app — Launch any app directly on the device via Android intent
  • play_music — Open YouTube Music with optional search query
• Whim.ai Chat — Standalone AI chat screen powered by Llama + OpenClaw with "Ask anything..." prompt
• MillyAl Voice — AI responses are spoken aloud using the MillyAl cloned voice (XTTS), with "Generating voice..." indicator during synthesis
• Organic Maps — Voice-activated navigation: "open maps and set destination to..."
• Screen Share FAB — Floating action button to open the Screen Share server (port 8091)
• PWA Support — Full Progressive Web App manifest with installable home screen icon, service worker, and standalone display mode
• Dark Theme — Matches the desktop Whim aesthetic with #1e1e1e background

Wake Word Command Protocol

When a voice command is recognized, Whim.m embeds a whim-cmd JSON block in the AI response to trigger device actions:

Running Whim.m Standalone

Output will show LAN IP and VPS tunnel URL for connecting from your phone.

APK Variants

06

QR Code System

Whim generates QR codes in two locations to enable instant phone-to-desktop connectivity without typing URLs:

1. TRV Cipher — Upload from Phone Dialog

Clicking "Upload from Phone" in the TRV Cipher tab opens a modal dialog with:

• A dynamically generated QR code encoding the Journal Ingest server URL (e.g., http://192.168.1.100:8088)
• The URL displayed as a clickable, copyable label
• A "Copy URL" button for clipboard access
• Real-time server status and upload counter

The QR code is generated using the qrcode Python library with error correction level M, rendered as a pixel-perfect canvas grid.

2. Screen Share (SS) Tab — QR Panel

The SS tab has a dedicated QR CODE card in the left column that displays a QR code for the screen share server URL. When the server starts, the QR is generated via qrcode.make() and displayed on a Tkinter canvas, scaled to fit the available space.

How to Use

1. Start the relevant server (Journal Ingest or Screen Share)
2. Ensure your phone is on the same WiFi network or connected via the VPS tunnel
3. Scan the QR code with your phone camera or QR scanner
4. The browser opens the Whim.m PWA or Screen Share page

07

Hybrid Tunnel Networking

Whim uses a hybrid two-mode connection strategy: a reverse SSH tunnel through a public VPS as the always-on primary, with Tailscale retained as an opt-in fallback for situations where rock-solid stability is needed (e.g., WiFi↔cellular handoffs). The mode is switchable from the mobile app or the desktop Control Panel.

Connection Modes

Switching Modes

• Mobile app — Connection mode dropdown in the top-right corner of the health bar (VPS Tunnel / Tailscale / Auto-detect)
• Desktop Control Panel — Whim tab → radio buttons push mode changes via /connection_mode API
• Config persistence — Mode saved to config/connection_mode.json

Auto-Reconnect with Exponential Backoff

When any connection drops, the mobile client automatically retries:

• Base delay: 3 seconds (+ 0–2s random jitter)
• Backoff: Doubles each attempt: 3s → 6s → 12s → 24s → 30s (capped)
• Max delay: 30 seconds
• Visual indicator: Pulsing red "Connection lost — reconnecting..." banner
• All fetch calls wrapped with fetchWithRetry (up to 3 attempts per request)

Tailscale Device IPs

How It Works

VPS Tunnel (default): The CARRARA desktop opens an outbound SSH connection to a public VPS. The VPS accepts inbound connections from mobile devices and forwards them through the tunnel back to the desktop. No ports need to be opened on the home router.

Tailscale (fallback): When enabled, phones connect directly to the PC's Tailscale IP (100.69.17.20) via WireGuard mesh. This is more stable during WiFi↔cellular transitions because Tailscale handles NAT traversal and connection migration natively.

Infrastructure

Tunnel Service

The tunnel runs as a persistent systemd service on CARRARA:

Desktop Status Indicators

The Whim Terminal header bar displays two auto-updating status dots that poll every 10 seconds:

System Tray Status

Whim also displays a system tray icon with three states:

Mobile Health Bar

The Whim.m mobile app health bar shows five indicators: tunnel, server, mic, ollama, and TS (Tailscale). The tunnel dot turns green when the phone can reach the Whim server through the VPS, confirming end-to-end tunnel connectivity. The TS dot turns green when Tailscale is running on the PC.

A connection mode dropdown in the top-right corner allows switching between VPS Tunnel, Tailscale, and Auto-detect modes. When disconnected, a pulsing red banner appears: "Connection lost — reconnecting..." with automatic exponential backoff retries.

Mobile Access via Tunnel

When the tunnel is active, the Whim.m standalone server prints the VPS URL alongside the LAN IP:

Troubleshooting

• Tunnel not connecting: ssh -v -R 8089:localhost:8089 root@104.207.140.242 -N
• Port in use on VPS: sudo lsof -i :8089
• Service status: sudo systemctl status whim-tunnel.service
• Restart tunnel: sudo systemctl restart whim-tunnel.service

08

LLMs & AI Stack

Whim uses a fully local AI inference stack powered by Ollama. All models run on the CARRARA machine's GPU with no external API calls.

Models in Use

Ollama Configuration

AI Endpoints

• Desktop Whim.AI — Directly calls http://localhost:11434/api/chat with streaming enabled
• Mobile Whim.m — Proxies /api/chat POST requests through the Ingest server to Ollama, enabling AI on the phone without direct Ollama access
• OpenClaw System Prompt — A comprehensive system prompt that defines OpenClaw as the AI persona with full tool access across all Whim subsystems

Health Monitoring

Both desktop and mobile clients poll /health to check Ollama availability. The health endpoint returns {"status": "ok", "ollama": true/false} by probing http://localhost:11434/api/tags.

09

Voice & Audio Pipeline

AVR Lab (XTTS Voice Synthesis)

The AVR Lab tab provides text-to-speech synthesis using Coqui XTTS v2 running in a dedicated conda environment (xtts).

• Model: tts_models/multilingual/multi-dataset/xtts_v2
• Voices Directory: ~/voices (speaker reference WAV files)
• Output: ~/xtts_out.wav (default) or ~/TableReads/
• Python: ~/miniconda3/envs/xtts/bin/python

TRV Cipher (Hear Me Out)

The TRV Cipher tab is a complete audio transcription workstation:

• Audio Files Browser — Lists all audio files from ~/Journal and ~/AUDIO_JR
• Spectrogram — Real-time FFT spectrogram visualization using NumPy (512-point Hanning window)
• Transport Controls — Play, Pause, Stop buttons with custom icons
• Scrub — Audio cleaning/processing tool
• Transcription — OpenAI Whisper integration via a transcription script
• Export — Save transcripts as ODT files, open in LibreOffice Writer
• Upload from Phone — QR code dialog for phone-to-desktop audio upload

Audio Flow

10

Voice Engine — Wake Word Tuning

The VOICE ENGINE tab is a dedicated audio diagnostics and wake word calibration environment built for use in noisy environments such as vehicles, outdoor settings, or anywhere ambient noise interferes with "Hey Whim" detection. It provides a real-time spectrogram, signal processing controls, and wake word sensitivity tuning — all in a three-column layout.

Whim-Scope (Live Spectrogram)

The top half of the tab displays a real-time frequency heatmap covering the 300 Hz – 8 kHz range, driven by a 512-point Hanning FFT at 16 kHz mono. Key visual features:

• Frequency Heatmap — Color-coded intensity from dark (silence) to bright orange/green (loud signal), updating at ~20 FPS
• Confidence Ghost Bar — A vertical bar on the right edge of the scope showing wake word match confidence in real-time. Grey below 50%, Blue at 50%+ match, Green at 90%+ trigger threshold
• 1–3 kHz Intelligibility Band — When enabled, the critical voice frequency band renders in a distinct cyan/blue tint with dashed boundary lines and a "1-3k" label. This is the frequency range where human voice carries the most intelligibility; if the "M" in "Whim" isn't being caught, low-end noise is masking the nasal resonance in this band.
• Wake Word Overlay — When the engine detects "Whim," a green ">>> WAKE WORD DETECTED <<<" banner flashes across the top of the scope
• Frequency Axis Labels — 300, 1000, 2000, 4000, 6000, 8000 Hz markers along the left edge

Column A: Gain & Noise Floor (Pre-Amp)

Column B: "Hey Whim" Sensitivity

Column C: Optimization & Hardware

Live readouts include inference latency (ms), buffer frame count, CPU usage, and active audio device name. All settings persist across sessions to ~/.openclaw/voice_engine.json.

Hotkeys

Audio Backend

Uses sounddevice (PortAudio) at 16 kHz mono with float32 samples. The audio callback pipeline processes in order: gain → HPF → parametric EQ → AGC → spectral subtraction → FFT → spectrogram → wake word detection. The wake word function is a placeholder (_ve_detect_wake_word) returning energy-based confidence, ready to swap in openWakeWord or Porcupine for actual custom "Hey Whim" inference.

11

Persona — Voice Personalities

The PERSONA tab is a voice personality manager that treats coined responses like playlists. Each voice clone (MillyAI, Revy, future voices) gets its own persona profile with a curated set of responses organized by behavioral situation. When Whim needs to respond to a trigger — wake word, command acknowledgment, error, idle chatter — it pulls from that persona's playlist instead of generating a generic response.

Three-Column Layout

• Column 1: VOICES — Persona selector linked to voice clones from ~/voices/. Create, duplicate, delete personas. The active persona (starred) is what Whim uses for all responses.
• Column 2: RESPONSE PLAYLIST — Filterable list of coined responses with trigger name, category, response text, and cached audio status. Category filter narrows the view. Add, edit, remove, preview, and batch-render all entries.
• Column 3: RESPONSE EDITOR + STATS — Edit individual entries: trigger, category, response text, confidence range, context. Render single entries via XTTS. Stats show total/cached counts, voice clone, cache size.

Behavioral Categories

Confidence-Gated Selection

Each response has a confidence range (e.g., 40–60%). The Voice Engine's wake word confidence score determines which response fires. At 90%+ confidence, wake responses fire. At 40–60%, partial-match misheard responses fire. Below 20%, the strongest "speak up" responses fire. This maps directly to the Confidence Ghost Bar in the Voice Engine tab.

Context-Aware Selection

The context field enables situational awareness. A response tagged "driving" only fires when connected via VPS tunnel (implying mobile/vehicle use). "Morning" fires between 5–10am. "table_read" only fires when AVR Lab is active. Multiple responses matching the same trigger + context are selected randomly to prevent repetition.

Pre-Render Pipeline

Responses are pre-rendered as cached WAV files via the XTTS conda environment (same GPU-accelerated pipeline as AVR Lab). Render All batch-processes every unrendered entry, skipping existing cache. Cached clips play in <100ms instead of waiting 2–5 seconds for live XTTS generation. Cache is stored at ~/voices/personas/[name]/cache/.

Default Persona: MillyAI

Ships with 42 coined responses across all 7 categories: 6 wake word, 8 acknowledgment, 8 misheard, 7 error, 6 narrative, 7 ambient. Ready to render with the MillyAI voice clone.

10

Signal & Discord Integration

Signal Messenger

Whim integrates with Signal via signal-cli running as a local HTTP service:

• Account: Linked phone number for sending/receiving
• HTTP Endpoint: http://127.0.0.1:8080
• Features: Send messages (sig.send), receive messages (sig.recv), list contacts (sig.contacts)
• Desktop App: Signal Desktop can be launched from /opt/Signal/signal-desktop
• DM Policy: Pairing mode for direct messages
• Group Policy: Open for group messages

Discord (OpenClaw Bot)

The Discord tab manages the OpenClaw bot (Enoch persona) with full action control:

• Desktop Launch: Start/stop Discord desktop client from /usr/share/discord/Discord
• Bot Config: View and edit OpenClaw bot configuration from openclaw.json
• Action Toggles: Individual switches for reactions, stickers, emoji uploads, sticker uploads, messages, search, channel info, voice status, moderation, and presence
• Voice TTS: Text-to-speech model overrides for Discord voice channels
• Heartbeat: Show OK heartbeat status
• Group Policy: Open access for all guilds

11

SmartThings IoT Control

The SmartThings tab provides a complete dashboard for Samsung SmartThings device management via the OpenClaw gateway.

Samsung SmartThings hub hardware (photographed from phone, Feb 21, 2026) -- the physical IoT bridge controlled via the Whim SmartThings tab

Features

• Device Scanning — Scan all connected SmartThings devices with auto-refresh (configurable interval, default 30s)
• Advanced Filtering — Search by name, filter by room, capability (switch, lock, thermostat, motion, contact, battery, light, valve, alarm, presence, sensor), offline-only, low battery (<20%), and favorites-only
• Device Table — Sortable treeview showing Favorite, Name, Room, Type, Online, Battery, Health, Last Event, and Capabilities columns
• Device Detail — Drill-down view with full device information
• Favorites — Double-click to star devices; favorites persist across sessions
• Recently Controlled — History log of device control actions with timestamps
• Rate Limiting — Visual indicator for API rate limit status

12

GeoF — Geofence Tracker

The GEOF tab is a geofencing and livestock tracking system designed for hilly terrain (Ozarks). It combines a canvas-based map with real-time collar monitoring via LoRa radio, GPS point-in-polygon fence checking, and ESP32-S3 collar firmware with deep sleep power management.

Architecture

Layout

Collar Status Indicators

Pin Sync & Fence Management

• Sync Pins — Import GPS coordinates from a mobile JSON file (supports [{lat, lon}] or {pins: [...]} format). Auto-builds fence polygon from 3+ pins.
• Load / Save Fence — Persists fence vertices and collar registry to ~/.openclaw/fence_config.json
• Clear Pins — Resets all pins and fence boundary
• Canvas Interaction — Click-drag to pan, scroll wheel to zoom (levels 1–20), coordinate display in status bar

LoRa Bridge Service

The LoRa bridge (services/lora_bridge.py) runs as a subprocess managed from the GeoF tab. It supports three modes:

The bridge performs ray-casting point-in-polygon geofence checks on every packet. If a collar reports a position outside the fence boundary, the packet is tagged with OUTSIDE_FENCE alert.

LoRa Configuration

ESP32-S3 Collar Firmware

Each livestock collar runs on an ESP32-S3 with GPS, LoRa radio (SX1276), and IMU accelerometer. The firmware (Collar/firmware/main.cpp) uses a deep sleep cycle:

• 20-minute heartbeat — Wake from deep sleep, acquire GPS fix, check fence boundary, send LoRa status packet (collar ID, lat/lon, battery %), return to sleep
• 5-minute motion interval — When the IMU detects movement, sleep interval decreases to 5 minutes for higher-resolution tracking
• Emergency broadcast — If GPS position is outside the fence polygon, immediately transmit 3 rapid emergency packets before sleeping
• Battery monitoring — ADC reads voltage divider on LiPo cell (4.2V=100%, 3.0V=0%)

Packet Format

Collars transmit CSV over LoRa: COLLAR_ID,LAT,LON,BATTERY,NAME[,OUTSIDE_FENCE]

File Structure

Heartbeat Monitor

The heartbeat monitor runs as a background timer in the Whim Terminal. Every 20 minutes it scans all registered collars and flags any that have gone silent as STALE or OFFLINE. Alerts appear in the LoRa Log panel and collar table rows change color accordingly.

13

NodeFlow — Visual Node Editor

The NodeFlow tab is a visual node-based flow editor that maps the real-time data pipeline inside Whim. It renders each active component — User Input, Whim Brain (LLM), Opus Droid, OpenClaw Telemetry, and Wisp/GPS — as draggable nodes on an infinite canvas, with dashed edges showing how data flows between them.

Full node graph showing all five default nodes connected with dashed edge lines and the Wisp/GPS endpoint visible

Selecting the Wisp/GPS node populates the Node Inspector with type, position, description, and upstream connections

Architecture

Default Nodes

Layout

Interaction

• Drag Nodes — Click and drag any node to reposition it on the canvas; edges redraw automatically
• Select Node — Click a node to populate the Node Inspector with its details
• Zoom — Scroll wheel zooms in/out on the canvas
• Pan — Right-click (or middle-click) drag to pan the viewport
• Refresh — Re-query all node states from the running system
• Auto-Poll — Toggle continuous polling to keep node data live
• Reset View — Snap zoom and pan back to the default viewport

Node Color Scheme

14

Archive Tab Editor

The Archive tab is a full-featured document editor that saves files to ~/ARCHIVE. All documents created in Whim are stored in this directory.

Document Actions

• New — Create a blank document with auto-dated metadata
• Open — Open .txt or .odt files from ~/ARCHIVE
• Save / Save As — Save with auto-generated filename (Date_Title.txt) and metadata header
• Publish — Mark a document as published with changelog entry
• Delete — Remove selected files from the archive
• Undo / Redo — Full undo/redo stack with unlimited history
• Find / Replace — Modal dialog for text search and bulk replacement
• Word Count — Live word, character, and line count
• Print Preview — (Planned feature)

Formatting Toolbar

• Font Family — Dropdown with system fonts + custom fonts from ~/.openclaw/WhimUI/fonts
• Font Size — 8-24pt range
• Bold / Italic / Underline — Toggle formatting on selection
• Highlight — Yellow highlight on selected text
• Alignment — Left, Center, Right
• Font Color — 16 preset colors + custom color picker
• Bullet Lists — Bullet, Dash, Circle, Square, Triangle, Numbered

File Browser

The right column shows all files in ~/ARCHIVE with refresh, open, and double-click-to-load. A changelog panel at the bottom tracks all document actions with timestamps.

Document Header Format

15

Screen Share System

The SS (Screen Share) tab enables bidirectional visual communication between the desktop and mobile devices.

Architecture

• Desktop → Phone: Captures the desktop screen using mss (Python screen capture), compresses as JPEG, and streams as MJPEG at /desktop_stream
• Phone → Desktop: The phone camera posts JPEG frames to /phone_frame via HTTP POST, displayed on the desktop canvas

Layout

Settings

• FPS: 1-30 frames per second (default: 10)
• Quality: JPEG compression 1-95 (default: 40)
• Camera: Auto-detected from /sys/class/video4linux
• Max resolution: Desktop stream downscaled to 1280px width

Endpoints

16

ADB Portal & Emulator

The Whim ADB Portal is a standalone GUI (whim_adb_portal.py) for managing APK installs and Android emulators, matching the Whim dark theme.

Device Management

• Auto-scan connected ADB devices with model detection
• Install APKs with automatic verification bypass
• Force reinstall with uninstall-before-install flow
• Batch install all Whim APKs at once
• Uninstall com.whim.m package
• Open ADB shell in a terminal emulator
• Take device screenshots (saved to ~/Pictures)

Emulator Profiles

SDK Management

The portal can download and set up the full Android SDK command-line tools (~2 GB), accept licenses, install platform-tools, emulator, and system images, create AVDs with custom device profiles, and launch emulators with GPU acceleration.

17

OpenClaw Gateway

The OpenClaw Gateway is the central command bus that connects the Whim desktop client to the AI agent infrastructure via WebSocket.

Protocol

• Version: Protocol 3
• Auth: Token-based authentication (auth.mode: "token")
• Client ID: tkui
• Client Version: 0.2.0
• Platform: Linux
• Mode: Operator
• Scopes: operator.read, operator.write, operator.approvals (optional)

Connection Flow

1. Connect to WebSocket URL (default: ws://127.0.0.1:18789)
2. Receive challenge with nonce and timestamp
3. Send connect request with protocol version, client info, auth token, and device signature
4. Enter bidirectional message loop (incoming events displayed in Events/Debug tab)

Sessions & Presence

The Sessions tab manages active OpenClaw sessions with auto-refresh, presets, crash recovery, and a Notion integration for session notes. The Presence tab shows real-time online status with heartbeat pings to each connected component.

Events/Debug Log

The Events/Debug tab provides a structured, filterable event log with:

• Module filter: ALL, WS, Gateway, AVR, TRV, Signal, Discord, Whim.ai, UI, Ingest, System
• Level filter: ALL, TRACE, DEBUG, INFO, WARN, ERROR
• Session ID and Request ID filtering
• Regex pattern search with history
• Saved queries for common filters
• JSON and text export
• Auto-scroll toggle and pause functionality
• Maximum 5000 entries with automatic cleanup

18

Settings Tab

The SETTINGS tab provides a three-column configuration panel for managing API keys, LLM models, and application preferences. All settings persist to ~/.openclaw/whim_settings.json.

Column 1: API Keys & Endpoints

Column 2: Model Management

Manages Ollama models directly from the Whim Terminal:

• Default Model — Dropdown to select which model Whim.AI uses (synced with header dropdown)
• Available Models — Lists all models loaded in Ollama with their sizes
• Pull Model — Download a new model by name (e.g. mistral:7b)
• Delete Model — Remove a model from Ollama to free disk space
• Refresh — Re-query Ollama for the current model list

Column 3: App Preferences

• Auto-start Journal Ingest — Launch the upload server on app start
• Auto-connect to Gateway — Connect to OpenClaw WebSocket on launch
• Monitor tunnel & Whim.m status — Background polling for SSH tunnel and Whim.m health
• Theme — Visual theme selector (Dark, Midnight, Solarized Dark)
• Paths — Shows configured directories for Journal, Archive, Config, and Voice Engine

LLM Model Dropdown (Header Bar)

A global model selector in the header bar lets you switch between local LLMs at any time without opening Settings. It shows all models available in Ollama (fetched on startup and via the refresh button). Selecting a model immediately updates Whim.AI's active model for the next prompt. Currently available:

• llama3.1:8b-16k — 4.9 GB, 16K context (default, fast)
• llama3.1:8b — 4.9 GB, standard context
• deepseek-r1:32b — 19.9 GB, reasoning model (slower, smarter)

19

Audio Capture Tool

A floating always-on-top tool window for capturing system audio output as lightweight audio files — no video, just audio. Designed for the use case of turning YouTube videos, podcasts, or any playing audio into portable files you can listen to in the car.

How It Works

Clicking the 🎧 Capture button in the header bar opens a compact floating window that stays on top of all other windows. It captures audio from PipeWire/PulseAudio monitor sources — virtual loopback devices that tap into whatever audio is playing through your speakers or HDMI output. No screen recording, no video — just the audio stream, encoded to a small file.

Controls

Output

Files save to ~/Journal/audio_captures/ with timestamps (e.g. capture_20260317_143022.mp3). The folder link in the tool opens the directory in the file manager. At 128k MP3, a 1-hour podcast capture is roughly 60 MB.

Typical Workflow

1. Start playing a YouTube video or podcast in the browser
2. Click 🎧 Capture in the Whim header bar
3. Select the HDMI or speaker monitor source
4. Click Record — the tool captures audio while you watch/listen
5. Click Stop when done — rename the file to something meaningful
6. Transfer the MP3 to your phone (via Whim.m Library, ADB, or file share) for car listening

20

Technical Stack & Configuration

Runtime Environment

Key Directories

Configuration File

The main configuration lives at ~/.openclaw/openclaw.json and controls:

• Model providers and fallback chains
• Gateway mode, auth tokens, and control UI settings
• Signal channel config (account, HTTP URL, auto-start, DM/group policies)
• Discord channel config (bot token, action toggles, voice TTS, heartbeat)
• Command settings (native commands, skill commands, restart behavior)
• Tool deny lists and message acknowledgment scopes

Singleton Instance

Whim enforces a single instance by binding TCP port 48891. If a second instance is launched, it sends a SHOW signal to the existing instance, which restores and focuses its window.

21

Desktop Environment Customizations

The CARRARA desktop runs Linux Mint with Cinnamon. The following customizations have been applied to the desktop environment for a cleaner workflow and ergonomic comfort.

Start Menu Cleanup

All non-pinned application entries have been removed from the start menu. Only taskbar-pinned favorites remain accessible via the start menu:

Removed .desktop overrides are backed up at ~/.local/share/applications/_backup_removed/. Custom app entries removed include: OpenClaw, Whim ADB Portal, Control Panel, Droid, Revy Acousto, and OnlineChat webapp. System app overrides (Discord, Signal, Audacity, LibreOffice, etc.) were also removed, reverting them to default system entries.

Additionally, all Preferences and Administration category entries (65 items) have been hidden from the start menu via NoDisplay=true overrides. This includes all Cinnamon settings sub-panels (Backgrounds, Themes, Keyboard, Display, etc.), system tools (Firewall, Timeshift, Driver Manager, Update Manager, etc.), and utility launchers. The main System Settings app remains accessible from the pinned taskbar for when settings changes are needed.

ALT Key Shortcuts Disabled

All Cinnamon keyboard shortcuts that use the ALT key have been disabled for ergonomic reasons (wrist rest positioning). This includes:

Window Management (removed)

Workspace Navigation (removed)

System & Media (removed)

22

Windows 11 Support

Whim Terminal runs natively on Windows 11 via a platform compatibility layer that abstracts OS-specific calls (paths, services, audio). The same core codebase powers both the Linux and Windows builds.

Prerequisites

Quick Start

Windows Path Mapping

Whim stores data in Windows-native locations:

Platform Differences

Architecture on Windows

The platform_compat.py module detects the OS at import time and provides correct path defaults, service checkers, audio source enumeration, and file-open commands. The whim_windows.py launcher sets DPI awareness, verifies Ollama, patches path constants, then loads the main app.

Connectivity on Windows

On Windows, the preferred connection method to mobile devices is Tailscale (direct mesh VPN). The Linux systemd SSH tunnel is not available natively on Windows, but Tailscale provides the same end-to-end encrypted connectivity with zero configuration.

Alternatively, use Windows OpenSSH to create a manual tunnel:

Known Limitations

• Voice Engine audio capture uses sounddevice instead of PipeWire monitor sources
• The Linux Control Panel (control_panel.py) is Cinnamon-specific and not included
• ADB Portal requires Android SDK platform-tools in PATH
• Some font rendering may differ slightly between Linux and Windows

22.5

iOS Support (Tahoe)

Whim.m is accessible on iOS devices via Safari or Chrome as a Progressive Web App (PWA). The iOS variant, codenamed Tahoe, connects to the same Whim server backend and provides the same five-tab experience (REC, LIBRARY, CHAT, WAKE, DEVICES) with platform-specific adaptations for Apple hardware.

Prerequisites

Configuration

Platform Differences — iOS vs Android

Known Limitations on iOS

• Background Wake Word — iOS aggressively suspends background WebKit processes. The "Listening for Hey Whim" feature only works while the app is in the foreground.
• Audio Codec — Safari's MediaRecorder outputs MP4/AAC instead of WebM/Opus. The server handles both formats transparently.
• Screen Share — iOS does not support getDisplayMedia() in PWAs, so the desktop-to-phone Screen Share viewer works but phone-to-desktop camera capture may be limited.
• File Picker — iOS restricts filesystem access to the Photo Library and Files app; no direct path selection.

23

Multi-Terminal Sync

The SYNC tab enables state synchronization across multiple Whim Terminal instances running on different machines (Linux + Windows). Seven sync approaches are available, managed through a unified engine.

Sync Approaches

What Gets Synced

Sync Modes

Conflict Resolution

The sync engine uses vector clocks for last-writer-wins conflict resolution. Each node maintains a logical clock that increments on every local change. When merging, the node with the higher clock value wins. For simultaneous edits at equal clocks, the node with the lexicographically higher ID wins (deterministic tie-breaking).

The CRDT layer (Approach 3) provides conflict-free merging for structured data like session lists and chat histories, ensuring eventual consistency without data loss.

Session Mirror

Cast a live Whim Terminal session to another machine for read-only viewing. Enter the host's Tailscale IP in the SYNC tab and click WATCH. The mirror updates in real-time. Optional control handoff allows the viewer to operate the remote session.

Phone Bridge

Uses connected Whim.m phones as store-and-forward relays. When desktop A pushes changes, the phone stores them. When desktop B comes online, it pulls buffered changes from the phone. Leverages the existing Whim.m HTTP server on port 8089.

Security

• API keys, tokens, secrets, and passwords are never synced between nodes
• All WebSocket traffic runs over Tailscale (WireGuard encrypted)
• VPS sync uses SSH key authentication only
• Sensitive fields are automatically stripped before transmission

Configuration

Sync config is stored at:

Quick Start