CLAUDE.md

1# CLAUDE.md
2 
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4 
5## What is this
6 
7Dictate is a voice-to-text tool for Claude Code on Linux. It records speech using faster-whisper and outputs text via clipboard or stdout. Supports local transcription or forwarding audio to a remote GPU server over TCP. Single Python script (`dictate`), no build system.
8 
9## Development
10 
11### Running locally
12 
13```bash
14bash install.sh                    # full install (system deps, venv, launcher)
15bash install-service.sh            # install and start systemd services
16bash update.sh                     # update after code changes (no venv rebuild)
17dictate --serve &                  # start daemon (keeps Whisper model in memory)
18dictate --once                     # send one request to daemon, print text
19dictate                            # standalone push-to-talk mode (no daemon)
20 
21# Network transcription (remote GPU)
22dictate --serve --listen 0.0.0.0:5555   # on GPU machine: headless transcription server
23dictate --serve --server GPU_IP:5555    # on laptop: daemon forwarding to remote
24dictate --once                          # client unchanged
25```
26 
27After install, reboot or re-login once for `input` group membership (required for evdev access).
28 
29After code changes, run `bash update.sh` to deploy — it copies the script, service files, and restarts services. No venv rebuild needed.
30 
31### Testing
32 
33No test suite. Manual testing:
34 
35```bash
36dictate --serve &                  # start daemon
37dictate --once                     # test transcription
38dictate --cpu --model small        # test CPU-only inference
39dictate --list-devices             # verify audio device detection
40dictate --stop                     # stop daemon
41 
42# Network transcription
43dictate --serve --listen 0.0.0.0:5555   # start remote server
44dictate --serve --server 127.0.0.1:5555 # start forwarding daemon
45dictate --once                          # test end-to-end
46```
47 
48Test local daemon, network forwarding, and push-to-talk separately — they share audio and transcription code but have different I/O paths.
49 
50## Architecture
51 
52Single Python script (`dictate`, ~720 lines) with five modes:
53 
541. **Push-to-talk** (default) — loads model, listens for key press via evdev, records, transcribes, copies to clipboard via `wl-copy`
552. **Daemon** (`--serve`) — keeps model loaded, listens on Unix socket (`~/.local/share/dictate/dictate.sock`), maintains rolling 1-second pre-buffer, handles one request at a time
563. **Daemon forwarding** (`--serve --server H:P`) — same as daemon but skips model loading; after recording, forwards audio over TCP to a remote transcription server
574. **TCP transcription server** (`--serve --listen H:P`) — headless, no mic; receives audio over TCP, transcribes with local Whisper model, returns text
585. **Client** (`--once`) — connects to daemon socket, sends JSON request with language + hints, reads newline-delimited JSON responses, prints final text to stdout (unchanged by network mode)
59 
60### Daemon ↔ Client protocol (Unix socket)
61 
62```
63Client → Daemon:  {"language": "en", "initial_prompt": "..."} + shutdown(SHUT_WR)
64Daemon → Client:  {"status": "recording"}\n
65                   {"status": "transcribing"}\n
66                   {"text": "transcribed text here"}\n
67```
68 
69### Daemon ↔ TCP server protocol (network transcription)
70 
71```
72Daemon → Server:
73  4 bytes:  header length (uint32 big-endian)
74  N bytes:  JSON header {"language": "en", "initial_prompt": "...", "audio_length": M}
75  M bytes:  raw float32 audio (16kHz mono)
76 
77Server → Daemon:
78  {"text": "transcribed text here"}\n
79```
80 
81### Audio pipeline
82 
83`sounddevice.InputStream` (16kHz mono float32) → RMS-based silence detection → numpy array → `faster-whisper model.transcribe()`. Silence threshold is calibrated from 0.5s ambient measurement on startup: `ambient * 1.5 + 0.01`, capped at 0.05. Calibration retries on silence (rms=0) or suspiciously high ambient (>0.03, e.g. device switching). A background mic monitor thread detects disconnects and re-calibrates on reconnect, sending desktop notifications via `notify-send`.
84 
85### Key functions
86 
87- `calibrate_mic()` — ambient RMS measurement with retry logic, sets speech threshold
88- `record_until_silence()` — records until post-speech silence or timeout, respects STOP_FLAG
89- `transcribe_audio()` — local transcription via faster-whisper model
90- `transcribe_remote()` — forwards audio to TCP server, returns text
91- `serve()` — daemon loop: socket listener + pre-buffer, transcribes locally or forwards to remote
92- `serve_tcp()` — headless TCP transcription server (for GPU machine)
93- `client_once()` — client: connect, send request, read JSON stream
94- `push_to_talk()` — standalone: evdev key detection + record + transcribe + clipboard
95- `load_hints()` — merges global (`~/.config/dictate/hints.d/`) and project (`.dictate-hints.d/`) hint files
96- `find_audio_device()` — prefers pipewire ALSA device for correct Bluetooth routing
97- `pick_defaults()` — CUDA auto-detection: GPU → medium/int8, CPU → small/int8
98- `parse_addr()` — parses HOST:PORT strings for network modes
99 
100### Claude Code integration
101 
102- `/dictate` command (`dictate.claude-command`) — loops `dictate --once`, accumulates utterances
103- `/dictate-hints` command (`dictate-hints.claude-command`) — auto-generates project vocabulary hints
104- `dictate-editor` — nvim wrapper with F5/F6/F7 voice keybindings, used as `EDITOR=dictate-editor claude`
105 
106## Key design decisions
107 
108- **Wayland only**: `wtype` doesn't work on GNOME Wayland, so clipboard via `wl-copy` is used
109- **PipeWire preference**: `default` ALSA device doesn't route Bluetooth mic correctly; must use pipewire device by name
110- **`hotwords` removed**: tested but degraded transcription with many terms; `initial_prompt` works better
111- **`hallucination_silence_threshold=2`**: prevents Whisper from hallucinating text on silence
112- **Threshold cap 0.05**: prevents false "no speech" from noisy calibration (e.g., AirPods connecting/disconnecting)
113- **Calibration retry on noise**: ambient RMS > 0.03 triggers retry — catches PipeWire route switching transients
114- **Mic health monitor**: background thread checks pre-buffer RMS every 5s, sends `notify-send` on disconnect, re-calibrates on reconnect
115- **Hints are per-request**: sent in client JSON, no daemon restart when switching projects
116- **Network transcription**: local daemon records and forwards raw audio over TCP; remote server is stateless and handles transcription only. `--once` client is completely unaware of network mode
117 
118## Jetson Orin Nano (aarch64)
119 
120JetPack 6.x ships Python 3.10 and no PyPI ctranslate2 CUDA wheels for aarch64. Two extra steps:
121 
1221. **Build ctranslate2 from source** (once, before `install.sh`):
123```bash
124   bash build-ctranslate2.sh
125```
126   This saves a wheel to `~/.local/share/dictate/wheels/`.
1272. **Run install.sh** — detects aarch64, installs the pre-built wheel, skips `nvidia-cublas-cu12` (CUDA libs from JetPack), uses `/usr/local/cuda/lib64` in launcher.
128 
129The `tomli` backport is installed automatically for Python < 3.11.
130 
131## Installed file locations
132 
133```
134~/.local/bin/dictate              # launcher (sets VENV, LD_LIBRARY_PATH)
135~/.local/bin/dictate-editor       # nvim wrapper
136~/.local/share/dictate/venv/      # Python venv
137~/.local/share/dictate/dictate.py # main script (copied from repo)
138~/.config/dictate/config.toml     # user config
139~/.config/dictate/hints.d/        # global vocabulary hints
140```
141