AGENTS.md

1# AGENTS.md — nullclaw Agent Engineering Protocol
2 
3This file defines the default working protocol for coding agents in this repository.
4Scope: entire repository.
5 
6## 1) Project Snapshot (Read First)
7 
8nullclaw is a Zig-first autonomous AI assistant runtime optimized for:
9 
10- minimal binary size (target: < 1 MB ReleaseSmall)
11- minimal memory footprint (target: < 5 MB peak RSS)
12- zero dependencies beyond libc and optional SQLite
13- full feature parity with ZeroClaw (Rust reference implementation)
14 
15Core architecture is **vtable-driven** and modular. All extension work is done by implementing
16vtable structs and registering them in factory functions.
17 
18Key extension points:
19 
20- `src/providers/root.zig` (`Provider`) — AI model providers
21- `src/channels/root.zig` (`Channel`) — messaging channels
22- `src/tools/root.zig` (`Tool`) — tool execution surface
23- `src/memory/root.zig` (`Memory`) — memory backends
24- `src/observability.zig` (`Observer`) — observability hooks
25- `src/runtime.zig` (`RuntimeAdapter`) — execution environments
26- `src/peripherals.zig` (`Peripheral`) — hardware boards (Arduino, STM32, RPi)
27 
28Current scale: **245 source files, ~204K lines of code, 5,640+ tests**.
29 
30Build and test:
31 
32```bash
33zig build                           # dev build
34zig build -Doptimize=ReleaseSmall  # release build
35zig build test --summary all        # run all tests
36```
37 
38## 2) Deep Architecture Observations (Why This Protocol Exists)
39 
40These codebase realities should drive every design decision:
41 
421. **Vtable + factory architecture is the stability backbone**
43   - Extension points are explicit and swappable via `ptr: *anyopaque` + `vtable: *const VTable`.
44   - Callers must OWN the implementing struct (local var or heap-alloc). Never return a vtable interface pointing to a temporary — the pointer will dangle.
45   - Most features should be added via vtable implementation + factory registration, not cross-cutting rewrites.
46 
472. **Binary size and memory are hard product constraints**
48   - `zig build -Doptimize=ReleaseSmall` is the release target. Every dependency and abstraction has a size cost.
49   - Avoid adding libc calls, runtime allocations, or large data tables without justification.
50   - `MaxRSS` during `zig build test` must stay well under 50 MB.
51 
523. **Security-critical surfaces are first-class**
53   - `src/gateway.zig`, `src/security/`, `src/tools/`, `src/runtime.zig` carry high blast radius.
54   - Defaults are secure-by-default (pairing, HTTPS-only, allowlists, AEAD encryption). Keep it that way.
55 
564. **Zig 0.16.0 API is the baseline — no newer features**
57   - HTTP client: `std.http.Client.fetch()` with `std.Io.Writer.Allocating` for response body capture.
58   - Child processes: `std.process.Child.init(argv, allocator)`, `.Pipe` (capitalized).
59   - stdout: `std.fs.File.stdout().writer(&buf)` → use `.interface` for `print`/`flush`.
60   - `std.io.getStdOut()` does NOT exist in the current Zig stdlib — use `std.fs.File.stdout()`.
61   - SQLite: linked via `/opt/homebrew/opt/sqlite/{lib,include}` on the compile step, not the module.
62   - `ArrayListUnmanaged`: init with `.empty`, pass allocator to every method.
63 
645. **All 5,640+ tests must pass at zero leaks**
65   - The test suite uses `std.testing.allocator` (leak-detecting GPA). Every allocation must be freed.
66   - `Config.load()` allocates — always wrap in `std.heap.ArenaAllocator` in tests and production.
67   - `ChaCha20Poly1305.decrypt` can segfault on tag failure with heap-allocated output on macOS with older Zig toolchains — use a stack buffer then `allocator.dupe()`.
68 
69## 3) Engineering Principles (Normative)
70 
71These principles are mandatory. They are implementation constraints, not suggestions.
72 
73### 3.1 KISS
74 
75Required:
76- Prefer straightforward control flow over meta-programming.
77- Prefer explicit comptime branches and typed structs over hidden dynamic behavior.
78- Keep error paths obvious and localized.
79 
80### 3.2 YAGNI
81 
82Required:
83- Do not add config keys, vtable methods, or feature flags without a concrete caller.
84- Do not introduce speculative abstractions.
85- Keep unsupported paths explicit (`return error.NotSupported`) rather than silent no-ops.
86 
87### 3.3 DRY + Rule of Three
88 
89Required:
90- Duplicate small local logic when it preserves clarity.
91- Extract shared helpers only after repeated, stable patterns (rule-of-three).
92- When extracting, preserve module boundaries and avoid hidden coupling.
93 
94### 3.4 Fail Fast + Explicit Errors
95 
96Required:
97- Prefer explicit errors for unsupported or unsafe states.
98- Never silently broaden permissions or capabilities.
99- In tests: `builtin.is_test` guards are acceptable to skip side effects (e.g., spawning browsers), but the guard must be explicit and documented.
100 
101### 3.5 Secure by Default + Least Privilege
102 
103Required:
104- Deny-by-default for access and exposure boundaries.
105- Never log secrets, raw tokens, or sensitive payloads.
106- All outbound URLs must be HTTPS. HTTP is rejected at the tool layer.
107- Keep network/filesystem/shell scope as narrow as possible.
108 
109### 3.6 Determinism + No Flaky Tests
110 
111Required:
112- Tests must not spawn real network connections, open browsers, or depend on system state.
113- Use `builtin.is_test` to bypass side effects (spawning, opening URLs, real hardware I/O).
114- Tests must be reproducible across macOS and Linux.
115 
116## 4) Repository Map (High-Level)
117 
118```
119src/
120  main.zig              CLI entrypoint and command routing
121  root.zig              module exports (lib root)
122  agent.zig             orchestration loop
123  config.zig            schema + config loading/merging (~/.nullclaw/config.json)
124  gateway.zig           webhook/HTTP gateway server
125  onboard.zig           interactive setup wizard
126  health.zig            component health registry
127  runtime.zig           runtime adapters (native, docker, wasm, cloudflare)
128  tunnel.zig            tunnel providers (cloudflared, ngrok, tailscale, custom)
129  skillforge.zig        skill discovery and integration
130  migration.zig         memory migration from other backends
131  hardware.zig          hardware discovery and management
132  peripherals.zig       hardware peripherals (Arduino, STM32/Nucleo, RPi)
133  security/             policy, pairing, secrets, sandbox backends
134  memory/               SQLite + markdown backends, embeddings, vector search
135  providers/            50+ AI provider implementations (9 core + 41 compatible services)
136  channels/             17 channel implementations
137  tools/                30+ tool implementations
138  agent/                agent loop, context, planner
139```
140 
141## 5) Risk Tiers by Path (Review Depth Contract)
142 
143- **Low risk**: docs, comments, test additions, minor formatting
144- **Medium risk**: most `src/**` behavior changes without boundary/security impact
145- **High risk**: `src/security/**`, `src/gateway.zig`, `src/tools/**`, `src/runtime.zig`, config schema, vtable interfaces
146 
147When uncertain, classify as higher risk.
148 
149## 6) Agent Workflow (Required)
150 
1511. **Read before write** — inspect existing module, vtable wiring, and adjacent tests before editing.
1522. **Define scope boundary** — one concern per change; avoid mixed feature+refactor+infra patches.
1533. **Implement minimal patch** — apply KISS/YAGNI/DRY rule-of-three explicitly.
1544. **Validate** — `zig build test --summary all` must show 0 failures and 0 leaks.
1555. **Document impact** — update comments/docs for behavior changes, risk, and side effects.
156 
157### 6.1 Code Naming Contract (Required)
158 
159Apply these naming rules consistently:
160 
161- Functions and methods: `camelCase` (e.g., `parseCommand`, `buildSimpleRequestBody`, `healthCheck`). This follows standard Zig convention.
162- Variables, fields, modules, files: `snake_case` (e.g., `workspace_dir`, `bot_user_id`, `config_parse.zig`).
163- Types, structs, enums, unions: `PascalCase` (e.g., `AnthropicProvider`, `BrowserTool`, `CommandRiskLevel`).
164- Value constants (numeric limits, URLs, timeouts): `SCREAMING_SNAKE_CASE` (e.g., `MAX_BODY_SIZE`, `DEFAULT_BASE_URL`, `KEY_LEN`).
165- Comptime array/table constants: `snake_case` (e.g., `high_risk_commands`, `compat_providers`, `default_allowed_commands`).
166- Vtable implementer naming: `<Name>Provider`, `<Name>Channel`, `<Name>Tool`, `<Name>Memory`, `<Name>Sandbox`.
167- Vtable function-pointer fields: `camelCase` for new vtables (e.g., `chatWithSystem`, `supportsNativeTools`, `getName`). Note: some older vtable fields use `snake_case` (`supports_streaming`, `record_event`); prefer `camelCase` for new additions and consolidate over time.
168- Factory registration keys: stable, lowercase, user-facing (e.g., `"openai"`, `"telegram"`, `"shell"`). Use hyphens for multi-word keys (e.g., `"together-ai"`, `"aws-bedrock"`).
169- Tests: named with space-separated descriptive phrases as the test block string (e.g., `"command risk low for read commands"`, `"pushover execute missing message"`). Prefix with the subject or subsystem when helpful. Fixtures use neutral names.
170 
171### 6.2 Architecture Boundary Contract (Required)
172 
173- Extend capabilities by adding vtable implementations + factory wiring first.
174- Keep dependency direction inward to contracts: concrete implementations depend on vtable/config/util, not on each other.
175- Avoid cross-subsystem coupling (provider code importing channel internals, tool code mutating gateway policy).
176- Keep module responsibilities single-purpose: orchestration in `agent/`, transport in `channels/`, model I/O in `providers/`, policy in `security/`, execution in `tools/`.
177 
178## 7) Change Playbooks
179 
180### 7.1 Adding a Provider
181 
182- Add `src/providers/<name>.zig` implementing `Provider.VTable` (`chatWithSystem`, `chat`, `supportsNativeTools`, `getName`, `deinit`).
183- Register in `src/providers/root.zig` factory.
184- `chatImpl` must extract system/user from `request.messages` (see existing providers for pattern).
185- Add tests for vtable wiring, error paths, and config parsing.
186 
187### 7.2 Adding a Channel
188 
189- Add `src/channels/<name>.zig` implementing `Channel.VTable`.
190- Keep `send`, `listen`, `name`, `isConfigured` semantics consistent with existing channels.
191- Cover auth/config/health behavior with tests.
192 
193### 7.3 Adding a Tool
194 
195- Add `src/tools/<name>.zig` implementing `Tool.VTable` (`execute`, `name`, `description`, `parameters_json`).
196- Validate and sanitize all inputs. Return `ToolResult`; never panic in the runtime path.
197- Add `builtin.is_test` guard if the tool spawns processes or opens network connections.
198- Register in `src/tools/root.zig`.
199 
200### 7.4 Adding a Peripheral
201 
202- Implement the `Peripheral` interface in `src/peripherals.zig`.
203- Peripherals expose `read`/`write` methods that delegate to real hardware I/O.
204- Use `probe-rs` CLI for STM32/Nucleo flash access; serial JSON protocol for Arduino.
205- Non-Linux platforms must return `error.UnsupportedOperation` (not silent 0).
206 
207### 7.5 Security / Runtime / Gateway Changes
208 
209- Include threat/risk notes in the commit or PR.
210- Add/update tests for failure modes and boundaries.
211- Keep observability useful but non-sensitive (no secrets in logs or errors).
212 
213### 7.6 Updating the Required Zig Version
214 
215Whenever a documentation or code change indicates that the required Zig
216toolchain version is changing (for example: `README.md`, `AGENTS.md`,
217`CLAUDE.md`, `docs/**/installation.md`, `docs/**/development.md`,
218`docs/**/termux.md`, `flake.nix`, `Dockerfile`, `.github/workflows/**`, or any
219`zig version` / "Zig 0.16.0" string in the repo), the per-distro install
220guides must be updated in the same change so they keep matching the pinned
221version.
222 
223Required steps:
224 
2251. Identify the new pinned Zig version (e.g. `0.16.1`, `0.17.0`).
2262. Visit `https://ziglang.org/download/` and locate the section for that exact
227   version. Do not guess URLs — fetch them from the official download page so
228   the filename, archive layout, and presence of the build are confirmed.
2293. For every architecture referenced in the install guides (at minimum
230   `x86_64-linux`, plus any others already documented), copy the canonical
231   tarball URL and checksum from `ziglang.org/download/`.
2324. Update every `docs/**/zig-installation.md` file (currently
233   `docs/en/zig-installation.md` and `docs/zh/zig-installation.md`) so that
234   every URL, checksum, filename, and extracted directory name reflects the new
235   version. The version number must appear consistently in:
236   - the linked download URL
237   - the `wget` command
238   - the checksum verification command
239   - the `tar -xf` filename
240   - the `export PATH="$PWD/zig-...:$PATH"` directory name
2415. Keep all `docs/**/zig-installation.md` translations in sync — if the
242   English version changes, the Chinese version (and any future translations)
243   must be updated in the same commit.
2446. Cross-check that the version pin matches everywhere else it appears
245   (`AGENTS.md` §2.4, `CLAUDE.md` build commands, other `docs/**` pages,
246   `flake.nix`, `Dockerfile`, CI workflows). Mismatches are a hard fail.
247 
248Do not bump the Zig version pin in one place without sweeping all of the
249above. A partial bump leaves users following copy-paste instructions that
250download a tarball that no longer matches the pinned toolchain.
251 
252## 8) Validation Matrix
253 
254Required before any code commit:
255 
256```bash
257zig build test --summary all        # all tests must pass, 0 leaks
258```
259 
260For release changes:
261 
262```bash
263zig build -Doptimize=ReleaseSmall  # must compile clean
264```
265 
266Before any version bump, release branch, or tag work: read `RELEASING.md` and follow it exactly. Do not tag feature branches.
267 
268Additional expectations by change type:
269 
270- **Docs/comments only**: no build required, but verify no broken code references.
271- **Security/runtime/gateway/tools**: include at least one boundary/failure-mode test.
272- **Provider additions**: test vtable wiring + graceful failure without credentials.
273 
274If full validation is impractical, document what was run and what was skipped.
275 
276### 8.1 Test Coverage Mandate
277 
278**Every code change must be accompanied by tests.** No exceptions.
279 
280- **Behavior changes**: add or update tests that directly exercise the changed code path.
281- **Bug fixes**: add a regression test that reproduces the original failure *before* the fix and passes after.
282- **Logging-only or pure error-propagation changes** (e.g., `catch |err|` + `log.err(...)`): unit testing may not be practical. In this case add a comment near the change explaining why formal test coverage is omitted. Example:
283```zig
284  // NOTE: No unit test for this log path — would require a mock session manager.
285  // Covered by manual integration testing against a running NullClaw instance.
286```
287- **Error path resource cleanup**: when a function allocates resources before returning an error, always free them before the `return error.Foo`. Verify with `zig build test` that the test allocator reports 0 leaks.
288- Tests that were added to cover a specific fix must have a comment citing the bug they guard against, e.g.:
289```zig
290  // Regression: GLM-5 returns content=null on context-limit; parseNativeResponse must not silently succeed.
291```
292 
293### 8.2 Git Hooks
294 
295The repository ships with pre-configured hooks in `.githooks/`. Activate once per clone:
296 
297```bash
298git config core.hooksPath .githooks
299```
300 
301Hooks:
302 
303| Hook | What it does |
304|------|-------------|
305| `pre-commit` | Runs `zig fmt --check src/` — blocks commit if any file is not formatted |
306| `pre-push` | Runs `zig build test --summary all` — blocks push if any test fails or leaks |
307 
308To bypass a hook in an emergency: `git commit --no-verify` / `git push --no-verify`.
309 
310## 9) Privacy and Sensitive Data (Required)
311 
312- Never commit real API keys, tokens, credentials, personal data, or private URLs.
313- Use neutral placeholders in tests: `"test-key"`, `"example.com"`, `"user_a"`.
314- Test fixtures must be impersonal and system-focused.
315- Review `git diff --cached` before push for accidental sensitive strings.
316 
317## 10) Anti-Patterns (Do Not)
318 
319- Do not add C dependencies or large Zig packages without strong justification (binary size impact).
320- Do not return vtable interfaces pointing to temporaries — dangling pointer.
321- Do not use `std.io.getStdOut()` — it does not exist in Zig 0.16.
322- Do not silently weaken security policy or access constraints.
323- Do not add speculative config/feature flags "just in case".
324- Do not skip `defer allocator.free(...)` — every allocation must be freed.
325- Do not use `ArrayListUnmanaged.writer()` as `?*Io.Writer` — incompatible types.
326- Do not modify unrelated modules "while here".
327- Do not include personal identity or sensitive information in tests, examples, docs, or commits.
328- Do not use `SQLITE_TRANSIENT` in auto-translated C code — use `SQLITE_STATIC` (null) instead.
329- Do not use heap-allocated output buffers in `ChaCha20Poly1305.decrypt` — use stack buffer + `allocator.dupe()`.
330 
331## 11) Handoff Template (Agent → Agent / Maintainer)
332 
333When handing off work, include:
334 
3351. What changed
3362. What did not change
3373. Validation run and results (`zig build test --summary all`)
3384. Remaining risks / unknowns
3395. Next recommended action
340 
341## 12) Vibe Coding Guardrails
342 
343When working in fast iterative mode:
344 
345- Keep each iteration reversible (small commits, clear rollback).
346- Validate assumptions with code search before implementing.
347- Prefer deterministic behavior over clever shortcuts.
348- Do not "ship and hope" on security-sensitive paths.
349- If uncertain about Zig 0.16 API, check `src/` for existing usage patterns before guessing.
350- If uncertain about architecture, read the vtable interface definition before implementing.
351