AGENTS.md

1# AGENTS.md
2 
3Behavioural guidance for AI agents working in this repository. Reference
4material for complex procedures lives next to the code — integration
5testing is documented in [`cmd/hi/README.md`](cmd/hi/README.md) and
6[`integration/README.md`](integration/README.md). Read those files
7before running tests or writing new ones.
8 
9Headscale is an open-source implementation of the Tailscale control server
10written in Go. It manages node registration, IP allocation, policy
11enforcement, and DERP routing for self-hosted tailnets.
12 
13## Interaction Rules
14 
15These rules govern how you work in this repo. They are listed first
16because they shape every other decision.
17 
18### Ask with comprehensive multiple-choice options
19 
20When you need to clarify intent, scope, or approach, use the
21`AskUserQuestion` tool (or a numbered list fallback) and present the user
22with a comprehensive set of options. Cover the likely branches explicitly
23and include an "other — please describe" escape.
24 
25- Bad: _"How should I handle expired nodes?"_
26- Good: _"How should expired nodes be handled? (a) Remain visible to peers
27  but marked expired (current behaviour); (b) Hidden from peers entirely;
28  (c) Hidden from peers but visible in admin API; (d) Other."_
29 
30This matters more than you think — open-ended questions waste a round
31trip and often produce a misaligned answer.
32 
33### Read the documented procedure before running complex commands
34 
35Before invoking any `hi` command, integration test, generator, or
36migration tool, read the referenced README in full —
37`cmd/hi/README.md` for running tests, `integration/README.md` for
38writing them. Never guess flags. If the procedure is not documented
39anywhere, ask the user rather than inventing one.
40 
41### Map once, then act
42 
43Use `Glob` / `Grep` to understand file structure, then execute. Do not
44re-explore the same area to "double-check" once you have a plan. Do not
45re-read files you edited in this session — the harness tracks state for
46you.
47 
48### Fail fast, report up
49 
50If a command fails twice with the same error, stop and report the exact
51error to the user with context. Do not loop through variants or
52"try one more thing". A repeated failure means your model of the problem
53is wrong.
54 
55### Confirm scope for multi-file changes
56 
57Before touching more than three files, show the user which files will
58change and why. Use plan mode (`ExitPlanMode`) for non-trivial work.
59 
60### Prefer editing existing files
61 
62Do not create new files unless strictly necessary. Do not generate helper
63abstractions, wrapper utilities, or "just in case" configuration. Three
64similar lines of code is better than a premature abstraction.
65 
66## Quick Start
67 
68```bash
69# Enter the nix dev shell (Go 1.26.1, buf, golangci-lint, prek)
70nix develop
71 
72# Full development workflow: fmt + lint + test + build
73make dev
74 
75# Individual targets
76make build           # build the headscale binary
77make test            # go test ./...
78make fmt             # format Go, docs, proto
79make lint            # lint Go, proto
80make generate        # regenerate protobuf code (after changes to proto/)
81make clean           # remove build artefacts
82 
83# Direct go test invocations
84go test ./...
85go test -race ./...
86 
87# Integration tests — read cmd/hi/README.md first
88go run ./cmd/hi doctor
89go run ./cmd/hi run "TestName"
90```
91 
92Go 1.26.1 minimum (per `go.mod:3`). `nix develop` pins the exact toolchain
93used in CI.
94 
95## Pre-Commit with prek
96 
97`prek` installs git hooks that run the same checks as CI.
98 
99```bash
100nix develop
101prek install            # one-time setup
102prek run                # run hooks on staged files
103prek run --all-files    # run hooks on the full tree
104```
105 
106Hooks cover: file hygiene (trailing whitespace, line endings, BOM),
107syntax validation (JSON/YAML/TOML/XML), merge-conflict markers, private
108key detection, nixpkgs-fmt, prettier, and `golangci-lint` via
109`--new-from-rev=HEAD~1` (see `.pre-commit-config.yaml:59`). A manual
110invocation with an `upstream/main` remote is equivalent:
111 
112```bash
113golangci-lint run --new-from-rev=upstream/main --timeout=5m --fix
114```
115 
116`git commit --no-verify` is acceptable only for WIP commits on feature
117branches — never on `main`.
118 
119## Project Layout
120 
121```
122headscale/
123├── cmd/
124│   ├── headscale/    # Main headscale server binary
125│   └── hi/           # Integration test runner (see cmd/hi/README.md)
126├── hscontrol/        # Core control plane
127├── integration/      # End-to-end Docker-based tests (see integration/README.md)
128├── proto/            # Protocol buffer definitions
129├── gen/              # Generated code (buf output — do not edit)
130├── docs/             # User and ACL reference documentation
131└── packaging/        # Distribution packaging
132```
133 
134### `hscontrol/` packages
135 
136- `app.go`, `handlers.go`, `grpcv1.go`, `noise.go`, `auth.go`, `oidc.go`,
137  `poll.go`, `metrics.go`, `debug.go`, `tailsql.go`, `platform_config.go`
138  — top-level server files
139- `state/` — central coordinator (`state.go`) and the copy-on-write
140  `NodeStore` (`node_store.go`). All cross-subsystem operations go
141  through `State`.
142- `db/` — GORM layer, migrations, schema. `node.go`, `users.go`,
143  `api_key.go`, `preauth_keys.go`, `ip.go`, `policy.go`.
144- `mapper/` — streaming batcher that distributes MapResponses to
145  clients: `batcher.go`, `node_conn.go`, `builder.go`, `mapper.go`.
146  Performance-critical.
147- `policy/` — `policy/v2/` is **the** policy implementation. The
148  top-level `policy.go` is thin wrappers. There is no v1 directory.
149- `routes/`, `dns/`, `derp/`, `types/`, `util/`, `templates/`, `capver/`
150  — routing, MagicDNS, relay, core types, helpers, client templates,
151  capability versioning.
152- `servertest/` — in-memory test harness for server-level tests that
153  don't need Docker. Prefer this over `integration/` when possible.
154- `assets/` — embedded UI assets.
155 
156### `cmd/hi/` files
157 
158`main.go`, `run.go`, `doctor.go`, `docker.go`, `cleanup.go`, `stats.go`,
159`README.md`. **Read `cmd/hi/README.md` before running any `hi` command.**
160 
161## Architecture Essentials
162 
163- **`hscontrol/state/state.go`** is the central coordinator. Cross-cutting
164  operations (node updates, policy evaluation, IP allocation) go through
165  the `State` type, not directly to the database.
166- **`NodeStore`** in `hscontrol/state/node_store.go` is a copy-on-write
167  in-memory cache backed by `atomic.Pointer[Snapshot]`. Every read is a
168  pointer load; writes rebuild a new snapshot and atomically swap. It is
169  the hot path for `MapRequest` processing and peer visibility.
170- **The map-request sync point** is
171  `State.UpdateNodeFromMapRequest()` in
172  `hscontrol/state/state.go:2351`. This is where Hostinfo changes,
173  endpoint updates, and route advertisements land in the NodeStore.
174- **Mapper subsystem** streams MapResponses via `batcher.go` and
175  `node_conn.go`. Changes here affect all connected clients.
176- **Node registration flow**: noise handshake (`noise.go`) → auth
177  (`auth.go`) → state/DB persistence (`state/`, `db/`) → initial map
178  (`mapper/`).
179 
180## Database Migration Rules
181 
182These rules are load-bearing — violating them corrupts production
183databases. The `migrationsRequiringFKDisabled` map in
184`hscontrol/db/db.go:962` is frozen as of 2025-07-02 (see the comment at
185`db.go:989`). All new migrations must:
186 
1871. **Never reorder existing migrations.** Migration order is immutable
188   once committed.
1892. **Only add new migrations to the end** of the migrations array.
1903. **Never disable foreign keys.** No new entries in
191   `migrationsRequiringFKDisabled`.
1924. **Use the migration ID format** `YYYYMMDDHHMM-short-description`
193   (timestamp + descriptive suffix). Example: `202602201200-clear-tagged-node-user-id`.
1945. **Never rename columns** that later migrations reference. Let
195   `AutoMigrate` create a new column if needed.
196 
197## Tags-as-Identity
198 
199Headscale enforces **tags XOR user ownership**: every node is either
200tagged (owned by tags) or user-owned (owned by a user namespace), never
201both. This is a load-bearing architectural rule.
202 
203- **Use `node.IsTagged()`** (`hscontrol/types/node.go:221`) to determine
204  ownership, not `node.UserID().Valid()`. A tagged node may still have
205  `UserID` set for "created by" tracking — `IsTagged()` is authoritative.
206- `IsUserOwned()` (`node.go:227`) returns `!IsTagged()`.
207- Tagged nodes are presented to Tailscale as the special
208  `TaggedDevices` user (`hscontrol/types/users.go`, ID `2147455555`).
209- `SetTags` validation is enforced by `validateNodeOwnership()` in
210  `hscontrol/state/tags.go`.
211- Examples and edge cases live in `hscontrol/types/node_tags_test.go`
212  and `hscontrol/grpcv1_test.go` (`TestSetTags_*`).
213 
214**Don't do this**:
215 
216```go
217if node.UserID().Valid() { /* assume user-owned */ }       // WRONG
218if node.UserID().Valid() && !node.IsTagged() { /* ok */ }  // correct
219```
220 
221## Policy Engine
222 
223`hscontrol/policy/v2/policy.go` is the policy implementation. The
224top-level `hscontrol/policy/policy.go` contains only wrapper functions
225around v2. There is no v1 directory.
226 
227Key concepts an agent will encounter:
228 
229- **Autogroups**: `autogroup:self`, `autogroup:member`, `autogroup:internet`
230- **Tag owners**: IP-based authorization for who can claim a tag
231- **Route approvals**: auto-approval of subnet routes by policy
232- **SSH policies**: SSH access control via grants
233- **HuJSON** parsing for policy files
234 
235For usage examples, read `hscontrol/policy/v2/policy_test.go`. For ACL
236reference documentation, see `docs/`.
237 
238## Integration Testing
239 
240**Before running any `hi` command, read `cmd/hi/README.md` in full.**
241Guessing at `hi` flags leads to broken runs and stale containers.
242 
243Test-authoring patterns (`EventuallyWithT`, `IntegrationSkip`, helper
244variants, scenario setup) are documented in `integration/README.md`.
245 
246Key reminders:
247 
248- Integration test functions **must** start with `IntegrationSkip(t)`.
249- External calls (`client.Status`, `headscale.ListNodes`, etc.) belong
250  inside `EventuallyWithT`; state-mutating commands (`tailscale set`)
251  must not.
252- Tests generate ~100 MB of logs per run under `control_logs/{runID}/`.
253  Prune old runs if disk is tight.
254- Flakes are almost always code, not infrastructure. Read `hs-*.stderr.log`
255  before blaming Docker.
256 
257## Code Conventions
258 
259- **Commit messages** follow Go-style `package: imperative description`.
260  Recent examples from `git log`:
261  - `db: scope DestroyUser to only delete the target user's pre-auth keys`
262  - `state: fix policy change race in UpdateNodeFromMapRequest`
263  - `integration: fix ACL tests for address-family-specific resolve`
264 
265  Not Conventional Commits. No `feat:`/`chore:`/`docs:` prefixes.
266 
267- **Protobuf regeneration**: changes under `proto/` require
268  `make generate` (which runs `buf generate`) and should land in a
269  **separate commit** from the callers that use the regenerated types.
270- **Formatting** is enforced by `golangci-lint` with `golines` (width 88)
271  and `gofumpt`. Run `make fmt` or rely on the pre-commit hook.
272- **Logging** uses `zerolog`. Prefer single-line chains
273  (`log.Info().Str(...).Msg(...)`). For 4+ fields or conditional fields,
274  build incrementally and **reassign** the event variable:
275  `e = e.Str("k", v)`. Forgetting to reassign silently drops the field.
276- **Tests**: prefer `hscontrol/servertest/` for server-level tests that
277  don't need Docker — faster than full integration tests.
278- **View types in read paths**: response serializers must read through
279  `NodeView`/`UserView`/`PreAuthKeyView` accessors. `AsStruct()` clones the
280  whole record on every read — it is only for DB-write/merge clones and mutable
281  working copies, never to build an API response. `grep AsStruct hscontrol/api`
282  must come back empty.
283 
284## Gotchas
285 
286- **Database**: SQLite for local dev, PostgreSQL for integration-heavy
287  tests (`go run ./cmd/hi run "..." --postgres`). Some race conditions
288  only surface on one backend.
289- **NodeStore writes** rebuild a full snapshot. Measure before changing
290  hot-path code.
291- **`.claude/agents/` is deprecated.** Do not create new agent files
292  there. Put behavioural guidance in this file and procedural guidance
293  in the nearest README.
294- **Do not edit `gen/`** — it is regenerated from `proto/` by
295  `make generate`.
296- **Proto changes + code changes should be two commits**, not one.
297