AGENTS.md

1# shrink — caveman-shrink, the tool-catalog compressor (commercial Go core + MIT launcher)
2 
3Compress MCP/OpenAI **tool definitions** before they fill the context window. A thin Go wrapper
4over the [engine](../engine/CLAUDE.md)'s `toolschema` compressor: it drops annotation metadata
5(examples, titles, `$comment`, `$schema`) and reduces long descriptions to their lead plus every
6constraint-bearing sentence. It keeps every **selection token** (tool/parameter names, types,
7enums, required) and argument-construction value (`default`, `const`, `$ref` targets)
8byte-for-byte. Everything is `inferred`.
9 
10## Layout
11- `shrink.go` — the library: `Shrink` (catalog → compressed, fail-open S4 transform, durable-CCR-backed), `Recover` (handle → original bytes), `Lint` (per-tool inferred token reductions), `SelectionProfile` (the structural surface — what must survive). `WithStore`/`WithStorePath` select recovery storage; default is `CAVEMAN_CCR_DB` / `~/.caveman/ccr.db`.
12- `cmd/caveman-shrink/` — the CLI: `caveman-shrink` / `shrink` (stdin→stdout), `lint <file>`, and `recover <handle>`.
13- `bin/caveman-shrink.mjs` + `package.json` — the `npx caveman-shrink` launcher around the prebuilt binary.
14 
15## Two correctness boundaries
16`SelectionProfile(input)` == `SelectionProfile(Shrink(input).Output)` always. This proves the
17structural selection surface (names/params/enums/required) survives byte-for-byte. It does **not**
18prove the model selects the same tool: descriptions are model-visible and long descriptions are
19lossy. Behavioral selection equivalence requires model-eval fixtures; keep results `inferred`.
20 
21Selection alone does not guarantee valid arguments. Short descriptions stay whole; long ones
22keep the lead plus every recognised constraint sentence (must/cannot/required/bounds/format/
23ISO/RFC/absolute/etc.). `default`, `const`, and internal `$ref` targets survive. Golden and
24adversarial conformance tests lock these construction surfaces; over-keep is safer than a retry.
25 
26## Conventions
27- Build/test: `make product-build PRODUCT=shrink` / `make product-test PRODUCT=shrink`.
28- The structural compressor lives in the **engine** (`compressors/toolschema.go`); shrink reuses it, never forks it.
29- **This is the dedicated product surface for the `toolschema` compressor.** Engine API/CLI callers
30  can also force it locally. Managed-gateway adapters keep tool arrays in the frozen prompt-cache
31  prefix and never hand them to it; gateway's separate S2 tool-search/deferral path is not
32  compression. Local reductions stay `inferred`. A future billed route for this transform needs
33  cache-versus-schema cost proof, stable byte-identical prefix output, and an eval gate.
34 
35## Gotchas (honesty invariants)
36- **S4 lossy, fail-open** — successful compression changes model-visible bytes; malformed/incompressible catalogs pass through unchanged (`ratio:0`, no handle).
37- **inferred-only** — token counts are estimates via the engine's offline counter; never `verified`, never re-projected.
38- **reversible for real** — a compressing shrink commits the exact original to the durable shared
39  CCR store before returning a handle. A separate process resolves it through `Recover` /
40  `caveman-shrink recover`; failure to open storage makes library `Shrink` return an error, while
41  the CLI catches that error and forwards the original bytes with no handle.
42 
43See ../../CLAUDE.md (root) · ../engine/CLAUDE.md
44