CLAUDE.md

1# cacheengine — standalone prompt-cache engine
2 
3Capability-driven planner plus provider-native wire compiler. Runtime needs no
4gateway process, network, database, or control plane. `engine.go` is
5provider/model agnostic; standalone wire compilers and `profiles.go` bind
6current provider contracts while
7`Driver`/profile resolver seams add providers without planner changes.
8`cachebench` alone owns optional live replay I/O.
9 
10## Layout
11 
12- `types.go` — public capability, plan, result, driver, observation contracts.
13- `engine.go` — stable-prefix frontier, break-even math, volatile/drift guard,
14  scoped affinity-key sharding.
15- `native.go`, `openai.go`, `wire_anthropic_bedrock.go` — strict built-in JSON
16  extraction and wire edits. Production graph imports no gateway adapter;
17  parity tests compare standalone Anthropic/Bedrock behavior to those adapters.
18- `profiles.go` — built-in model thresholds, TTLs, economics, attribution.
19- `raw_usage.go` — provider cache-counter normalization; no dollar minting.
20- `cmd/cache-experiment` — offline transformation/economics fixtures only.
21- `cachebench/` + `cmd/cachebench` — strict 97% request/token gates,
22  semantic-equivalence check, bounded public-corpus import, trace export, and
23  provider-observation replay.
24- `cmd/cache-replay` — opt-in live replay: exact trace reconstruction, provider
25  HTTP auth/SigV4, pre-send full-trace equivalence, bounded absolute-time
26  concurrency, task-verifier protocol, private retained evidence, no retry.
27- `cachebench/scripts/` — hash-pinned LMCache download and parquet-to-JSONL
28  streaming helpers; PyArrow is optional benchmark tooling, never runtime code.
29 
30## Correctness invariants
31 
32- Original bytes survive malformed, unsupported, caller-managed, record,
33  non-PAYG, volatile, drifting, or untransformable requests.
34- Never reorder or rewrite semantic prompt content. Add provider-native cache
35  controls only; OpenAI string-to-content-block conversion is wire-equivalent
36  and only used where explicit breakpoint grammar requires it.
37- `Scope` is required and hashed before provider-visible routing keys. Native
38  profiles bind one provider explicitly.
39- `Epoch` means one frozen stable prefix. Changed bytes require `StartEpoch`;
40  silent prefix drift passes through.
41- `ExpectedCalls` means reuse inside provider TTL. Unknown token counts or cache
42  economics remain zero/unavailable, never guessed.
43- Standalone applied results are `inferred`; pass-through/observe-only results
44  are `none`; `VerifiedSavingsUSD` is always zero. Managed gateway owns any
45  verified accounting method.
46- Custom drivers return original bytes and zero optimizer IDs on uncertainty.
47- Public boundary: never import `cloud/...`.
48 
49## Proof
50 
51```bash
52cd public
53go test -race ./cacheengine/...
54go vet ./cacheengine/...
55go test ./cacheengine -run '^$' -fuzz '^FuzzOptimizeMalformedBuiltinsPassThrough$' -fuzztime=5s
56go test ./cacheengine/cachebench -run '^$' -fuzz '^FuzzAgentCorpusJSONLFailClosed$' -fuzztime=5s
57go test ./cacheengine/cachebench -run '^$' -fuzz '^FuzzObservationJSONLFailClosed$' -fuzztime=5s
58go test ./cacheengine/cachebench -run '^$' -fuzz '^FuzzTraceJSONLFailClosed$' -fuzztime=5s
59go test ./cacheengine/cachebench -run '^$' -fuzz '^FuzzVerificationCommandOutputFailClosed$' -fuzztime=5s
60go test -run '^$' -bench BenchmarkOptimizeOpenAIExplicit -benchmem ./cacheengine
61go run ./cacheengine/cmd/cache-experiment
62go run ./cacheengine/cmd/cachebench
63go run ./cacheengine/cmd/cache-replay -help
64```
65 
66See `README.md` for embedding and custom-driver examples.
67