AGENTS.md

1# src/hooks/runtime-fallback/ (Reactive Provider Error Recovery)
2 
3**Generated:** 2026-07-17 (7d664b96b)
4 
5## OVERVIEW
6 
729 source files (~2530 LOC). Session Tier hook that **reactively** switches to fallback models when API providers return errors at runtime (429, 503, quota exhausted, cooldown signals). Distinct from `model-fallback` (which applies preemptively at chat.params).
8 
9## RUNTIME-FALLBACK vs MODEL-FALLBACK
10 
11| Aspect | runtime-fallback | model-fallback |
12|--------|-----------------|----------------|
13| **Trigger** | Reactive (after error occurs) | Proactive (at request time) |
14| **Event** | session.error, message.updated, session.status | chat.params |
15| **Config source** | `categories[].fallback_models`, `agents[].fallback_models` | `AGENT_MODEL_REQUIREMENTS` hardcoded chains |
16| **State** | Per-session FallbackState + cooldown tracking | Module-global pendingModelFallbacks |
17| **Use case** | Provider errors during execution | Pre-configured agent fallback chains |
18 
19They operate **independently**; no direct integration.
20 
21## ERROR DETECTION
22 
23### HTTP Status Codes (configurable)
24Default retry codes: `429, 500, 502, 503, 504`
25 
26### Error Message Patterns (constants.ts)
27```
28/rate.?limit/i, /too.?many.?requests/i, /quota.*reset.*after/i,
29/exhausted.*capacity/i, /all.*credentials.*for.*model/i,
30/cool(?:ing)?.?down/i, /model.*not.*supported/i,
31/service.?unavailable/i, /overloaded/i, /temporarily.?unavailable/i
32```
33 
34### Error Type Classification (error-classifier.ts)
35- `missing_api_key`: provider rejects auth
36- `model_not_found`: model unavailable
37- `quota_exceeded`: billing/quota hit
38- Auto-retry signal detection via `auto-retry-signal.ts`: extracts "retrying in ~2 weeks" style signals, triggers immediate fallback
39 
40## FALLBACK STATE MACHINE
41 
42```typescript
43interface FallbackState {
44  originalModel: string
45  currentModel: string
46  fallbackIndex: number
47  failedModels: Map<string, number>  // model → cooldown-until timestamp
48  attemptCount: number
49  pendingFallbackModel?: string
50}
51```
52 
53## FALLBACK CHAIN RESOLUTION (fallback-models.ts)
54 
55Priority order (`getRawFallbackModelsForSession`):
561. **Session category** `fallback_models` (via SessionCategoryRegistry)
572. **Agent config** `fallback_models`, then the agent's own category `fallback_models` (`tryGetFallbackFromAgent`)
583. **Session ID pattern match** (detect agent from session ID, then step 2 logic)
59 
60When the resolved agent is `plan` and `sisyphus_agent` is enabled (`disabled !== true`, `planner_enabled !== false`, `replace_plan !== false`), it inherits `prometheus`'s fallback models after steps 2 and 3. Explicit `plan` `fallback_models` (even `[]`) takes precedence and suppresses inheritance.
61 
62## RETRY FLOW
63 
64```
65session.error / message.updated (with error) / session.status (retry signal)
66  → isRetryableError(error)?
67  → getFallbackModelsForSession(sessionID, agent)
68  → findNextAvailableFallback(): skip cooldown models
69  → prepareFallback(): update state, mark current failed
70  → dispatchFallbackRetry(): toast notification + promptAsync with new model
71  → 30s timeout: abort and try next if exceeded
72```
73 
74## COOLDOWN MECHANISM
75 
76Failed models enter 60s cooldown. `findNextAvailableFallback()` skips models in cooldown, preventing thrashing on persistently failing models.
77 
78## KEY FILES
79 
80| File | Purpose |
81|------|---------|
82| `hook.ts` | `createRuntimeFallbackHook()`: composes all handlers |
83| `event-handler.ts` | Route session lifecycle (created, error, stop, idle) |
84| `message-update-handler.ts` | Handle error parts in `message.updated` |
85| `session-status-handler.ts` | Handle provider retry signals in session.status |
86| `chat-message-handler.ts` | Apply fallback model override on chat.message |
87| `error-classifier.ts` | `isRetryableError()`, `classifyErrorType()` |
88| `auto-retry-signal.ts` | Extract "retrying in..." signals |
89| `fallback-state.ts` | State machine: createFallbackState, prepareFallback, findNextAvailableFallback, isModelInCooldown |
90| `fallback-models.ts` | Resolve chain from config hierarchy (strings + raw objects) |
91| `fallback-bootstrap-model.ts` | Derive initial model when state missing |
92| `fallback-retry-dispatcher.ts` | Toast + dispatch retry orchestration |
93| `auto-retry.ts` | Abort, timeout scheduling, cleanup |
94| `agent-resolver.ts` | Session → agent name normalization |
95| `retry-model-payload.ts` | Build model payload (providerID/modelID/variant/reasoningEffort) |
96| `visible-assistant-response.ts` | Detect if assistant produced real output vs just errors |
97| `last-user-retry-parts.ts` | Extract last user message parts for retry |
98 
99## NOTES
100 
101- Cooldown and failure tracking are **per-session**; concurrent sessions don't share state
102- `visible-assistant-response.ts` prevents retry if the assistant already produced a partial valid response
103- Runtime-fallback is registered in the Session Tier via `create-session-hooks.ts`
104