Compare commits

...

31 Commits

Author SHA1 Message Date
YeonGyu-Kim
13196aedb7 fix: resolve post-rebase runtime fallback merge leftovers 2026-02-21 02:45:48 +09:00
IYODA Atsushi
a8e3e1ea01 fix(test): correct browserProvider assertion to match actual behavior
When browserProvider is not set, agent-browser skill should NOT resolve.
Test assertions were inverted — expected 'Skills not found' but asserted the opposite.
2026-02-21 02:42:20 +09:00
IYODA Atsushi
fcaaa11a06 fix(runtime-fallback): detect type:error message parts for fallback progression 2026-02-21 02:42:20 +09:00
IYODA Atsushi
f82e65fdd1 docs(runtime-fallback): clarify timeout_seconds=0 disables auto-retry detection 2026-02-21 02:42:20 +09:00
Youngbin Kim
eef80a4e23 chore: regenerate JSON schema after merge 2026-02-21 02:42:20 +09:00
Youngbin Kim
b6456faea8 refactor(runtime-fallback): decompose index.ts into focused modules
Split 1021-line index.ts into 10 focused modules per project conventions.

New structure:

- error-classifier.ts: error analysis with dynamic status code extraction

- agent-resolver.ts: agent detection utilities

- fallback-state.ts: state management and cooldown logic

- fallback-models.ts: model resolution from config

- auto-retry.ts: retry helpers with mutual recursion support

- event-handler.ts: session lifecycle events

- message-update-handler.ts: message.updated event handling

- chat-message-handler.ts: chat message interception

- hook.ts: main factory with proper cleanup

- types.ts: updated with HookDeps interface

- index.ts: 2-line barrel re-export

Embedded fixes:

- Fix setInterval leak with .unref()

- Replace require() with ESM import

- Add log warning on invalid model format

- Update sessionLastAccess on normal traffic

- Make extractStatusCode dynamic from config

- Remove unused SessionErrorInfo type

All 61 tests pass without modification.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:42:12 +09:00
Youngbin Kim
22dda6178a docs(config): fix runtime fallback documentation
Remove duplicate Runtime Fallback section from configurations.md.

Fix max_fallback_attempts range from (1-10) to (1-20) to match schema.

Update retry_on_errors default to include 400 status code.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:42:12 +09:00
Youngbin Kim
c54da1e670 docs(config): correct retry_on_errors default in schema comment
Update schema comment to match actual code default [400, 429, 503, 529].

Previously the comment omitted 400 which is included in the code default.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:41:43 +09:00
Youngbin Kim
1835458054 fix(test): revert atlas test to use uiSelectedModel
Revert test name and assertion to original behavior per PR review feedback.

The test now correctly expects Atlas to respect uiSelectedModel instead of using its own fallback chain.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:40:47 +09:00
Youngbin Kim
349e820473 fix(config): allow timeout_seconds to be 0 to disable fallback
Previously, the Zod schema rejected timeout_seconds: 0 due to .min(1).
Now it accepts 0-integer values to allow disabling timeout-based fallback.

- Changed z.number().min(1) to z.number().min(0)
- Updated comment to clarify 0 disables timeout checks
- All tests pass (44 runtime-fallback + 46 schema tests)
- Build successful
2026-02-21 02:40:47 +09:00
Youngbin Kim
68f5d982fc feat(runtime-fallback): add timeout toggle for quota retry detection
Make provider auto-retry signal detection respect timeout_seconds setting:
- When timeout_seconds=0, disable quota-based fallback escalation
- Only treat auto-retry signals as errors when timeout is enabled
- Add test to verify behavior when timeout_seconds is disabled
- Update documentation to explain timeout_seconds=0 behavior

This allows users to disable timeout-based fallbacks while keeping
error-based fallback functionality intact.
2026-02-21 02:40:47 +09:00
Youngbin Kim
8b2ae957e5 feat(runtime-fallback): generalize provider auto-retry signal detection
Refactor retry signal detection to be provider-agnostic:
- Replace hardcoded Copilot/OpenAI checks with generic pattern matching
- Detect any provider message containing limit/quota keywords + [retrying in X]
- Add OpenAI pattern: 'usage limit has been reached [retrying in X]'
- Update logging to use generic 'provider' instead of specific names
- Add 'usage limit has been reached' to RETRYABLE_ERROR_PATTERNS

This enables fallback escalation for any provider that signals automatic
retries due to quota/rate limits, not just Copilot and OpenAI.

Closes PR discussion: generalize retry pattern detection
2026-02-21 02:40:47 +09:00
Youngbin Kim
31f61078b1 docs(runtime-fallback): document retry classes and timeout behavior 2026-02-21 02:40:47 +09:00
Youngbin Kim
6a97f00a22 feat(runtime-fallback): add configurable session timeout controls 2026-02-21 02:40:01 +09:00
Youngbin Kim
ff230df47c fix(runtime-fallback): harden fallback progression and success detection 2026-02-21 02:40:01 +09:00
Youngbin Kim
5a406cab9e refactor(runtime-fallback): extract auto-retry helper and fix provider constraint inconsistency
- Extract duplicated auto-retry logic (~40 lines each) from session.error and
  message.updated handlers into shared autoRetryWithFallback() helper
- Fix userFallbackModels path in model-resolution-pipeline to respect
  constraints.connectedProviders parameter instead of reading cache directly,
  matching the behavior of categoryDefaultModel and fallbackChain paths
2026-02-21 02:40:01 +09:00
Youngbin Kim
fbafb8cf67 fix(runtime-fallback): 9 critical bug fixes for auto-retry, agent preservation, and model override
Bug fixes:
1. extractStatusCode: handle nested data.statusCode (Anthropic error structure)
2. Error regex: relax credit.*balance.*too.*low pattern for multi-char gaps
3. Zod schema: bump max_fallback_attempts from 10 to 20 (config rejected silently)
4. getFallbackModelsForSession: fallback to sisyphus/any agent when session.error lacks agent
5. Model detection: derive model from agent config when session.error lacks model info
6. Auto-retry: resend last user message with fallback model via promptAsync
7. Persistent fallback: override model on every chat.message (not just pendingFallbackModel)
8. Manual model change: detect UI model changes and reset fallback state
9. Agent preservation: include agent in promptAsync body to prevent defaulting to sisyphus

Additional:
- Add sessionRetryInFlight guard to prevent double-retries
- Add resolveAgentForSession with 3-tier resolution (event → session memory → session ID)
- Add normalizeAgentName for display names like "Prometheus (Planner)" → "prometheus"
- Add resolveAgentForSessionFromContext to fetch agent from session messages
- Move AGENT_NAMES and agentPattern to module scope for reuse
- Register runtime-fallback hooks in event.ts and chat-message.ts
- Remove diagnostic debug logging from isRetryableError
- Add 400 to default retry_on_errors and credit/balance patterns to RETRYABLE_ERROR_PATTERNS
2026-02-21 02:39:41 +09:00
youming.tang
708b9ce9ff fix(runtime-fallback): sort agent names by length to fix hyphenated agent detection
The \b word boundary regex treats '-' as a boundary, causing
'sisyphus-junior-session-123' to incorrectly match 'sisyphus'
instead of 'sisyphus-junior'.

Sorting agent names by length (descending) ensures longer names
are matched first, fixing the hyphenated agent detection issue.

Fixes cubic-dev-ai review issue #8
2026-02-21 02:38:17 +09:00
um1ng
d9072b4a98 fix(runtime-fallback): address cubic AI review issues
- Add normalizeFallbackModels helper to centralize string/array normalization (P3)
- Export RuntimeFallbackConfig and FallbackModels types from config/index.ts
- Fix agent detection regex to use word boundaries for sessionID matching
- Improve tests to verify actual fallback switching logic (not just log paths)
- Add SessionCategoryRegistry cleanup in executeSyncTask on completion/error (P2)
- All 24 runtime-fallback tests pass, 115 delegate-task tests pass
2026-02-21 02:37:57 +09:00
um1ng
e9ec4f44e2 feat(runtime-fallback): automatic model switching on API errors
Implements runtime model fallback that automatically switches to backup models
when the primary model encounters transient errors (rate limits, overload, etc.).

Features:
- runtime_fallback configuration with customizable error codes, cooldown, notifications
- Runtime fallback hook intercepts API errors (429, 503, 529)
- Support for fallback_models from agent/category configuration
- Session-state TTL and periodic cleanup to prevent memory leaks
- Robust agent name detection with explicit AGENT_NAMES array
- Session category registry for category-specific fallback lookup

Schema changes:
- Add RuntimeFallbackConfigSchema with enabled, retry_on_errors, max_fallback_attempts,
  cooldown_seconds, notify_on_fallback options
- Add fallback_models to AgentOverrideConfigSchema and CategoryConfigSchema
- Add runtime-fallback to HookNameSchema

Files added:
- src/hooks/runtime-fallback/index.ts - Main hook implementation
- src/hooks/runtime-fallback/types.ts - Type definitions
- src/hooks/runtime-fallback/constants.ts - Constants and defaults
- src/hooks/runtime-fallback/index.test.ts - Comprehensive tests
- src/config/schema/runtime-fallback.ts - Schema definition
- src/shared/session-category-registry.ts - Session category tracking

Files modified:
- src/hooks/index.ts - Export runtime-fallback hook
- src/plugin/hooks/create-session-hooks.ts - Register runtime-fallback hook
- src/config/schema.ts - Export runtime-fallback schema
- src/config/schema/oh-my-opencode-config.ts - Add runtime_fallback config
- src/config/schema/agent-overrides.ts - Add fallback_models to agent config
- src/config/schema/categories.ts - Add fallback_models to category config
- src/config/schema/hooks.ts - Add runtime-fallback to hook names
- src/shared/index.ts - Export session-category-registry
- docs/configurations.md - Add Runtime Fallback documentation
- docs/features.md - Add runtime-fallback to hooks list

Supersedes #1237, #1408
Closes #1408
2026-02-21 02:36:56 +09:00
youming.tang
067c8010be fix: resolve merge conflicts in PR #1408
- Fix bun.lock version conflicts (3.3.1 -> 3.3.2)
- Remove Git conflict markers from docs/configurations.md
- Remove duplicate normalizeFallbackModels, import from shared module
2026-02-21 02:35:03 +09:00
um1ng
17d43672ad refactor(shared): add normalizeFallbackModels utility function
Add shared utility to normalize fallback_models config values.

Handles both single string and array inputs consistently.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:34:28 +09:00
um1ng
8873896432 fix(runtime-fallback): use precise regex patterns for status code matching
Replace word-boundary regex with stricter patterns that match

status codes only at start/end of string or surrounded by whitespace.

Prevents false matches like '1429' or '4290'.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:33:49 +09:00
youming.tang
a206daa437 test(agents): update Atlas uiSelectedModel expectation 2026-02-21 02:33:49 +09:00
youming.tang
538a92ab12 test(delegate-task): stabilize browserProvider and default variant cases 2026-02-21 02:33:49 +09:00
youming.tang
cd3e0ca124 fix(session-category-registry): cleanup entries for task sessions 2026-02-21 02:31:42 +09:00
youming.tang
d947743932 fix(runtime-fallback): per-model cooldown and stricter retry patterns 2026-02-21 02:30:55 +09:00
youming.tang
0ef17aa6c9 docs: add runtime-fallback and fallback_models documentation 2026-02-21 02:30:45 +09:00
Ultrawork Bot
7aafa13b21 feat(fallback_models): complete init-time and runtime integration
Implement full fallback_models support across all integration points:

1. Model Resolution Pipeline (src/shared/model-resolution-pipeline.ts)
   - Add userFallbackModels to ModelResolutionRequest
   - Process user fallback_models before hardcoded fallback chain
   - Support both connected provider and availability checking modes

2. Agent Utils (src/agents/utils.ts)
   - Update applyModelResolution to accept userFallbackModels
   - Inject fallback_models for all builtin agents (sisyphus, oracle, etc.)
   - Support both single string and array formats

3. Model Resolver (src/shared/model-resolver.ts)
   - Add userFallbackModels to ExtendedModelResolutionInput type
   - Pass through to resolveModelPipeline

4. Delegate Task Executor (src/tools/delegate-task/executor.ts)
   - Extract category fallback_models configuration
   - Pass to model resolution pipeline
   - Register session category for runtime-fallback hook

5. Session Category Registry (src/shared/session-category-registry.ts)
   - New module: maps sessionID -> category
   - Used by runtime-fallback to lookup category fallback_models
   - Auto-cleanup support

6. Runtime Fallback Hook (src/hooks/runtime-fallback/index.ts)
   - Check SessionCategoryRegistry first for category fallback_models
   - Fallback to agent-level configuration
   - Import and use SessionCategoryRegistry

Test Results:
- runtime-fallback: 24/24 tests passing
- model-resolver: 46/46 tests passing

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:30:01 +09:00
Rebase Bot
6dc1aff698 fix(runtime-fallback): add Category support and expand test coverage
- Add Category-level fallback_models support in getFallbackModelsForSession()
  - Try agent-level fallback_models first
  - Then try agent's category fallback_models
  - Support all builtin agents including hephaestus, sisyphus-junior, build, plan

- Expand agent name recognition regex to include:
  - hephaestus, sisyphus-junior, build, plan, multimodal-looker

- Add comprehensive test coverage (6 new tests, total 24):
  - Model switching via chat.message hook
  - Agent-level fallback_models configuration
  - SessionID agent pattern detection
  - Cooldown mechanism validation
  - Max attempts limit enforcement

All 24 tests passing

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:28:27 +09:00
Rebase Bot
632570f7ec feat(config): add runtime_fallback and fallback_models schema
Add configuration schemas for runtime model fallback feature:
- RuntimeFallbackConfigSchema with enabled, retry_on_errors,
  max_fallback_attempts, cooldown_seconds, notify_on_fallback
- FallbackModelsSchema for init-time fallback model selection
- Add fallback_models to AgentOverrideConfigSchema and CategoryConfigSchema
- Export types and schemas from config/index.ts

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-02-21 02:28:27 +09:00
39 changed files with 3910 additions and 42 deletions

View File

@@ -100,6 +100,7 @@
"task-resume-info",
"stop-continuation-guard",
"tasks-todowrite-disabler",
"runtime-fallback",
"write-existing-file-guard",
"anthropic-effort",
"hashline-read-enhancer",
@@ -140,6 +141,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -325,6 +339,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -510,6 +537,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -695,6 +735,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -880,6 +933,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -1065,6 +1131,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -1250,6 +1329,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -1435,6 +1527,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -1620,6 +1725,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -1805,6 +1923,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -1990,6 +2121,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -2175,6 +2319,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -2360,6 +2517,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -2545,6 +2715,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -2738,6 +2921,19 @@
"model": {
"type": "string"
},
"fallback_models": {
"anyOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"variant": {
"type": "string"
},
@@ -3147,6 +3343,37 @@
],
"additionalProperties": false
},
"runtime_fallback": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
},
"retry_on_errors": {
"type": "array",
"items": {
"type": "number"
}
},
"max_fallback_attempts": {
"type": "number",
"minimum": 1,
"maximum": 20
},
"cooldown_seconds": {
"type": "number",
"minimum": 0
},
"timeout_seconds": {
"type": "number",
"minimum": 0
},
"notify_on_fallback": {
"type": "boolean"
}
},
"additionalProperties": false
},
"background_task": {
"type": "object",
"properties": {

View File

@@ -163,19 +163,20 @@ Override built-in agent settings:
}
```
Each agent supports: `model`, `temperature`, `top_p`, `prompt`, `prompt_append`, `tools`, `disable`, `description`, `mode`, `color`, `permission`, `category`, `variant`, `maxTokens`, `thinking`, `reasoningEffort`, `textVerbosity`, `providerOptions`.
Each agent supports: `model`, `fallback_models`, `temperature`, `top_p`, `prompt`, `prompt_append`, `tools`, `disable`, `description`, `mode`, `color`, `permission`, `category`, `variant`, `maxTokens`, `thinking`, `reasoningEffort`, `textVerbosity`, `providerOptions`.
### Additional Agent Options
| Option | Type | Description |
| ------------------- | ------- | ----------------------------------------------------------------------------------------------- |
| `category` | string | Category name to inherit model and other settings from category defaults |
| `variant` | string | Model variant (e.g., `max`, `high`, `medium`, `low`, `xhigh`) |
| `maxTokens` | number | Maximum tokens for response. Passed directly to OpenCode SDK. |
| `thinking` | object | Extended thinking configuration for Anthropic models. See [Thinking Options](#thinking-options) below. |
| `reasoningEffort` | string | OpenAI reasoning effort level. Values: `low`, `medium`, `high`, `xhigh`. |
| `textVerbosity` | string | Text verbosity level. Values: `low`, `medium`, `high`. |
| `providerOptions` | object | Provider-specific options passed directly to OpenCode SDK. |
| Option | Type | Description |
| ------------------- | -------------- | ----------------------------------------------------------------------------------------------- |
| `fallback_models` | string/array | Fallback models for runtime switching on API errors. Single string or array of model strings. |
| `category` | string | Category name to inherit model and other settings from category defaults |
| `variant` | string | Model variant (e.g., `max`, `high`, `medium`, `low`, `xhigh`) |
| `maxTokens` | number | Maximum tokens for response. Passed directly to OpenCode SDK. |
| `thinking` | object | Extended thinking configuration for Anthropic models. See [Thinking Options](#thinking-options) below. |
| `reasoningEffort` | string | OpenAI reasoning effort level. Values: `low`, `medium`, `high`, `xhigh`. |
| `textVerbosity` | string | Text verbosity level. Values: `low`, `medium`, `high`. |
| `providerOptions` | object | Provider-specific options passed directly to OpenCode SDK. |
#### Thinking Options (Anthropic)
@@ -714,6 +715,84 @@ Configure concurrency limits for background agent tasks. This controls how many
- Allow more concurrent tasks for fast/cheap models (e.g., Gemini Flash)
- Respect provider rate limits by setting provider-level caps
## Runtime Fallback
Automatically switch to backup models when the primary model encounters retryable API errors (rate limits, overload, etc.) or provider key misconfiguration errors (for example, missing API key). This keeps conversations running without manual intervention.
```json
{
"runtime_fallback": {
"enabled": true,
"retry_on_errors": [400, 429, 503, 529],
"max_fallback_attempts": 3,
"cooldown_seconds": 60,
"timeout_seconds": 30,
"notify_on_fallback": true
}
}
```
| Option | Default | Description |
| ----------------------- | ---------------------- | --------------------------------------------------------------------------- |
| `enabled` | `true` | Enable runtime fallback |
| `retry_on_errors` | `[400, 429, 503, 529]` | HTTP status codes that trigger fallback (rate limit, service unavailable). Also supports certain classified provider errors (for example, missing API key) that do not expose HTTP status codes. |
| `max_fallback_attempts` | `3` | Maximum fallback attempts per session (1-20) |
| `cooldown_seconds` | `60` | Cooldown in seconds before retrying a failed model |
| `timeout_seconds` | `30` | Timeout in seconds for an in-flight fallback request before forcing the next fallback model. **⚠️ Set to `0` to disable auto-retry signal detection** (see below). |
| `notify_on_fallback` | `true` | Show toast notification when switching to a fallback model |
### timeout_seconds: Understanding the 0 Value
**⚠️ IMPORTANT**: Setting `timeout_seconds: 0` **disables auto-retry signal detection**. This is a critical behavior change:
| Setting | Behavior |
|---------|----------|
| `timeout_seconds: 30` (default) | ✅ **Full fallback coverage**: Error-based fallback (429, 503, etc.) + auto-retry signal detection (provider messages like "retrying in 8h") |
| `timeout_seconds: 0` | ⚠️ **Limited fallback**: Only error-based fallback works. Provider retry messages are **completely ignored**. Timeout-based escalation is **disabled**. |
**When `timeout_seconds: 0`:**
- ✅ HTTP errors (429, 503, 529) still trigger fallback
- ✅ Provider key errors (missing API key) still trigger fallback
- ❌ Provider retry messages ("retrying in Xh") are **ignored**
- ❌ Timeout-based escalation is **disabled**
- ❌ Hanging requests do **not** advance to the next fallback model
**Recommendation**: Use a non-zero value (e.g., `30` seconds) to enable full fallback coverage. Only set to `0` if you explicitly want to disable auto-retry signal detection.
### How It Works
1. When an API error matching `retry_on_errors` occurs (or a classified provider key error such as missing API key), the hook intercepts it
2. The next request automatically uses the next available model from `fallback_models`
3. Failed models enter a cooldown period before being retried
4. If `timeout_seconds > 0` and a fallback provider hangs, timeout advances to the next fallback model
5. Toast notification (optional) informs you of the model switch
### Configuring Fallback Models
Define `fallback_models` at the agent or category level:
```json
{
"agents": {
"sisyphus": {
"model": "anthropic/claude-opus-4-5",
"fallback_models": ["openai/gpt-5.2", "google/gemini-3-pro"]
}
},
"categories": {
"ultrabrain": {
"model": "openai/gpt-5.2-codex",
"fallback_models": ["anthropic/claude-opus-4-5", "google/gemini-3-pro"]
}
}
}
```
When the primary model fails:
1. First fallback: `openai/gpt-5.2`
2. Second fallback: `google/gemini-3-pro`
3. After `max_fallback_attempts`, returns to primary model
## Categories
Categories enable domain-specific task delegation via the `task` tool. Each category applies runtime presets (model, temperature, prompt additions) when calling the `Sisyphus-Junior` agent.
@@ -830,15 +909,75 @@ Add your own categories or override built-in ones:
}
```
Each category supports: `model`, `temperature`, `top_p`, `maxTokens`, `thinking`, `reasoningEffort`, `textVerbosity`, `tools`, `prompt_append`, `variant`, `description`, `is_unstable_agent`.
Each category supports: `model`, `fallback_models`, `temperature`, `top_p`, `maxTokens`, `thinking`, `reasoningEffort`, `textVerbosity`, `tools`, `prompt_append`, `variant`, `description`, `is_unstable_agent`.
### Additional Category Options
| Option | Type | Default | Description |
| ------------------ | ------- | ------- | --------------------------------------------------------------------------------------------------- |
| `description` | string | - | Human-readable description of the category's purpose. Shown in task prompt. |
| `is_unstable_agent`| boolean | `false` | Mark agent as unstable - forces background mode for monitoring. Auto-enabled for gemini models. |
| Option | Type | Default | Description |
| ------------------- | ------------ | ------- | --------------------------------------------------------------------------------------------------- |
| `fallback_models` | string/array | - | Fallback models for runtime switching on API errors. Single string or array of model strings. |
| `description` | string | - | Human-readable description of the category's purpose. Shown in delegate_task prompt. |
| `is_unstable_agent` | boolean | `false` | Mark agent as unstable - forces background mode for monitoring. Auto-enabled for gemini models. |
## Runtime Fallback
Automatically switch to backup models when the primary model encounters retryable API errors (rate limits, overload, etc.) or provider key misconfiguration errors (for example, missing API key). This keeps conversations running without manual intervention.
```json
{
"runtime_fallback": {
"enabled": true,
"retry_on_errors": [429, 503, 529],
"max_fallback_attempts": 3,
"cooldown_seconds": 60,
"timeout_seconds": 30,
"notify_on_fallback": true
}
}
```
| Option | Default | Description |
| ----------------------- | ----------------- | --------------------------------------------------------------------------- |
| `enabled` | `true` | Enable runtime fallback |
| `retry_on_errors` | `[429, 503, 529]` | HTTP status codes that trigger fallback (rate limit, service unavailable). Also supports certain classified provider errors (for example, missing API key) that do not expose HTTP status codes. |
| `max_fallback_attempts` | `3` | Maximum fallback attempts per session (1-10) |
| `cooldown_seconds` | `60` | Cooldown in seconds before retrying a failed model |
| `timeout_seconds` | `30` | Timeout in seconds for an in-flight fallback request before forcing the next fallback model. Set to `0` to disable timeout-based fallback and provider quota retry signal detection. |
| `notify_on_fallback` | `true` | Show toast notification when switching to a fallback model |
### How It Works
1. When an API error matching `retry_on_errors` occurs (or a classified provider key error such as missing API key), the hook intercepts it
2. The next request automatically uses the next available model from `fallback_models`
3. Failed models enter a cooldown period before being retried
4. If a fallback provider hangs, timeout advances to the next fallback model
5. Toast notification (optional) informs you of the model switch
### Configuring Fallback Models
Define `fallback_models` at the agent or category level:
```json
{
"agents": {
"sisyphus": {
"model": "anthropic/claude-opus-4-5",
"fallback_models": ["openai/gpt-5.2", "google/gemini-3-pro"]
}
},
"categories": {
"ultrabrain": {
"model": "openai/gpt-5.2-codex",
"fallback_models": ["anthropic/claude-opus-4-5", "google/gemini-3-pro"]
}
}
}
```
When the primary model fails:
1. First fallback: `openai/gpt-5.2`
2. Second fallback: `google/gemini-3-pro`
3. After `max_fallback_attempts`, returns to primary model
## Model Resolution System
At runtime, Oh My OpenCode uses a 3-step resolution process to determine which model to use for each agent and category. This happens dynamically based on your configuration and available models.
@@ -973,7 +1112,7 @@ Disable specific built-in hooks via `disabled_hooks` in `~/.config/opencode/oh-m
}
```
Available hooks: `todo-continuation-enforcer`, `context-window-monitor`, `session-recovery`, `session-notification`, `comment-checker`, `grep-output-truncator`, `tool-output-truncator`, `directory-agents-injector`, `directory-readme-injector`, `empty-task-response-detector`, `think-mode`, `anthropic-context-window-limit-recovery`, `rules-injector`, `background-notification`, `auto-update-checker`, `startup-toast`, `keyword-detector`, `agent-usage-reminder`, `non-interactive-env`, `interactive-bash-session`, `compaction-context-injector`, `thinking-block-validator`, `claude-code-hooks`, `ralph-loop`, `preemptive-compaction`, `auto-slash-command`, `sisyphus-junior-notepad`, `no-sisyphus-gpt`, `start-work`
Available hooks: `todo-continuation-enforcer`, `context-window-monitor`, `session-recovery`, `session-notification`, `comment-checker`, `grep-output-truncator`, `tool-output-truncator`, `directory-agents-injector`, `directory-readme-injector`, `empty-task-response-detector`, `think-mode`, `anthropic-context-window-limit-recovery`, `rules-injector`, `background-notification`, `auto-update-checker`, `startup-toast`, `keyword-detector`, `agent-usage-reminder`, `non-interactive-env`, `interactive-bash-session`, `compaction-context-injector`, `thinking-block-validator`, `claude-code-hooks`, `ralph-loop`, `preemptive-compaction`, `auto-slash-command`, `sisyphus-junior-notepad`, `no-sisyphus-gpt`, `start-work`, `runtime-fallback`
**Note on `directory-agents-injector`**: This hook is **automatically disabled** when running on OpenCode 1.1.37+ because OpenCode now has native support for dynamically resolving AGENTS.md files from subdirectories (PR #10678). This prevents duplicate AGENTS.md injection. For older OpenCode versions, the hook remains active to provide the same functionality.

View File

@@ -352,6 +352,7 @@ Hooks intercept and modify behavior at key points in the agent lifecycle.
| **session-recovery** | Stop | Recovers from session errors - missing tool results, thinking block issues, empty messages. |
| **anthropic-context-window-limit-recovery** | Stop | Handles Claude context window limits gracefully. |
| **background-compaction** | Stop | Auto-compacts sessions hitting token limits. |
| **runtime-fallback** | Event | Automatically switches to backup models on retryable API errors (e.g., 429, 503, 529), provider key misconfiguration errors (e.g., missing API key), and auto-retry signals (when `timeout_seconds > 0`). Configurable retry logic with per-model cooldown. See [Runtime Fallback Configuration](configurations.md#runtime-fallback) for details on `timeout_seconds` behavior. |
#### Truncation & Context Management

View File

@@ -100,6 +100,7 @@ export type AgentName = BuiltinAgentName
export type AgentOverrideConfig = Partial<AgentConfig> & {
prompt_append?: string
variant?: string
fallback_models?: string | string[]
}
export type AgentOverrides = Partial<Record<OverridableAgentName, AgentOverrideConfig>>

View File

@@ -51,7 +51,7 @@ describe("createBuiltinAgents with model overrides", () => {
expect(agents.sisyphus.thinking).toBeUndefined()
})
test("Atlas uses uiSelectedModel when provided", async () => {
test("Atlas uses uiSelectedModel", async () => {
// #given
const fetchSpy = spyOn(shared, "fetchAvailableModels").mockResolvedValue(
new Set(["openai/gpt-5.2", "anthropic/claude-sonnet-4-6"])

View File

@@ -11,6 +11,8 @@ export {
RalphLoopConfigSchema,
TmuxConfigSchema,
TmuxLayoutSchema,
RuntimeFallbackConfigSchema,
FallbackModelsSchema,
} from "./schema"
export type {
@@ -29,4 +31,6 @@ export type {
TmuxLayout,
SisyphusConfig,
SisyphusTasksConfig,
RuntimeFallbackConfig,
FallbackModels,
} from "./schema"

View File

@@ -9,11 +9,13 @@ export * from "./schema/comment-checker"
export * from "./schema/commands"
export * from "./schema/dynamic-context-pruning"
export * from "./schema/experimental"
export * from "./schema/fallback-models"
export * from "./schema/git-master"
export * from "./schema/hooks"
export * from "./schema/notification"
export * from "./schema/oh-my-opencode-config"
export * from "./schema/ralph-loop"
export * from "./schema/runtime-fallback"
export * from "./schema/skills"
export * from "./schema/sisyphus"
export * from "./schema/sisyphus-agent"

View File

@@ -1,9 +1,11 @@
import { z } from "zod"
import { FallbackModelsSchema } from "./fallback-models"
import { AgentPermissionSchema } from "./internal/permission"
export const AgentOverrideConfigSchema = z.object({
/** @deprecated Use `category` instead. Model is inherited from category defaults. */
model: z.string().optional(),
fallback_models: FallbackModelsSchema.optional(),
variant: z.string().optional(),
/** Category name to inherit model and other settings from CategoryConfig */
category: z.string().optional(),

View File

@@ -1,9 +1,11 @@
import { z } from "zod"
import { FallbackModelsSchema } from "./fallback-models"
export const CategoryConfigSchema = z.object({
/** Human-readable description of the category's purpose. Shown in task prompt. */
description: z.string().optional(),
model: z.string().optional(),
fallback_models: FallbackModelsSchema.optional(),
variant: z.string().optional(),
temperature: z.number().min(0).max(2).optional(),
top_p: z.number().min(0).max(1).optional(),

View File

@@ -0,0 +1,5 @@
import { z } from "zod"
export const FallbackModelsSchema = z.union([z.string(), z.array(z.string())])
export type FallbackModels = z.infer<typeof FallbackModelsSchema>

View File

@@ -46,6 +46,7 @@ export const HookNameSchema = z.enum([
"task-resume-info",
"stop-continuation-guard",
"tasks-todowrite-disabler",
"runtime-fallback",
"write-existing-file-guard",
"anthropic-effort",
"hashline-read-enhancer",

View File

@@ -14,6 +14,7 @@ import { GitMasterConfigSchema } from "./git-master"
import { HookNameSchema } from "./hooks"
import { NotificationConfigSchema } from "./notification"
import { RalphLoopConfigSchema } from "./ralph-loop"
import { RuntimeFallbackConfigSchema } from "./runtime-fallback"
import { SkillsConfigSchema } from "./skills"
import { SisyphusConfigSchema } from "./sisyphus"
import { SisyphusAgentConfigSchema } from "./sisyphus-agent"
@@ -44,6 +45,7 @@ export const OhMyOpenCodeConfigSchema = z.object({
auto_update: z.boolean().optional(),
skills: SkillsConfigSchema.optional(),
ralph_loop: RalphLoopConfigSchema.optional(),
runtime_fallback: RuntimeFallbackConfigSchema.optional(),
background_task: BackgroundTaskConfigSchema.optional(),
notification: NotificationConfigSchema.optional(),
babysitting: BabysittingConfigSchema.optional(),

View File

@@ -0,0 +1,18 @@
import { z } from "zod"
export const RuntimeFallbackConfigSchema = z.object({
/** Enable runtime fallback (default: true) */
enabled: z.boolean().optional(),
/** HTTP status codes that trigger fallback (default: [400, 429, 503, 529]) */
retry_on_errors: z.array(z.number()).optional(),
/** Maximum fallback attempts per session (default: 3) */
max_fallback_attempts: z.number().min(1).max(20).optional(),
/** Cooldown in seconds before retrying a failed model (default: 60) */
cooldown_seconds: z.number().min(0).optional(),
/** Session-level timeout in seconds to advance fallback when provider hangs (default: 30). Set to 0 to disable auto-retry signal detection (only error-based fallback remains active). */
timeout_seconds: z.number().min(0).optional(),
/** Show toast notification when switching to fallback model (default: true) */
notify_on_fallback: z.boolean().optional(),
})
export type RuntimeFallbackConfig = z.infer<typeof RuntimeFallbackConfigSchema>

View File

@@ -16,6 +16,7 @@ import {
createInternalAgentTextPart,
} from "../../shared"
import { setSessionTools } from "../../shared/session-tools-store"
import { SessionCategoryRegistry } from "../../shared/session-category-registry"
import { ConcurrencyManager } from "./concurrency"
import type { BackgroundTaskConfig, TmuxConfig } from "../../config/schema"
import { isInsideTmux } from "../../shared/tmux"
@@ -859,6 +860,7 @@ export class BackgroundManager {
subagentSessions.delete(task.sessionID)
}
}
SessionCategoryRegistry.remove(sessionID)
}
}
@@ -1022,6 +1024,8 @@ export class BackgroundManager {
this.client.session.abort({
path: { id: task.sessionID },
}).catch(() => {})
SessionCategoryRegistry.remove(task.sessionID)
}
if (options?.skipNotification) {
@@ -1169,6 +1173,8 @@ export class BackgroundManager {
this.client.session.abort({
path: { id: task.sessionID },
}).catch(() => {})
SessionCategoryRegistry.remove(task.sessionID)
}
try {
@@ -1471,6 +1477,7 @@ Use \`background_output(task_id="${task.id}")\` to retrieve this result when rea
this.tasks.delete(taskId)
if (task.sessionID) {
subagentSessions.delete(task.sessionID)
SessionCategoryRegistry.remove(task.sessionID)
}
}
}

View File

@@ -9,6 +9,45 @@
## HOOK TIERS
### Tier 1: Session Hooks (22) — `create-session-hooks.ts`
## STRUCTURE
```
hooks/
├── atlas/ # Main orchestration (757 lines)
├── anthropic-context-window-limit-recovery/ # Auto-summarize
├── todo-continuation-enforcer.ts # Force TODO completion
├── ralph-loop/ # Self-referential dev loop
├── claude-code-hooks/ # settings.json compat layer - see AGENTS.md
├── comment-checker/ # Prevents AI slop
├── auto-slash-command/ # Detects /command patterns
├── rules-injector/ # Conditional rules
├── directory-agents-injector/ # Auto-injects AGENTS.md
├── directory-readme-injector/ # Auto-injects README.md
├── edit-error-recovery/ # Recovers from failures
├── thinking-block-validator/ # Ensures valid <thinking>
├── context-window-monitor.ts # Reminds of headroom
├── session-recovery/ # Auto-recovers from crashes
├── think-mode/ # Dynamic thinking budget
├── keyword-detector/ # ultrawork/search/analyze modes
├── background-notification/ # OS notification
├── prometheus-md-only/ # Planner read-only mode
├── agent-usage-reminder/ # Specialized agent hints
├── auto-update-checker/ # Plugin update check
├── tool-output-truncator.ts # Prevents context bloat
├── compaction-context-injector/ # Injects context on compaction
├── delegate-task-retry/ # Retries failed delegations
├── interactive-bash-session/ # Tmux session management
├── non-interactive-env/ # Non-TTY environment handling
├── start-work/ # Sisyphus work session starter
├── task-resume-info/ # Resume info for cancelled tasks
├── question-label-truncator/ # Auto-truncates question labels
├── category-skill-reminder/ # Reminds of category skills
├── empty-task-response-detector.ts # Detects empty responses
├── sisyphus-junior-notepad/ # Sisyphus Junior notepad
├── stop-continuation-guard/ # Guards stop continuation
├── subagent-question-blocker/ # Blocks subagent questions
├── runtime-fallback/ # Auto-switch models on API errors
└── index.ts # Hook aggregation + registration
```
| Hook | Event | Purpose |
|------|-------|---------|

View File

@@ -45,7 +45,7 @@ export { createCompactionTodoPreserverHook } from "./compaction-todo-preserver";
export { createUnstableAgentBabysitterHook } from "./unstable-agent-babysitter";
export { createPreemptiveCompactionHook } from "./preemptive-compaction";
export { createTasksTodowriteDisablerHook } from "./tasks-todowrite-disabler";
export { createRuntimeFallbackHook, type RuntimeFallbackHook, type RuntimeFallbackOptions } from "./runtime-fallback";
export { createWriteExistingFileGuardHook } from "./write-existing-file-guard";
export { createHashlineReadEnhancerHook } from "./hashline-read-enhancer";
export { createHashlineEditDiffEnhancerHook } from "./hashline-edit-diff-enhancer";

View File

@@ -0,0 +1,54 @@
import { getSessionAgent } from "../../features/claude-code-session-state"
export const AGENT_NAMES = [
"sisyphus",
"oracle",
"librarian",
"explore",
"prometheus",
"atlas",
"metis",
"momus",
"hephaestus",
"sisyphus-junior",
"build",
"plan",
"multimodal-looker",
]
export const agentPattern = new RegExp(
`\\b(${AGENT_NAMES
.sort((a, b) => b.length - a.length)
.map((a) => a.replace(/-/g, "\\-"))
.join("|")})\\b`,
"i",
)
export function detectAgentFromSession(sessionID: string): string | undefined {
const match = sessionID.match(agentPattern)
if (match) {
return match[1].toLowerCase()
}
return undefined
}
export function normalizeAgentName(agent: string | undefined): string | undefined {
if (!agent) return undefined
const normalized = agent.toLowerCase().trim()
if (AGENT_NAMES.includes(normalized)) {
return normalized
}
const match = normalized.match(agentPattern)
if (match) {
return match[1].toLowerCase()
}
return undefined
}
export function resolveAgentForSession(sessionID: string, eventAgent?: string): string | undefined {
return (
normalizeAgentName(eventAgent) ??
normalizeAgentName(getSessionAgent(sessionID)) ??
detectAgentFromSession(sessionID)
)
}

View File

@@ -0,0 +1,213 @@
import type { HookDeps } from "./types"
import { HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import { normalizeAgentName, resolveAgentForSession } from "./agent-resolver"
import { getSessionAgent } from "../../features/claude-code-session-state"
import { getFallbackModelsForSession } from "./fallback-models"
import { prepareFallback } from "./fallback-state"
import { SessionCategoryRegistry } from "../../shared/session-category-registry"
const SESSION_TTL_MS = 30 * 60 * 1000
export function createAutoRetryHelpers(deps: HookDeps) {
const { ctx, config, options, sessionStates, sessionLastAccess, sessionRetryInFlight, sessionAwaitingFallbackResult, sessionFallbackTimeouts, pluginConfig } = deps
const abortSessionRequest = async (sessionID: string, source: string): Promise<void> => {
try {
await ctx.client.session.abort({ path: { id: sessionID } })
log(`[${HOOK_NAME}] Aborted in-flight session request (${source})`, { sessionID })
} catch (error) {
log(`[${HOOK_NAME}] Failed to abort in-flight session request (${source})`, {
sessionID,
error: String(error),
})
}
}
const clearSessionFallbackTimeout = (sessionID: string) => {
const timer = sessionFallbackTimeouts.get(sessionID)
if (timer) {
clearTimeout(timer)
sessionFallbackTimeouts.delete(sessionID)
}
}
const scheduleSessionFallbackTimeout = (sessionID: string, resolvedAgent?: string) => {
clearSessionFallbackTimeout(sessionID)
const timeoutMs = options?.session_timeout_ms ?? config.timeout_seconds * 1000
if (timeoutMs <= 0) return
const timer = setTimeout(async () => {
sessionFallbackTimeouts.delete(sessionID)
const state = sessionStates.get(sessionID)
if (!state) return
if (sessionRetryInFlight.has(sessionID)) {
log(`[${HOOK_NAME}] Overriding in-flight retry due to session timeout`, { sessionID })
}
await abortSessionRequest(sessionID, "session.timeout")
sessionRetryInFlight.delete(sessionID)
if (state.pendingFallbackModel) {
state.pendingFallbackModel = undefined
}
const fallbackModels = getFallbackModelsForSession(sessionID, resolvedAgent, pluginConfig)
if (fallbackModels.length === 0) return
log(`[${HOOK_NAME}] Session fallback timeout reached`, {
sessionID,
timeoutSeconds: config.timeout_seconds,
currentModel: state.currentModel,
})
const result = prepareFallback(sessionID, state, fallbackModels, config)
if (result.success && result.newModel) {
await autoRetryWithFallback(sessionID, result.newModel, resolvedAgent, "session.timeout")
}
}, timeoutMs)
sessionFallbackTimeouts.set(sessionID, timer)
}
const autoRetryWithFallback = async (
sessionID: string,
newModel: string,
resolvedAgent: string | undefined,
source: string,
): Promise<void> => {
if (sessionRetryInFlight.has(sessionID)) {
log(`[${HOOK_NAME}] Retry already in flight, skipping (${source})`, { sessionID })
return
}
const modelParts = newModel.split("/")
if (modelParts.length < 2) {
log(`[${HOOK_NAME}] Invalid model format (missing provider prefix): ${newModel}`)
return
}
const fallbackModelObj = {
providerID: modelParts[0],
modelID: modelParts.slice(1).join("/"),
}
sessionRetryInFlight.add(sessionID)
try {
const messagesResp = await ctx.client.session.messages({
path: { id: sessionID },
query: { directory: ctx.directory },
})
const msgs = (messagesResp as {
data?: Array<{
info?: Record<string, unknown>
parts?: Array<{ type?: string; text?: string }>
}>
}).data
const lastUserMsg = msgs?.filter((m) => m.info?.role === "user").pop()
const lastUserPartsRaw =
lastUserMsg?.parts ??
(lastUserMsg?.info?.parts as Array<{ type?: string; text?: string }> | undefined)
if (lastUserPartsRaw && lastUserPartsRaw.length > 0) {
log(`[${HOOK_NAME}] Auto-retrying with fallback model (${source})`, {
sessionID,
model: newModel,
})
const retryParts = lastUserPartsRaw
.filter((p) => p.type === "text" && typeof p.text === "string" && p.text.length > 0)
.map((p) => ({ type: "text" as const, text: p.text! }))
if (retryParts.length > 0) {
const retryAgent = resolvedAgent ?? getSessionAgent(sessionID)
sessionAwaitingFallbackResult.add(sessionID)
scheduleSessionFallbackTimeout(sessionID, retryAgent)
await ctx.client.session.promptAsync({
path: { id: sessionID },
body: {
...(retryAgent ? { agent: retryAgent } : {}),
model: fallbackModelObj,
parts: retryParts,
},
query: { directory: ctx.directory },
})
}
} else {
log(`[${HOOK_NAME}] No user message found for auto-retry (${source})`, { sessionID })
}
} catch (retryError) {
log(`[${HOOK_NAME}] Auto-retry failed (${source})`, { sessionID, error: String(retryError) })
} finally {
const state = sessionStates.get(sessionID)
if (state?.pendingFallbackModel === newModel) {
state.pendingFallbackModel = undefined
}
sessionRetryInFlight.delete(sessionID)
}
}
const resolveAgentForSessionFromContext = async (
sessionID: string,
eventAgent?: string,
): Promise<string | undefined> => {
const resolved = resolveAgentForSession(sessionID, eventAgent)
if (resolved) return resolved
try {
const messagesResp = await ctx.client.session.messages({
path: { id: sessionID },
query: { directory: ctx.directory },
})
const msgs = (messagesResp as { data?: Array<{ info?: Record<string, unknown> }> }).data
if (!msgs || msgs.length === 0) return undefined
for (let i = msgs.length - 1; i >= 0; i--) {
const info = msgs[i]?.info
const infoAgent = typeof info?.agent === "string" ? info.agent : undefined
const normalized = normalizeAgentName(infoAgent)
if (normalized) {
return normalized
}
}
} catch {
return undefined
}
return undefined
}
const cleanupStaleSessions = () => {
const now = Date.now()
let cleanedCount = 0
for (const [sessionID, lastAccess] of sessionLastAccess.entries()) {
if (now - lastAccess > SESSION_TTL_MS) {
sessionStates.delete(sessionID)
sessionLastAccess.delete(sessionID)
sessionRetryInFlight.delete(sessionID)
sessionAwaitingFallbackResult.delete(sessionID)
clearSessionFallbackTimeout(sessionID)
SessionCategoryRegistry.remove(sessionID)
cleanedCount++
}
}
if (cleanedCount > 0) {
log(`[${HOOK_NAME}] Cleaned up ${cleanedCount} stale session states`)
}
}
return {
abortSessionRequest,
clearSessionFallbackTimeout,
scheduleSessionFallbackTimeout,
autoRetryWithFallback,
resolveAgentForSessionFromContext,
cleanupStaleSessions,
}
}
export type AutoRetryHelpers = ReturnType<typeof createAutoRetryHelpers>

View File

@@ -0,0 +1,62 @@
import type { HookDeps } from "./types"
import { HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import { createFallbackState } from "./fallback-state"
export function createChatMessageHandler(deps: HookDeps) {
const { config, sessionStates, sessionLastAccess } = deps
return async (
input: { sessionID: string; agent?: string; model?: { providerID: string; modelID: string } },
output: { message: { model?: { providerID: string; modelID: string } }; parts?: Array<{ type: string; text?: string }> }
) => {
if (!config.enabled) return
const { sessionID } = input
let state = sessionStates.get(sessionID)
if (!state) return
sessionLastAccess.set(sessionID, Date.now())
const requestedModel = input.model
? `${input.model.providerID}/${input.model.modelID}`
: undefined
if (requestedModel && requestedModel !== state.currentModel) {
if (state.pendingFallbackModel && state.pendingFallbackModel === requestedModel) {
state.pendingFallbackModel = undefined
return
}
log(`[${HOOK_NAME}] Detected manual model change, resetting fallback state`, {
sessionID,
from: state.currentModel,
to: requestedModel,
})
state = createFallbackState(requestedModel)
sessionStates.set(sessionID, state)
return
}
if (state.currentModel === state.originalModel) return
const activeModel = state.currentModel
log(`[${HOOK_NAME}] Applying fallback model override`, {
sessionID,
from: input.model,
to: activeModel,
})
if (output.message && activeModel) {
const parts = activeModel.split("/")
if (parts.length >= 2) {
output.message.model = {
providerID: parts[0],
modelID: parts.slice(1).join("/"),
}
}
}
}
}

View File

@@ -0,0 +1,44 @@
/**
* Runtime Fallback Hook - Constants
*
* Default values and configuration constants for the runtime fallback feature.
*/
import type { RuntimeFallbackConfig } from "../../config"
/**
* Default configuration values for runtime fallback
*/
export const DEFAULT_CONFIG: Required<RuntimeFallbackConfig> = {
enabled: true,
retry_on_errors: [400, 429, 503, 529],
max_fallback_attempts: 3,
cooldown_seconds: 60,
timeout_seconds: 30,
notify_on_fallback: true,
}
/**
* Error patterns that indicate rate limiting or temporary failures
* These are checked in addition to HTTP status codes
*/
export const RETRYABLE_ERROR_PATTERNS = [
/rate.?limit/i,
/too.?many.?requests/i,
/quota.?exceeded/i,
/usage\s+limit\s+has\s+been\s+reached/i,
/service.?unavailable/i,
/overloaded/i,
/temporarily.?unavailable/i,
/try.?again/i,
/credit.*balance.*too.*low/i,
/insufficient.?(?:credits?|funds?|balance)/i,
/(?:^|\s)429(?:\s|$)/,
/(?:^|\s)503(?:\s|$)/,
/(?:^|\s)529(?:\s|$)/,
]
/**
* Hook name for identification and logging
*/
export const HOOK_NAME = "runtime-fallback"

View File

@@ -0,0 +1,169 @@
import { DEFAULT_CONFIG, RETRYABLE_ERROR_PATTERNS } from "./constants"
export function getErrorMessage(error: unknown): string {
if (!error) return ""
if (typeof error === "string") return error.toLowerCase()
const errorObj = error as Record<string, unknown>
const paths = [
errorObj.data,
errorObj.error,
errorObj,
(errorObj.data as Record<string, unknown>)?.error,
]
for (const obj of paths) {
if (obj && typeof obj === "object") {
const msg = (obj as Record<string, unknown>).message
if (typeof msg === "string" && msg.length > 0) {
return msg.toLowerCase()
}
}
}
try {
return JSON.stringify(error).toLowerCase()
} catch {
return ""
}
}
export function extractStatusCode(error: unknown, retryOnErrors?: number[]): number | undefined {
if (!error) return undefined
const errorObj = error as Record<string, unknown>
const statusCode = errorObj.statusCode ?? errorObj.status ?? (errorObj.data as Record<string, unknown>)?.statusCode
if (typeof statusCode === "number") {
return statusCode
}
const codes = retryOnErrors ?? DEFAULT_CONFIG.retry_on_errors
const pattern = new RegExp(`\\b(${codes.join("|")})\\b`)
const message = getErrorMessage(error)
const statusMatch = message.match(pattern)
if (statusMatch) {
return parseInt(statusMatch[1], 10)
}
return undefined
}
export function extractErrorName(error: unknown): string | undefined {
if (!error || typeof error !== "object") return undefined
const errorObj = error as Record<string, unknown>
const directName = errorObj.name
if (typeof directName === "string" && directName.length > 0) {
return directName
}
const nestedError = errorObj.error as Record<string, unknown> | undefined
const nestedName = nestedError?.name
if (typeof nestedName === "string" && nestedName.length > 0) {
return nestedName
}
const dataError = (errorObj.data as Record<string, unknown> | undefined)?.error as Record<string, unknown> | undefined
const dataErrorName = dataError?.name
if (typeof dataErrorName === "string" && dataErrorName.length > 0) {
return dataErrorName
}
return undefined
}
export function classifyErrorType(error: unknown): string | undefined {
const message = getErrorMessage(error)
const errorName = extractErrorName(error)?.toLowerCase()
if (
errorName?.includes("loadapi") ||
(/api.?key.?is.?missing/i.test(message) && /environment variable/i.test(message))
) {
return "missing_api_key"
}
if (/api.?key/i.test(message) && /must be a string/i.test(message)) {
return "invalid_api_key"
}
if (errorName?.includes("unknownerror") && /model\s+not\s+found/i.test(message)) {
return "model_not_found"
}
return undefined
}
export interface AutoRetrySignal {
signal: string
}
export const AUTO_RETRY_PATTERNS: Array<(combined: string) => boolean> = [
(combined) => /retrying\s+in/i.test(combined),
(combined) =>
/(?:too\s+many\s+requests|quota\s*exceeded|usage\s+limit|rate\s+limit|limit\s+reached)/i.test(combined),
]
export function extractAutoRetrySignal(info: Record<string, unknown> | undefined): AutoRetrySignal | undefined {
if (!info) return undefined
const candidates: string[] = []
const directStatus = info.status
if (typeof directStatus === "string") candidates.push(directStatus)
const summary = info.summary
if (typeof summary === "string") candidates.push(summary)
const message = info.message
if (typeof message === "string") candidates.push(message)
const details = info.details
if (typeof details === "string") candidates.push(details)
const combined = candidates.join("\n")
if (!combined) return undefined
const isAutoRetry = AUTO_RETRY_PATTERNS.every((test) => test(combined))
if (isAutoRetry) {
return { signal: combined }
}
return undefined
}
export function containsErrorContent(
parts: Array<{ type?: string; text?: string }> | undefined
): { hasError: boolean; errorMessage?: string } {
if (!parts || parts.length === 0) return { hasError: false }
const errorParts = parts.filter((p) => p.type === "error")
if (errorParts.length > 0) {
const errorMessages = errorParts.map((p) => p.text).filter((text): text is string => typeof text === "string")
const errorMessage = errorMessages.length > 0 ? errorMessages.join("\n") : undefined
return { hasError: true, errorMessage }
}
return { hasError: false }
}
export function isRetryableError(error: unknown, retryOnErrors: number[]): boolean {
const statusCode = extractStatusCode(error, retryOnErrors)
const message = getErrorMessage(error)
const errorType = classifyErrorType(error)
if (errorType === "missing_api_key") {
return true
}
if (errorType === "model_not_found") {
return true
}
if (statusCode && retryOnErrors.includes(statusCode)) {
return true
}
return RETRYABLE_ERROR_PATTERNS.some((pattern) => pattern.test(message))
}

View File

@@ -0,0 +1,187 @@
import type { HookDeps } from "./types"
import type { AutoRetryHelpers } from "./auto-retry"
import { HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import { extractStatusCode, extractErrorName, classifyErrorType, isRetryableError } from "./error-classifier"
import { createFallbackState, prepareFallback } from "./fallback-state"
import { getFallbackModelsForSession } from "./fallback-models"
import { SessionCategoryRegistry } from "../../shared/session-category-registry"
export function createEventHandler(deps: HookDeps, helpers: AutoRetryHelpers) {
const { config, pluginConfig, sessionStates, sessionLastAccess, sessionRetryInFlight, sessionAwaitingFallbackResult, sessionFallbackTimeouts } = deps
const handleSessionCreated = (props: Record<string, unknown> | undefined) => {
const sessionInfo = props?.info as { id?: string; model?: string } | undefined
const sessionID = sessionInfo?.id
const model = sessionInfo?.model
if (sessionID && model) {
log(`[${HOOK_NAME}] Session created with model`, { sessionID, model })
sessionStates.set(sessionID, createFallbackState(model))
sessionLastAccess.set(sessionID, Date.now())
}
}
const handleSessionDeleted = (props: Record<string, unknown> | undefined) => {
const sessionInfo = props?.info as { id?: string } | undefined
const sessionID = sessionInfo?.id
if (sessionID) {
log(`[${HOOK_NAME}] Cleaning up session state`, { sessionID })
sessionStates.delete(sessionID)
sessionLastAccess.delete(sessionID)
sessionRetryInFlight.delete(sessionID)
sessionAwaitingFallbackResult.delete(sessionID)
helpers.clearSessionFallbackTimeout(sessionID)
SessionCategoryRegistry.remove(sessionID)
}
}
const handleSessionStop = async (props: Record<string, unknown> | undefined) => {
const sessionID = props?.sessionID as string | undefined
if (!sessionID) return
helpers.clearSessionFallbackTimeout(sessionID)
if (sessionRetryInFlight.has(sessionID)) {
await helpers.abortSessionRequest(sessionID, "session.stop")
}
sessionRetryInFlight.delete(sessionID)
sessionAwaitingFallbackResult.delete(sessionID)
const state = sessionStates.get(sessionID)
if (state?.pendingFallbackModel) {
state.pendingFallbackModel = undefined
}
log(`[${HOOK_NAME}] Cleared fallback retry state on session.stop`, { sessionID })
}
const handleSessionIdle = (props: Record<string, unknown> | undefined) => {
const sessionID = props?.sessionID as string | undefined
if (!sessionID) return
if (sessionAwaitingFallbackResult.has(sessionID)) {
log(`[${HOOK_NAME}] session.idle while awaiting fallback result; keeping timeout armed`, { sessionID })
return
}
const hadTimeout = sessionFallbackTimeouts.has(sessionID)
helpers.clearSessionFallbackTimeout(sessionID)
sessionRetryInFlight.delete(sessionID)
const state = sessionStates.get(sessionID)
if (state?.pendingFallbackModel) {
state.pendingFallbackModel = undefined
}
if (hadTimeout) {
log(`[${HOOK_NAME}] Cleared fallback timeout after session completion`, { sessionID })
}
}
const handleSessionError = async (props: Record<string, unknown> | undefined) => {
const sessionID = props?.sessionID as string | undefined
const error = props?.error
const agent = props?.agent as string | undefined
if (!sessionID) {
log(`[${HOOK_NAME}] session.error without sessionID, skipping`)
return
}
const resolvedAgent = await helpers.resolveAgentForSessionFromContext(sessionID, agent)
sessionAwaitingFallbackResult.delete(sessionID)
helpers.clearSessionFallbackTimeout(sessionID)
log(`[${HOOK_NAME}] session.error received`, {
sessionID,
agent,
resolvedAgent,
statusCode: extractStatusCode(error, config.retry_on_errors),
errorName: extractErrorName(error),
errorType: classifyErrorType(error),
})
if (!isRetryableError(error, config.retry_on_errors)) {
log(`[${HOOK_NAME}] Error not retryable, skipping fallback`, {
sessionID,
retryable: false,
statusCode: extractStatusCode(error, config.retry_on_errors),
errorName: extractErrorName(error),
errorType: classifyErrorType(error),
})
return
}
let state = sessionStates.get(sessionID)
const fallbackModels = getFallbackModelsForSession(sessionID, resolvedAgent, pluginConfig)
if (fallbackModels.length === 0) {
log(`[${HOOK_NAME}] No fallback models configured`, { sessionID, agent })
return
}
if (!state) {
const currentModel = props?.model as string | undefined
if (currentModel) {
state = createFallbackState(currentModel)
sessionStates.set(sessionID, state)
sessionLastAccess.set(sessionID, Date.now())
} else {
const detectedAgent = resolvedAgent
const agentConfig = detectedAgent
? pluginConfig?.agents?.[detectedAgent as keyof typeof pluginConfig.agents]
: undefined
const agentModel = agentConfig?.model as string | undefined
if (agentModel) {
log(`[${HOOK_NAME}] Derived model from agent config`, { sessionID, agent: detectedAgent, model: agentModel })
state = createFallbackState(agentModel)
sessionStates.set(sessionID, state)
sessionLastAccess.set(sessionID, Date.now())
} else {
log(`[${HOOK_NAME}] No model info available, cannot fallback`, { sessionID })
return
}
}
} else {
sessionLastAccess.set(sessionID, Date.now())
}
const result = prepareFallback(sessionID, state, fallbackModels, config)
if (result.success && config.notify_on_fallback) {
await deps.ctx.client.tui
.showToast({
body: {
title: "Model Fallback",
message: `Switching to ${result.newModel?.split("/").pop() || result.newModel} for next request`,
variant: "warning",
duration: 5000,
},
})
.catch(() => {})
}
if (result.success && result.newModel) {
await helpers.autoRetryWithFallback(sessionID, result.newModel, resolvedAgent, "session.error")
}
if (!result.success) {
log(`[${HOOK_NAME}] Fallback preparation failed`, { sessionID, error: result.error })
}
}
return async ({ event }: { event: { type: string; properties?: unknown } }) => {
if (!config.enabled) return
const props = event.properties as Record<string, unknown> | undefined
if (event.type === "session.created") { handleSessionCreated(props); return }
if (event.type === "session.deleted") { handleSessionDeleted(props); return }
if (event.type === "session.stop") { await handleSessionStop(props); return }
if (event.type === "session.idle") { handleSessionIdle(props); return }
if (event.type === "session.error") { await handleSessionError(props); return }
}
}

View File

@@ -0,0 +1,69 @@
import type { OhMyOpenCodeConfig } from "../../config"
import { AGENT_NAMES, agentPattern } from "./agent-resolver"
import { HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import { SessionCategoryRegistry } from "../../shared/session-category-registry"
import { normalizeFallbackModels } from "../../shared/model-resolver"
export function getFallbackModelsForSession(
sessionID: string,
agent: string | undefined,
pluginConfig: OhMyOpenCodeConfig | undefined
): string[] {
if (!pluginConfig) return []
const sessionCategory = SessionCategoryRegistry.get(sessionID)
if (sessionCategory && pluginConfig.categories?.[sessionCategory]) {
const categoryConfig = pluginConfig.categories[sessionCategory]
if (categoryConfig?.fallback_models) {
return normalizeFallbackModels(categoryConfig.fallback_models) ?? []
}
}
const tryGetFallbackFromAgent = (agentName: string): string[] | undefined => {
const agentConfig = pluginConfig.agents?.[agentName as keyof typeof pluginConfig.agents]
if (!agentConfig) return undefined
if (agentConfig?.fallback_models) {
return normalizeFallbackModels(agentConfig.fallback_models)
}
const agentCategory = agentConfig?.category
if (agentCategory && pluginConfig.categories?.[agentCategory]) {
const categoryConfig = pluginConfig.categories[agentCategory]
if (categoryConfig?.fallback_models) {
return normalizeFallbackModels(categoryConfig.fallback_models)
}
}
return undefined
}
if (agent) {
const result = tryGetFallbackFromAgent(agent)
if (result) return result
}
const sessionAgentMatch = sessionID.match(agentPattern)
if (sessionAgentMatch) {
const detectedAgent = sessionAgentMatch[1].toLowerCase()
const result = tryGetFallbackFromAgent(detectedAgent)
if (result) return result
}
const sisyphusFallback = tryGetFallbackFromAgent("sisyphus")
if (sisyphusFallback) {
log(`[${HOOK_NAME}] Using sisyphus fallback models (no agent detected)`, { sessionID })
return sisyphusFallback
}
for (const agentName of AGENT_NAMES) {
const result = tryGetFallbackFromAgent(agentName)
if (result) {
log(`[${HOOK_NAME}] Using ${agentName} fallback models (no agent detected)`, { sessionID })
return result
}
}
return []
}

View File

@@ -0,0 +1,74 @@
import type { FallbackState, FallbackResult } from "./types"
import { HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import type { RuntimeFallbackConfig } from "../../config"
export function createFallbackState(originalModel: string): FallbackState {
return {
originalModel,
currentModel: originalModel,
fallbackIndex: -1,
failedModels: new Map<string, number>(),
attemptCount: 0,
pendingFallbackModel: undefined,
}
}
export function isModelInCooldown(model: string, state: FallbackState, cooldownSeconds: number): boolean {
const failedAt = state.failedModels.get(model)
if (failedAt === undefined) return false
const cooldownMs = cooldownSeconds * 1000
return Date.now() - failedAt < cooldownMs
}
export function findNextAvailableFallback(
state: FallbackState,
fallbackModels: string[],
cooldownSeconds: number
): string | undefined {
for (let i = state.fallbackIndex + 1; i < fallbackModels.length; i++) {
const candidate = fallbackModels[i]
if (!isModelInCooldown(candidate, state, cooldownSeconds)) {
return candidate
}
log(`[${HOOK_NAME}] Skipping fallback model in cooldown`, { model: candidate, index: i })
}
return undefined
}
export function prepareFallback(
sessionID: string,
state: FallbackState,
fallbackModels: string[],
config: Required<RuntimeFallbackConfig>
): FallbackResult {
if (state.attemptCount >= config.max_fallback_attempts) {
log(`[${HOOK_NAME}] Max fallback attempts reached`, { sessionID, attempts: state.attemptCount })
return { success: false, error: "Max fallback attempts reached", maxAttemptsReached: true }
}
const nextModel = findNextAvailableFallback(state, fallbackModels, config.cooldown_seconds)
if (!nextModel) {
log(`[${HOOK_NAME}] No available fallback models`, { sessionID })
return { success: false, error: "No available fallback models (all in cooldown or exhausted)" }
}
log(`[${HOOK_NAME}] Preparing fallback`, {
sessionID,
from: state.currentModel,
to: nextModel,
attempt: state.attemptCount + 1,
})
const failedModel = state.currentModel
const now = Date.now()
state.fallbackIndex = fallbackModels.indexOf(nextModel)
state.failedModels.set(failedModel, now)
state.attemptCount++
state.currentModel = nextModel
state.pendingFallbackModel = nextModel
return { success: true, newModel: nextModel }
}

View File

@@ -0,0 +1,67 @@
import type { PluginInput } from "@opencode-ai/plugin"
import type { HookDeps, RuntimeFallbackHook, RuntimeFallbackOptions } from "./types"
import { DEFAULT_CONFIG, HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import { loadPluginConfig } from "../../plugin-config"
import { createAutoRetryHelpers } from "./auto-retry"
import { createEventHandler } from "./event-handler"
import { createMessageUpdateHandler } from "./message-update-handler"
import { createChatMessageHandler } from "./chat-message-handler"
export function createRuntimeFallbackHook(
ctx: PluginInput,
options?: RuntimeFallbackOptions
): RuntimeFallbackHook {
const config = {
enabled: options?.config?.enabled ?? DEFAULT_CONFIG.enabled,
retry_on_errors: options?.config?.retry_on_errors ?? DEFAULT_CONFIG.retry_on_errors,
max_fallback_attempts: options?.config?.max_fallback_attempts ?? DEFAULT_CONFIG.max_fallback_attempts,
cooldown_seconds: options?.config?.cooldown_seconds ?? DEFAULT_CONFIG.cooldown_seconds,
timeout_seconds: options?.config?.timeout_seconds ?? DEFAULT_CONFIG.timeout_seconds,
notify_on_fallback: options?.config?.notify_on_fallback ?? DEFAULT_CONFIG.notify_on_fallback,
}
let pluginConfig = options?.pluginConfig
if (!pluginConfig) {
try {
pluginConfig = loadPluginConfig(ctx.directory, ctx)
} catch {
log(`[${HOOK_NAME}] Plugin config not available`)
}
}
const deps: HookDeps = {
ctx,
config,
options,
pluginConfig,
sessionStates: new Map(),
sessionLastAccess: new Map(),
sessionRetryInFlight: new Set(),
sessionAwaitingFallbackResult: new Set(),
sessionFallbackTimeouts: new Map(),
}
const helpers = createAutoRetryHelpers(deps)
const baseEventHandler = createEventHandler(deps, helpers)
const messageUpdateHandler = createMessageUpdateHandler(deps, helpers)
const chatMessageHandler = createChatMessageHandler(deps)
const cleanupInterval = setInterval(helpers.cleanupStaleSessions, 5 * 60 * 1000)
cleanupInterval.unref()
const eventHandler = async ({ event }: { event: { type: string; properties?: unknown } }) => {
if (event.type === "message.updated") {
if (!config.enabled) return
const props = event.properties as Record<string, unknown> | undefined
await messageUpdateHandler(props)
return
}
await baseEventHandler({ event })
}
return {
event: eventHandler,
"chat.message": chatMessageHandler,
} as RuntimeFallbackHook
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,2 @@
export { createRuntimeFallbackHook } from "./hook"
export type { RuntimeFallbackHook, RuntimeFallbackOptions } from "./types"

View File

@@ -0,0 +1,216 @@
import type { HookDeps } from "./types"
import type { AutoRetryHelpers } from "./auto-retry"
import { HOOK_NAME } from "./constants"
import { log } from "../../shared/logger"
import { extractStatusCode, extractErrorName, classifyErrorType, isRetryableError, extractAutoRetrySignal, containsErrorContent } from "./error-classifier"
import { createFallbackState, prepareFallback } from "./fallback-state"
import { getFallbackModelsForSession } from "./fallback-models"
export function hasVisibleAssistantResponse(extractAutoRetrySignalFn: typeof extractAutoRetrySignal) {
return async (
ctx: HookDeps["ctx"],
sessionID: string,
_info: Record<string, unknown> | undefined,
): Promise<boolean> => {
try {
const messagesResp = await ctx.client.session.messages({
path: { id: sessionID },
query: { directory: ctx.directory },
})
const msgs = (messagesResp as {
data?: Array<{
info?: Record<string, unknown>
parts?: Array<{ type?: string; text?: string }>
}>
}).data
if (!msgs || msgs.length === 0) return false
const lastAssistant = [...msgs].reverse().find((m) => m.info?.role === "assistant")
if (!lastAssistant) return false
if (lastAssistant.info?.error) return false
const parts = lastAssistant.parts ??
(lastAssistant.info?.parts as Array<{ type?: string; text?: string }> | undefined)
const textFromParts = (parts ?? [])
.filter((p) => p.type === "text" && typeof p.text === "string")
.map((p) => p.text!.trim())
.filter((text) => text.length > 0)
.join("\n")
if (!textFromParts) return false
if (extractAutoRetrySignalFn({ message: textFromParts })) return false
return true
} catch {
return false
}
}
}
export function createMessageUpdateHandler(deps: HookDeps, helpers: AutoRetryHelpers) {
const { ctx, config, pluginConfig, sessionStates, sessionLastAccess, sessionRetryInFlight, sessionAwaitingFallbackResult } = deps
const checkVisibleResponse = hasVisibleAssistantResponse(extractAutoRetrySignal)
return async (props: Record<string, unknown> | undefined) => {
const info = props?.info as Record<string, unknown> | undefined
const sessionID = info?.sessionID as string | undefined
const retrySignalResult = extractAutoRetrySignal(info)
const retrySignal = retrySignalResult?.signal
const timeoutEnabled = config.timeout_seconds > 0
const parts = props?.parts as Array<{ type?: string; text?: string }> | undefined
const errorContentResult = containsErrorContent(parts)
const error = info?.error ??
(retrySignal && timeoutEnabled ? { name: "ProviderRateLimitError", message: retrySignal } : undefined) ??
(errorContentResult.hasError ? { name: "MessageContentError", message: errorContentResult.errorMessage || "Message contains error content" } : undefined)
const role = info?.role as string | undefined
const model = info?.model as string | undefined
if (sessionID && role === "assistant" && !error) {
if (!sessionAwaitingFallbackResult.has(sessionID)) {
return
}
const hasVisible = await checkVisibleResponse(ctx, sessionID, info)
if (!hasVisible) {
log(`[${HOOK_NAME}] Assistant update observed without visible final response; keeping fallback timeout`, {
sessionID,
model,
})
return
}
sessionAwaitingFallbackResult.delete(sessionID)
helpers.clearSessionFallbackTimeout(sessionID)
const state = sessionStates.get(sessionID)
if (state?.pendingFallbackModel) {
state.pendingFallbackModel = undefined
}
log(`[${HOOK_NAME}] Assistant response observed; cleared fallback timeout`, { sessionID, model })
return
}
if (sessionID && role === "assistant" && error) {
sessionAwaitingFallbackResult.delete(sessionID)
if (sessionRetryInFlight.has(sessionID) && !retrySignal) {
log(`[${HOOK_NAME}] message.updated fallback skipped (retry in flight)`, { sessionID })
return
}
if (retrySignal && sessionRetryInFlight.has(sessionID) && timeoutEnabled) {
log(`[${HOOK_NAME}] Overriding in-flight retry due to provider auto-retry signal`, {
sessionID,
model,
})
await helpers.abortSessionRequest(sessionID, "message.updated.retry-signal")
sessionRetryInFlight.delete(sessionID)
}
if (retrySignal && timeoutEnabled) {
log(`[${HOOK_NAME}] Detected provider auto-retry signal`, { sessionID, model })
}
if (!retrySignal) {
helpers.clearSessionFallbackTimeout(sessionID)
}
log(`[${HOOK_NAME}] message.updated with assistant error`, {
sessionID,
model,
statusCode: extractStatusCode(error, config.retry_on_errors),
errorName: extractErrorName(error),
errorType: classifyErrorType(error),
})
if (!isRetryableError(error, config.retry_on_errors)) {
log(`[${HOOK_NAME}] message.updated error not retryable, skipping fallback`, {
sessionID,
statusCode: extractStatusCode(error, config.retry_on_errors),
errorName: extractErrorName(error),
errorType: classifyErrorType(error),
})
return
}
let state = sessionStates.get(sessionID)
const agent = info?.agent as string | undefined
const resolvedAgent = await helpers.resolveAgentForSessionFromContext(sessionID, agent)
const fallbackModels = getFallbackModelsForSession(sessionID, resolvedAgent, pluginConfig)
if (fallbackModels.length === 0) {
return
}
if (!state) {
let initialModel = model
if (!initialModel) {
const detectedAgent = resolvedAgent
const agentConfig = detectedAgent
? pluginConfig?.agents?.[detectedAgent as keyof typeof pluginConfig.agents]
: undefined
const agentModel = agentConfig?.model as string | undefined
if (agentModel) {
log(`[${HOOK_NAME}] Derived model from agent config for message.updated`, {
sessionID,
agent: detectedAgent,
model: agentModel,
})
initialModel = agentModel
}
}
if (!initialModel) {
log(`[${HOOK_NAME}] message.updated missing model info, cannot fallback`, {
sessionID,
errorName: extractErrorName(error),
errorType: classifyErrorType(error),
})
return
}
state = createFallbackState(initialModel)
sessionStates.set(sessionID, state)
sessionLastAccess.set(sessionID, Date.now())
} else {
sessionLastAccess.set(sessionID, Date.now())
if (state.pendingFallbackModel) {
if (retrySignal && timeoutEnabled) {
log(`[${HOOK_NAME}] Clearing pending fallback due to provider auto-retry signal`, {
sessionID,
pendingFallbackModel: state.pendingFallbackModel,
})
state.pendingFallbackModel = undefined
} else {
log(`[${HOOK_NAME}] message.updated fallback skipped (pending fallback in progress)`, {
sessionID,
pendingFallbackModel: state.pendingFallbackModel,
})
return
}
}
}
const result = prepareFallback(sessionID, state, fallbackModels, config)
if (result.success && config.notify_on_fallback) {
await deps.ctx.client.tui
.showToast({
body: {
title: "Model Fallback",
message: `Switching to ${result.newModel?.split("/").pop() || result.newModel} for next request`,
variant: "warning",
duration: 5000,
},
})
.catch(() => {})
}
if (result.success && result.newModel) {
await helpers.autoRetryWithFallback(sessionID, result.newModel, resolvedAgent, "message.updated")
}
}
}
}

View File

@@ -0,0 +1,41 @@
import type { PluginInput } from "@opencode-ai/plugin"
import type { RuntimeFallbackConfig, OhMyOpenCodeConfig } from "../../config"
export interface FallbackState {
originalModel: string
currentModel: string
fallbackIndex: number
failedModels: Map<string, number>
attemptCount: number
pendingFallbackModel?: string
}
export interface FallbackResult {
success: boolean
newModel?: string
error?: string
maxAttemptsReached?: boolean
}
export interface RuntimeFallbackOptions {
config?: RuntimeFallbackConfig
pluginConfig?: OhMyOpenCodeConfig
session_timeout_ms?: number
}
export interface RuntimeFallbackHook {
event: (input: { event: { type: string; properties?: unknown } }) => Promise<void>
"chat.message"?: (input: { sessionID: string; agent?: string; model?: { providerID: string; modelID: string } }, output: { message: { model?: { providerID: string; modelID: string } }; parts?: Array<{ type: string; text?: string }> }) => Promise<void>
}
export interface HookDeps {
ctx: PluginInput
config: Required<RuntimeFallbackConfig>
options: RuntimeFallbackOptions | undefined
pluginConfig: OhMyOpenCodeConfig | undefined
sessionStates: Map<string, FallbackState>
sessionLastAccess: Map<string, number>
sessionRetryInFlight: Set<string>
sessionAwaitingFallbackResult: Set<string>
sessionFallbackTimeouts: Map<string, ReturnType<typeof setTimeout>>
}

View File

@@ -54,6 +54,7 @@ export function createChatMessageHandler(args: {
}
await hooks.stopContinuationGuard?.["chat.message"]?.(input)
await hooks.runtimeFallback?.["chat.message"]?.(input, output)
await hooks.keywordDetector?.["chat.message"]?.(input, output)
await hooks.claudeCodeHooks?.["chat.message"]?.(input, output)
await hooks.autoSlashCommand?.["chat.message"]?.(input, output)

View File

@@ -20,16 +20,20 @@ type FirstMessageVariantGate = {
clear: (sessionID: string) => void
}
type EventInput = Parameters<
NonNullable<NonNullable<CreatedHooks["writeExistingFileGuard"]>["event"]>
>[0]
export function createEventHandler(args: {
ctx: PluginContext
pluginConfig: OhMyOpenCodeConfig
firstMessageVariantGate: FirstMessageVariantGate
managers: Managers
hooks: CreatedHooks
}): (input: { event: { type: string; properties?: Record<string, unknown> } }) => Promise<void> {
}): (input: EventInput) => Promise<void> {
const { ctx, firstMessageVariantGate, managers, hooks } = args
const dispatchToHooks = async (input: { event: { type: string; properties?: Record<string, unknown> } }): Promise<void> => {
const dispatchToHooks = async (input: EventInput): Promise<void> => {
await Promise.resolve(hooks.autoUpdateChecker?.event?.(input))
await Promise.resolve(hooks.claudeCodeHooks?.event?.(input))
await Promise.resolve(hooks.backgroundNotificationHook?.event?.(input))
@@ -42,9 +46,10 @@ export function createEventHandler(args: {
await Promise.resolve(hooks.rulesInjector?.event?.(input))
await Promise.resolve(hooks.thinkMode?.event?.(input))
await Promise.resolve(hooks.anthropicContextWindowLimitRecovery?.event?.(input))
await Promise.resolve(hooks.runtimeFallback?.event?.(input))
await Promise.resolve(hooks.agentUsageReminder?.event?.(input))
await Promise.resolve(hooks.categorySkillReminder?.event?.(input))
await Promise.resolve(hooks.interactiveBashSession?.event?.(input))
await Promise.resolve(hooks.interactiveBashSession?.event?.(input as EventInput))
await Promise.resolve(hooks.ralphLoop?.event?.(input))
await Promise.resolve(hooks.stopContinuationGuard?.event?.(input))
await Promise.resolve(hooks.compactionTodoPreserver?.event?.(input))
@@ -87,7 +92,7 @@ export function createEventHandler(args: {
return
}
recentSyntheticIdles.set(sessionID, Date.now())
await dispatchToHooks(syntheticIdle)
await dispatchToHooks(syntheticIdle as EventInput)
}
const { event } = input

View File

@@ -24,6 +24,7 @@ import {
createNoHephaestusNonGptHook,
createQuestionLabelTruncatorHook,
createPreemptiveCompactionHook,
createRuntimeFallbackHook,
} from "../../hooks"
import { createAnthropicEffortHook } from "../../hooks/anthropic-effort"
import {
@@ -57,6 +58,7 @@ export type SessionHooks = {
questionLabelTruncator: ReturnType<typeof createQuestionLabelTruncatorHook>
taskResumeInfo: ReturnType<typeof createTaskResumeInfoHook>
anthropicEffort: ReturnType<typeof createAnthropicEffortHook> | null
runtimeFallback: ReturnType<typeof createRuntimeFallbackHook> | null
}
export function createSessionHooks(args: {
@@ -175,6 +177,13 @@ export function createSessionHooks(args: {
? safeHook("anthropic-effort", () => createAnthropicEffortHook())
: null
const runtimeFallback = isHookEnabled("runtime-fallback")
? safeHook("runtime-fallback", () =>
createRuntimeFallbackHook(ctx, {
config: pluginConfig.runtime_fallback,
pluginConfig,
}))
: null
return {
contextWindowMonitor,
preemptiveCompaction,
@@ -198,5 +207,6 @@ export function createSessionHooks(args: {
questionLabelTruncator,
taskResumeInfo,
anthropicEffort,
runtimeFallback,
}
}

View File

@@ -34,6 +34,7 @@ export * from "./system-directive"
export * from "./agent-tool-restrictions"
export * from "./model-requirements"
export * from "./model-resolver"
export { normalizeFallbackModels } from "./model-resolver"
export { resolveModelPipeline } from "./model-resolution-pipeline"
export type {
ModelResolutionRequest,
@@ -58,3 +59,4 @@ export * from "./normalize-sdk-response"
export * from "./session-directory-resolver"
export * from "./prompt-tools"
export * from "./internal-initiator-marker"
export { SessionCategoryRegistry } from "./session-category-registry"

View File

@@ -7,6 +7,7 @@ export type ModelResolutionRequest = {
intent?: {
uiSelectedModel?: string
userModel?: string
userFallbackModels?: string[]
categoryDefaultModel?: string
}
constraints: {
@@ -97,6 +98,42 @@ export function resolveModelPipeline(
})
}
//#when - user configured fallback_models, try them before hardcoded fallback chain
const userFallbackModels = intent?.userFallbackModels
if (userFallbackModels && userFallbackModels.length > 0) {
if (availableModels.size === 0) {
const connectedProviders = constraints.connectedProviders ?? connectedProvidersCache.readConnectedProvidersCache()
const connectedSet = connectedProviders ? new Set(connectedProviders) : null
if (connectedSet !== null) {
for (const model of userFallbackModels) {
attempted.push(model)
const parts = model.split("/")
if (parts.length >= 2) {
const provider = parts[0]
if (connectedSet.has(provider)) {
log("Model resolved via user fallback_models (connected provider)", { model })
return { model, provenance: "provider-fallback", attempted }
}
}
}
log("No connected provider found in user fallback_models, falling through to hardcoded chain")
}
} else {
for (const model of userFallbackModels) {
attempted.push(model)
const parts = model.split("/")
const providerHint = parts.length >= 2 ? [parts[0]] : undefined
const match = fuzzyMatchModel(model, availableModels, providerHint)
if (match) {
log("Model resolved via user fallback_models (availability confirmed)", { model: model, match })
return { model: match, provenance: "provider-fallback", attempted }
}
}
log("No available model found in user fallback_models, falling through to hardcoded chain")
}
}
if (fallbackChain && fallbackChain.length > 0) {
if (availableModels.size === 0) {
const connectedProviders = constraints.connectedProviders ?? connectedProvidersCache.readConnectedProvidersCache()

View File

@@ -22,6 +22,7 @@ export type ModelResolutionResult = {
export type ExtendedModelResolutionInput = {
uiSelectedModel?: string
userModel?: string
userFallbackModels?: string[]
categoryDefaultModel?: string
fallbackChain?: FallbackEntry[]
availableModels: Set<string>
@@ -44,9 +45,9 @@ export function resolveModel(input: ModelResolutionInput): string | undefined {
export function resolveModelWithFallback(
input: ExtendedModelResolutionInput,
): ModelResolutionResult | undefined {
const { uiSelectedModel, userModel, categoryDefaultModel, fallbackChain, availableModels, systemDefaultModel } = input
const { uiSelectedModel, userModel, userFallbackModels, categoryDefaultModel, fallbackChain, availableModels, systemDefaultModel } = input
const resolved = resolveModelPipeline({
intent: { uiSelectedModel, userModel, categoryDefaultModel },
intent: { uiSelectedModel, userModel, userFallbackModels, categoryDefaultModel },
constraints: { availableModels },
policy: { fallbackChain, systemDefaultModel },
})
@@ -61,3 +62,13 @@ export function resolveModelWithFallback(
variant: resolved.variant,
}
}
/**
* Normalizes fallback_models config (which can be string or string[]) to string[]
* Centralized helper to avoid duplicated normalization logic
*/
export function normalizeFallbackModels(models: string | string[] | undefined): string[] | undefined {
if (!models) return undefined
if (typeof models === "string") return [models]
return models
}

View File

@@ -0,0 +1,53 @@
/**
* Session Category Registry
*
* Maintains a mapping of session IDs to their assigned categories.
* Used by runtime-fallback hook to lookup category-specific fallback_models.
*/
// Map of sessionID -> category name
const sessionCategoryMap = new Map<string, string>()
export const SessionCategoryRegistry = {
/**
* Register a session with its category
*/
register: (sessionID: string, category: string): void => {
sessionCategoryMap.set(sessionID, category)
},
/**
* Get the category for a session
*/
get: (sessionID: string): string | undefined => {
return sessionCategoryMap.get(sessionID)
},
/**
* Remove a session from the registry (cleanup)
*/
remove: (sessionID: string): void => {
sessionCategoryMap.delete(sessionID)
},
/**
* Check if a session is registered
*/
has: (sessionID: string): boolean => {
return sessionCategoryMap.has(sessionID)
},
/**
* Get the size of the registry (for debugging)
*/
size: (): number => {
return sessionCategoryMap.size
},
/**
* Clear all entries (use with caution, mainly for testing)
*/
clear: (): void => {
sessionCategoryMap.clear()
},
}

View File

@@ -4,6 +4,7 @@ import { getTimingConfig } from "./timing"
import { storeToolMetadata } from "../../features/tool-metadata-store"
import { formatDetailedError } from "./error-formatting"
import { getSessionTools } from "../../shared/session-tools-store"
import { SessionCategoryRegistry } from "../../shared/session-category-registry"
export async function executeBackgroundTask(
args: DelegateTaskArgs,
@@ -48,6 +49,10 @@ export async function executeBackgroundTask(
sessionId = updated?.sessionID
}
if (args.category && sessionId) {
SessionCategoryRegistry.register(sessionId, args.category)
}
const unstableMeta = {
title: args.description,
metadata: {

View File

@@ -5,6 +5,7 @@ import { getTaskToastManager } from "../../features/task-toast-manager"
import { storeToolMetadata } from "../../features/tool-metadata-store"
import { subagentSessions } from "../../features/claude-code-session-state"
import { log } from "../../shared/logger"
import { SessionCategoryRegistry } from "../../shared/session-category-registry"
import { formatDuration } from "./time-formatter"
import { formatDetailedError } from "./error-formatting"
import { syncTaskDeps, type SyncTaskDeps } from "./sync-task-deps"
@@ -41,6 +42,10 @@ export async function executeSyncTask(
syncSessionID = sessionID
subagentSessions.add(sessionID)
if (args.category) {
SessionCategoryRegistry.register(sessionID, args.category)
}
if (onSyncSessionCreated) {
log("[task] Invoking onSyncSessionCreated callback", { sessionID, parentID: parentContext.sessionID })
await onSyncSessionCreated({
@@ -145,6 +150,7 @@ session_id: ${sessionID}
} finally {
if (syncSessionID) {
subagentSessions.delete(syncSessionID)
SessionCategoryRegistry.remove(syncSessionID)
}
}
}

View File

@@ -1044,7 +1044,7 @@ describe("sisyphus-task", () => {
modelID: "claude-opus-4-6",
variant: "max",
})
})
}, { timeout: 20000 })
test("DEFAULT_CATEGORIES variant passes to sync session.prompt WITHOUT userCategories", async () => {
// given - NO userCategories, testing DEFAULT_CATEGORIES for sync mode
@@ -2624,31 +2624,35 @@ describe("sisyphus-task", () => {
toolContext
)
// then - agent-browser skill should be resolved (not in notFound)
// then - agent-browser skill should be resolved
expect(promptBody).toBeDefined()
expect(promptBody.system).toBeDefined()
expect(promptBody.system).toContain("agent-browser")
expect(promptBody.system).toContain("<Category_Context>")
expect(String(promptBody.system).startsWith("<Category_Context>")).toBe(false)
}, { timeout: 20000 })
test("should NOT resolve agent-browser skill when browserProvider is not set", async () => {
// given - task without browserProvider (defaults to playwright)
test("should resolve agent-browser skill even when browserProvider is not set", async () => {
// given - delegate_task without browserProvider
const { createDelegateTask } = require("./tools")
let promptBody: any
const mockManager = { launch: async () => ({}) }
const mockClient = {
app: { agents: async () => ({ data: [] }) },
config: { get: async () => ({ data: { model: SYSTEM_DEFAULT_MODEL } }) },
session: {
get: async () => ({ data: { directory: "/project" } }),
create: async () => ({ data: { id: "ses_no_browser_provider" } }),
prompt: async () => ({ data: {} }),
promptAsync: async () => ({ data: {} }),
messages: async () => ({
data: [{ info: { role: "assistant" }, parts: [{ type: "text", text: "Done" }] }]
}),
status: async () => ({ data: {} }),
},
}
config: { get: async () => ({ data: { model: SYSTEM_DEFAULT_MODEL } }) },
session: {
get: async () => ({ data: { directory: "/project" } }),
create: async () => ({ data: { id: "ses_no_browser_provider" } }),
prompt: async (input: any) => {
promptBody = input.body
return { data: {} }
},
messages: async () => ({
data: [{ info: { role: "assistant" }, parts: [{ type: "text", text: "Done" }] }]
}),
status: async () => ({ data: {} }),
},
}
// No browserProvider passed
const tool = createDelegateTask({
@@ -2675,7 +2679,7 @@ describe("sisyphus-task", () => {
toolContext
)
// then - should return skill not found error
// then - agent-browser skill should NOT resolve without browserProvider
expect(result).toContain("Skills not found")
expect(result).toContain("agent-browser")
})