feat(model-settings-compat): add variant/reasoningEffort compatibility resolver

- Registry-based model family detection (provider-agnostic)
- Variant and reasoningEffort ladder downgrade logic
- Three-tier resolution: metadata override → family heuristic → unknown drop
- Comprehensive test suite covering all model families
This commit is contained in:
Ravi Tharuma
2026-03-18 20:37:42 +01:00
parent fb085538eb
commit d4f962b55d
13 changed files with 971 additions and 127 deletions

View File

@@ -0,0 +1,86 @@
# Model Settings Compatibility Resolver Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Centralize compatibility handling for `variant` and `reasoningEffort` so an already-selected model receives the best valid settings for that exact model.
**Architecture:** Introduce a pure shared resolver in `src/shared/` that computes compatible settings and records downgrades/removals. Integrate it first in `chat.params`, then keep Claude-specific effort logic as a thin layer rather than a special-case policy owner.
**Tech Stack:** TypeScript, Bun test, existing shared model normalization/utilities, OpenCode plugin `chat.params` path.
---
### Task 1: Create the pure compatibility resolver
**Files:**
- Create: `src/shared/model-settings-compatibility.ts`
- Create: `src/shared/model-settings-compatibility.test.ts`
- Modify: `src/shared/index.ts`
- [ ] **Step 1: Write failing tests for exact keep behavior**
- [ ] **Step 2: Write failing tests for downgrade behavior (`max` -> `high`, `xhigh` -> `high` where needed)**
- [ ] **Step 3: Write failing tests for unsupported-value removal**
- [ ] **Step 4: Write failing tests for model-family distinctions (Opus vs Sonnet/Haiku, GPT-family variants)**
- [ ] **Step 5: Implement the pure resolver with explicit capability ladders**
- [ ] **Step 6: Export the resolver from `src/shared/index.ts`**
- [ ] **Step 7: Run `bun test src/shared/model-settings-compatibility.test.ts`**
- [ ] **Step 8: Commit**
### Task 2: Integrate resolver into chat.params
**Files:**
- Modify: `src/plugin/chat-params.ts`
- Modify: `src/plugin/chat-params.test.ts`
- [ ] **Step 1: Write failing tests showing `chat.params` applies resolver output to runtime settings**
- [ ] **Step 2: Ensure tests cover both `variant` and `reasoningEffort` decisions**
- [ ] **Step 3: Update `chat-params.ts` to call the shared resolver before hook-specific adjustments**
- [ ] **Step 4: Preserve existing prompt-param-store merging behavior**
- [ ] **Step 5: Run `bun test src/plugin/chat-params.test.ts`**
- [ ] **Step 6: Commit**
### Task 3: Re-scope anthropic-effort around the resolver
**Files:**
- Modify: `src/hooks/anthropic-effort/hook.ts`
- Modify: `src/hooks/anthropic-effort/index.test.ts`
- [ ] **Step 1: Write failing tests that codify the intended remaining Anthropic-specific behavior after centralization**
- [ ] **Step 2: Reduce `anthropic-effort` to Claude/Anthropic-specific effort injection where still needed**
- [ ] **Step 3: Remove duplicated compatibility policy from the hook if the shared resolver now owns it**
- [ ] **Step 4: Run `bun test src/hooks/anthropic-effort/index.test.ts`**
- [ ] **Step 5: Commit**
### Task 4: Add integration/regression coverage across real request paths
**Files:**
- Modify: `src/plugin/chat-params.test.ts`
- Modify: `src/hooks/anthropic-effort/index.test.ts`
- Add tests only where needed in nearby suites
- [ ] **Step 1: Add regression test for non-Opus Claude with `variant=max` resolving to compatible settings without ad hoc path-only logic**
- [ ] **Step 2: Add regression test for GPT-style `reasoningEffort` compatibility**
- [ ] **Step 3: Add regression test showing supported values remain unchanged**
- [ ] **Step 4: Run the focused test set**
- [ ] **Step 5: Commit**
### Task 5: Verify full quality bar
**Files:**
- No intended code changes
- [ ] **Step 1: Run `bun run typecheck`**
- [ ] **Step 2: Run a focused suite for the touched files**
- [ ] **Step 3: If clean, run `bun test`**
- [ ] **Step 4: Review diff for accidental scope creep**
- [ ] **Step 5: Commit any final cleanup**
### Task 6: Prepare PR metadata
**Files:**
- No repo file change required unless docs are updated further
- [ ] **Step 1: Write a human summary explaining this is settings compatibility, not model fallback**
- [ ] **Step 2: Document scope: Phase 1 covers `variant` and `reasoningEffort` only**
- [ ] **Step 3: Document explicit non-goals: no model switching, no automatic upscaling in Phase 1**
- [ ] **Step 4: Request review**

View File

@@ -0,0 +1,164 @@
# Model Settings Compatibility Resolver Design
## Goal
Introduce a central resolver that takes an already-selected model and a set of desired model settings, then returns the best compatible configuration for that exact model.
This is explicitly separate from model fallback.
## Problem
Today, logic for `variant` and `reasoningEffort` compatibility is scattered across multiple places:
- `hooks/anthropic-effort`
- `plugin/chat-params`
- agent/category/fallback config layers
- delegate/background prompt plumbing
That creates inconsistent behavior:
- some paths clamp unsupported levels
- some paths pass them through unchanged
- some paths silently drop them
- some paths use model-family-specific assumptions that do not generalize
The result is brittle request behavior even when the chosen model itself is valid.
## Scope
Phase 1 covers only:
- `variant`
- `reasoningEffort`
Out of scope for Phase 1:
- model fallback itself
- `thinking`
- `maxTokens`
- `temperature`
- `top_p`
- automatic upward remapping of settings
## Desired behavior
Given a fixed model and desired settings:
1. If a desired value is supported, keep it.
2. If not supported, downgrade to the nearest lower compatible value.
3. If no compatible value exists, drop the field.
4. Do not switch models.
5. Do not automatically upgrade settings in Phase 1.
## Architecture
Add a central module:
- `src/shared/model-settings-compatibility.ts`
Core API:
```ts
type DesiredModelSettings = {
variant?: string
reasoningEffort?: string
}
type ModelSettingsCompatibilityInput = {
providerID: string
modelID: string
desired: DesiredModelSettings
}
type ModelSettingsCompatibilityChange = {
field: "variant" | "reasoningEffort"
from: string
to?: string
reason: string
}
type ModelSettingsCompatibilityResult = {
variant?: string
reasoningEffort?: string
changes: ModelSettingsCompatibilityChange[]
}
```
## Compatibility model
Phase 1 should be **metadata-first where the platform exposes reliable capability data**, and only fall back to family-based rules when that metadata is absent.
### Variant compatibility
Preferred source of truth:
- OpenCode/provider model metadata (`variants`)
Fallback when metadata is unavailable:
- family-based ladders
Examples of fallback ladders:
- Claude Opus family: `low`, `medium`, `high`, `max`
- Claude Sonnet/Haiku family: `low`, `medium`, `high`
- OpenAI GPT family: conservative family fallback only when metadata is missing
- Unknown family: drop unsupported values conservatively
### Reasoning effort compatibility
Current Phase 1 source of truth:
- conservative model/provider family heuristics
Reason:
- the currently available OpenCode SDK/provider metadata exposes model `variants`, but does not expose an equivalent per-model capability list for `reasoningEffort` levels
Examples:
- GPT/OpenAI-style models: `low`, `medium`, `high`, `xhigh` where supported by family heuristics
- Claude family via current OpenCode path: treat `reasoningEffort` as unsupported in Phase 1 and remove it
The resolver should remain pure model/settings logic only. Transport restrictions remain the responsibility of the request-building path.
## Separation of concerns
This design intentionally separates:
- model selection (`resolveModel...`, fallback chains)
- settings compatibility (this resolver)
- request transport compatibility (`chat.params`, prompt body constraints)
That keeps responsibilities clear:
- choose model first
- normalize settings second
- build request third
## First integration point
Phase 1 should first integrate into `chat.params`.
Why:
- it is already the centralized path for request-time tuning
- it can influence provider-facing options without leaking unsupported fields into prompt payload bodies
- it avoids trying to patch every prompt constructor at once
## Rollout plan
### Phase 1
- add resolver module and tests
- integrate into `chat.params`
- migrate `anthropic-effort` to either use the resolver or become a thin Claude-specific supplement around it
### Phase 2
- expand to `thinking`, `maxTokens`, `temperature`, `top_p`
- formalize request-path capability tables if needed
### Phase 3
- centralize all variant/reasoning normalization away from scattered hooks and ad hoc callers
## Risks
- Overfitting family rules to current model naming conventions
- Accidentally changing request semantics on paths that currently rely on implicit behavior
- Mixing provider transport limitations with model capability logic
## Mitigations
- Keep resolver pure and narrowly scoped in Phase 1
- Add explicit regression tests for keep/downgrade/drop decisions
- Integrate at one central point first (`chat.params`)
- Preserve existing behavior where desired values are already valid
## Recommendation
Proceed with the central resolver as a new, isolated implementation in a dedicated branch/worktree.
This is the clean long-term path and is more reviewable than continuing to add special-case clamps in hooks.

View File

@@ -371,6 +371,26 @@ describe("CategoryConfigSchema", () => {
} }
}) })
test("accepts reasoningEffort values none and minimal", () => {
// given
const noneConfig = { reasoningEffort: "none" }
const minimalConfig = { reasoningEffort: "minimal" }
// when
const noneResult = CategoryConfigSchema.safeParse(noneConfig)
const minimalResult = CategoryConfigSchema.safeParse(minimalConfig)
// then
expect(noneResult.success).toBe(true)
expect(minimalResult.success).toBe(true)
if (noneResult.success) {
expect(noneResult.data.reasoningEffort).toBe("none")
}
if (minimalResult.success) {
expect(minimalResult.data.reasoningEffort).toBe("minimal")
}
})
test("rejects non-string variant", () => { test("rejects non-string variant", () => {
// given // given
const config = { model: "openai/gpt-5.4", variant: 123 } const config = { model: "openai/gpt-5.4", variant: 123 }

View File

@@ -35,7 +35,7 @@ export const AgentOverrideConfigSchema = z.object({
}) })
.optional(), .optional(),
/** Reasoning effort level (OpenAI). Overrides category and default settings. */ /** Reasoning effort level (OpenAI). Overrides category and default settings. */
reasoningEffort: z.enum(["low", "medium", "high", "xhigh"]).optional(), reasoningEffort: z.enum(["none", "minimal", "low", "medium", "high", "xhigh"]).optional(),
/** Text verbosity level. */ /** Text verbosity level. */
textVerbosity: z.enum(["low", "medium", "high"]).optional(), textVerbosity: z.enum(["low", "medium", "high"]).optional(),
/** Provider-specific options. Passed directly to OpenCode SDK. */ /** Provider-specific options. Passed directly to OpenCode SDK. */

View File

@@ -16,7 +16,7 @@ export const CategoryConfigSchema = z.object({
budgetTokens: z.number().optional(), budgetTokens: z.number().optional(),
}) })
.optional(), .optional(),
reasoningEffort: z.enum(["low", "medium", "high", "xhigh"]).optional(), reasoningEffort: z.enum(["none", "minimal", "low", "medium", "high", "xhigh"]).optional(),
textVerbosity: z.enum(["low", "medium", "high"]).optional(), textVerbosity: z.enum(["low", "medium", "high"]).optional(),
tools: z.record(z.string(), z.boolean()).optional(), tools: z.record(z.string(), z.boolean()).optional(),
prompt_append: z.string().optional(), prompt_append: z.string().optional(),

View File

@@ -1,6 +1,6 @@
import { log, normalizeModelID } from "../../shared" import { log, normalizeModelID } from "../../shared"
const OPUS_PATTERN = /claude-opus/i const OPUS_PATTERN = /claude-.*opus/i
function isClaudeProvider(providerID: string, modelID: string): boolean { function isClaudeProvider(providerID: string, modelID: string): boolean {
if (["anthropic", "google-vertex-anthropic", "opencode"].includes(providerID)) return true if (["anthropic", "google-vertex-anthropic", "opencode"].includes(providerID)) return true

View File

@@ -45,75 +45,31 @@ function createMockParams(overrides: {
} }
describe("createAnthropicEffortHook", () => { describe("createAnthropicEffortHook", () => {
describe("opus 4-6 with variant max", () => { describe("opus family with variant max", () => {
it("should inject effort max for anthropic opus-4-6 with variant max", async () => { it("injects effort max for anthropic opus-4-6", async () => {
//#given anthropic opus-4-6 model with variant max
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({}) const { input, output } = createMockParams({})
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should be injected into options
expect(output.options.effort).toBe("max") expect(output.options.effort).toBe("max")
}) })
it("should inject effort max for github-copilot claude-opus-4-6", async () => { it("injects effort max for another opus family model such as opus-4-5", async () => {
//#given github-copilot provider with claude-opus-4-6
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ const { input, output } = createMockParams({ modelID: "claude-opus-4-5" })
providerID: "github-copilot",
modelID: "claude-opus-4-6",
})
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should be injected (github-copilot resolves to anthropic)
expect(output.options.effort).toBe("max") expect(output.options.effort).toBe("max")
}) })
it("should inject effort max for opencode provider with claude-opus-4-6", async () => { it("injects effort max for dotted opus ids", async () => {
//#given opencode provider with claude-opus-4-6
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ const { input, output } = createMockParams({ modelID: "claude-opus-4.6" })
providerID: "opencode",
modelID: "claude-opus-4-6",
})
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should be injected
expect(output.options.effort).toBe("max")
})
it("should inject effort max for google-vertex-anthropic provider", async () => {
//#given google-vertex-anthropic provider with claude-opus-4-6
const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({
providerID: "google-vertex-anthropic",
modelID: "claude-opus-4-6",
})
//#when chat.params hook is called
await hook["chat.params"](input, output)
//#then effort should be injected
expect(output.options.effort).toBe("max")
})
it("should handle normalized model ID with dots (opus-4.6)", async () => {
//#given model ID with dots instead of hyphens
const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({
modelID: "claude-opus-4.6",
})
//#when chat.params hook is called
await hook["chat.params"](input, output)
//#then should normalize and inject effort
expect(output.options.effort).toBe("max") expect(output.options.effort).toBe("max")
}) })
@@ -133,39 +89,30 @@ describe("createAnthropicEffortHook", () => {
}) })
}) })
describe("conditions NOT met - should skip", () => { describe("skip conditions", () => {
it("should NOT inject effort when variant is not max", async () => { it("does nothing when variant is not max", async () => {
//#given opus-4-6 with variant high (not max)
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ variant: "high" }) const { input, output } = createMockParams({ variant: "high" })
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should NOT be injected
expect(output.options.effort).toBeUndefined() expect(output.options.effort).toBeUndefined()
}) })
it("should NOT inject effort when variant is undefined", async () => { it("does nothing when variant is undefined", async () => {
//#given opus-4-6 with no variant
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ variant: undefined }) const { input, output } = createMockParams({ variant: undefined })
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should NOT be injected
expect(output.options.effort).toBeUndefined() expect(output.options.effort).toBeUndefined()
}) })
it("should clamp effort to high for non-opus claude model with variant max", async () => { it("should clamp effort to high for non-opus claude model with variant max", async () => {
//#given claude-sonnet-4-6 (not opus) with variant max //#given claude-sonnet-4-6 (not opus) with variant max
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ const { input, output } = createMockParams({ modelID: "claude-sonnet-4-6" })
modelID: "claude-sonnet-4-6",
})
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should be clamped to high (not max) //#then effort should be clamped to high (not max)
@@ -173,74 +120,24 @@ describe("createAnthropicEffortHook", () => {
expect(input.message.variant).toBe("high") expect(input.message.variant).toBe("high")
}) })
it("should NOT inject effort for non-anthropic provider with non-claude model", async () => { it("does nothing for non-claude providers/models", async () => {
//#given openai provider with gpt model
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ const { input, output } = createMockParams({ providerID: "openai", modelID: "gpt-5.4" })
providerID: "openai",
modelID: "gpt-5.4",
})
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then effort should NOT be injected
expect(output.options.effort).toBeUndefined()
})
it("should NOT throw when model.modelID is undefined", async () => {
//#given model with undefined modelID (runtime edge case)
const hook = createAnthropicEffortHook()
const input = {
sessionID: "test-session",
agent: { name: "sisyphus" },
model: { providerID: "anthropic", modelID: undefined as unknown as string },
provider: { id: "anthropic" },
message: { variant: "max" as const },
}
const output = { temperature: 0.1, options: {} }
//#when chat.params hook is called with undefined modelID
await hook["chat.params"](input, output)
//#then should gracefully skip without throwing
expect(output.options.effort).toBeUndefined() expect(output.options.effort).toBeUndefined()
}) })
}) })
describe("preserves existing options", () => { describe("existing options", () => {
it("should NOT overwrite existing effort if already set", async () => { it("does not overwrite existing effort", async () => {
//#given options already have effort set
const hook = createAnthropicEffortHook() const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({ const { input, output } = createMockParams({ existingOptions: { effort: "high" } })
existingOptions: { effort: "high" },
})
//#when chat.params hook is called
await hook["chat.params"](input, output) await hook["chat.params"](input, output)
//#then existing effort should be preserved
expect(output.options.effort).toBe("high") expect(output.options.effort).toBe("high")
}) })
it("should preserve other existing options when injecting effort", async () => {
//#given options with existing thinking config
const hook = createAnthropicEffortHook()
const { input, output } = createMockParams({
existingOptions: {
thinking: { type: "enabled", budgetTokens: 31999 },
},
})
//#when chat.params hook is called
await hook["chat.params"](input, output)
//#then effort should be added without affecting thinking
expect(output.options.effort).toBe("max")
expect(output.options.thinking).toEqual({
type: "enabled",
budgetTokens: 31999,
})
})
}) })
}) })

View File

@@ -32,7 +32,13 @@ export function createPluginInterface(args: {
return { return {
tool: tools, tool: tools,
"chat.params": createChatParamsHandler({ anthropicEffort: hooks.anthropicEffort }), "chat.params": async (input: unknown, output: unknown) => {
const handler = createChatParamsHandler({
anthropicEffort: hooks.anthropicEffort,
client: ctx.client,
})
await handler(input, output)
},
"chat.headers": createChatHeadersHandler({ ctx }), "chat.headers": createChatHeadersHandler({ ctx }),
@@ -68,9 +74,5 @@ export function createPluginInterface(args: {
ctx, ctx,
hooks, hooks,
}), }),
"tool.definition": async (input, output) => {
await hooks.todoDescriptionOverride?.["tool.definition"]?.(input, output)
},
} }
} }

View File

@@ -1,4 +1,6 @@
import { normalizeSDKResponse } from "../shared/normalize-sdk-response"
import { getSessionPromptParams } from "../shared/session-prompt-params-state" import { getSessionPromptParams } from "../shared/session-prompt-params-state"
import { resolveCompatibleModelSettings } from "../shared"
export type ChatParamsInput = { export type ChatParamsInput = {
sessionID: string sessionID: string
@@ -19,6 +21,25 @@ export type ChatParamsOutput = {
options: Record<string, unknown> options: Record<string, unknown>
} }
type ProviderListClient = {
provider?: {
list?: () => Promise<unknown>
}
}
type ProviderModelMetadata = {
variants?: Record<string, unknown>
}
type ProviderListEntry = {
id?: string
models?: Record<string, ProviderModelMetadata>
}
type ProviderListData = {
all?: ProviderListEntry[]
}
function isRecord(value: unknown): value is Record<string, unknown> { function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null return typeof value === "object" && value !== null
} }
@@ -49,7 +70,11 @@ function buildChatParamsInput(raw: unknown): ChatParamsHookInput | null {
if (!agentName) return null if (!agentName) return null
const providerID = model.providerID const providerID = model.providerID
const modelID = model.modelID const modelID = typeof model.modelID === "string"
? model.modelID
: typeof model.id === "string"
? model.id
: undefined
const providerId = provider.id const providerId = provider.id
const variant = message.variant const variant = message.variant
@@ -76,8 +101,33 @@ function isChatParamsOutput(raw: unknown): raw is ChatParamsOutput {
return isRecord(raw.options) return isRecord(raw.options)
} }
async function getVariantCapabilities(
client: ProviderListClient | undefined,
model: { providerID: string; modelID: string },
): Promise<string[] | undefined> {
const providerList = client?.provider?.list
if (typeof providerList !== "function") {
return undefined
}
try {
const response = await providerList()
const data = normalizeSDKResponse<ProviderListData>(response, {})
const providerEntry = data.all?.find((entry) => entry.id === model.providerID)
const variants = providerEntry?.models?.[model.modelID]?.variants
if (!variants) {
return undefined
}
return Object.keys(variants)
} catch {
return undefined
}
}
export function createChatParamsHandler(args: { export function createChatParamsHandler(args: {
anthropicEffort: { "chat.params"?: (input: ChatParamsHookInput, output: ChatParamsOutput) => Promise<void> } | null anthropicEffort: { "chat.params"?: (input: ChatParamsHookInput, output: ChatParamsOutput) => Promise<void> } | null
client?: ProviderListClient
}): (input: unknown, output: unknown) => Promise<void> { }): (input: unknown, output: unknown) => Promise<void> {
return async (input, output): Promise<void> => { return async (input, output): Promise<void> => {
const normalizedInput = buildChatParamsInput(input) const normalizedInput = buildChatParamsInput(input)
@@ -100,6 +150,37 @@ export function createChatParamsHandler(args: {
} }
} }
const variantCapabilities = await getVariantCapabilities(args.client, normalizedInput.model)
const compatibility = resolveCompatibleModelSettings({
providerID: normalizedInput.model.providerID,
modelID: normalizedInput.model.modelID,
desired: {
variant: normalizedInput.message.variant,
reasoningEffort: typeof output.options.reasoningEffort === "string"
? output.options.reasoningEffort
: undefined,
},
capabilities: {
variants: variantCapabilities,
},
})
if (normalizedInput.rawMessage) {
if (compatibility.variant !== undefined) {
normalizedInput.rawMessage.variant = compatibility.variant
} else {
delete normalizedInput.rawMessage.variant
}
}
normalizedInput.message = normalizedInput.rawMessage as { variant?: string }
if (compatibility.reasoningEffort !== undefined) {
output.options.reasoningEffort = compatibility.reasoningEffort
} else if ("reasoningEffort" in output.options) {
delete output.options.reasoningEffort
}
await args.anthropicEffort?.["chat.params"]?.(normalizedInput, output) await args.anthropicEffort?.["chat.params"]?.(normalizedInput, output)
} }
} }

View File

@@ -43,6 +43,7 @@ export type {
ModelResolutionResult, ModelResolutionResult,
} from "./model-resolution-types" } from "./model-resolution-types"
export * from "./model-availability" export * from "./model-availability"
export * from "./model-settings-compatibility"
export * from "./fallback-model-availability" export * from "./fallback-model-availability"
export * from "./connected-providers-cache" export * from "./connected-providers-cache"
export * from "./context-limit-resolver" export * from "./context-limit-resolver"

View File

@@ -0,0 +1,302 @@
import { describe, expect, test } from "bun:test"
import { resolveCompatibleModelSettings } from "./model-settings-compatibility"
describe("resolveCompatibleModelSettings", () => {
test("keeps supported Claude Opus variant unchanged", () => {
const result = resolveCompatibleModelSettings({
providerID: "anthropic",
modelID: "claude-opus-4-6",
desired: { variant: "max" },
})
expect(result).toEqual({
variant: "max",
reasoningEffort: undefined,
changes: [],
})
})
test("uses model metadata first for variant support", () => {
const result = resolveCompatibleModelSettings({
providerID: "anthropic",
modelID: "claude-opus-4-6",
desired: { variant: "max" },
capabilities: { variants: ["low", "medium", "high"] },
})
expect(result).toEqual({
variant: "high",
reasoningEffort: undefined,
changes: [
{
field: "variant",
from: "max",
to: "high",
reason: "unsupported-by-model-metadata",
},
],
})
})
test("prefers metadata over family heuristics even when family would allow a higher level", () => {
const result = resolveCompatibleModelSettings({
providerID: "anthropic",
modelID: "claude-opus-4-6",
desired: { variant: "max" },
capabilities: { variants: ["low", "medium"] },
})
expect(result.variant).toBe("medium")
expect(result.changes).toEqual([
{
field: "variant",
from: "max",
to: "medium",
reason: "unsupported-by-model-metadata",
},
])
})
test("downgrades unsupported Claude Sonnet max variant to high when metadata is absent", () => {
const result = resolveCompatibleModelSettings({
providerID: "anthropic",
modelID: "claude-sonnet-4-6",
desired: { variant: "max" },
})
expect(result.variant).toBe("high")
expect(result.changes).toEqual([
{
field: "variant",
from: "max",
to: "high",
reason: "unsupported-by-model-family",
},
])
})
test("keeps supported GPT reasoningEffort unchanged", () => {
const result = resolveCompatibleModelSettings({
providerID: "openai",
modelID: "gpt-5.4",
desired: { reasoningEffort: "high" },
})
expect(result).toEqual({
variant: undefined,
reasoningEffort: "high",
changes: [],
})
})
test("downgrades gpt-5 reasoningEffort max to xhigh", () => {
// given
const input = {
providerID: "openai",
modelID: "gpt-5.4",
desired: { reasoningEffort: "max" },
}
// when
const result = resolveCompatibleModelSettings(input)
// then
expect(result.reasoningEffort).toBe("xhigh")
expect(result.changes).toEqual([
{
field: "reasoningEffort",
from: "max",
to: "xhigh",
reason: "unsupported-by-model-family",
},
])
})
test("keeps supported OpenAI reasoning-family effort for o-series models", () => {
const result = resolveCompatibleModelSettings({
providerID: "openai",
modelID: "o3-mini",
desired: { reasoningEffort: "high" },
})
expect(result).toEqual({
variant: undefined,
reasoningEffort: "high",
changes: [],
})
})
test("downgrades openai reasoning-family effort xhigh to high", () => {
// given
const input = {
providerID: "openai",
modelID: "o3-mini",
desired: { reasoningEffort: "xhigh" },
}
// when
const result = resolveCompatibleModelSettings(input)
// then
expect(result.reasoningEffort).toBe("high")
expect(result.changes).toEqual([
{
field: "reasoningEffort",
from: "xhigh",
to: "high",
reason: "unsupported-by-model-family",
},
])
})
test("drops reasoningEffort for gpt-5 mini models", () => {
// given
const input = {
providerID: "openai",
modelID: "gpt-5.4-mini",
desired: { reasoningEffort: "high" },
}
// when
const result = resolveCompatibleModelSettings(input)
// then
expect(result.reasoningEffort).toBeUndefined()
expect(result.changes).toEqual([
{
field: "reasoningEffort",
from: "high",
to: undefined,
reason: "unsupported-by-model-family",
},
])
})
test("treats non-openai o-series models as unknown", () => {
// given
const input = {
providerID: "ollama",
modelID: "o3",
desired: { reasoningEffort: "high" },
}
// when
const result = resolveCompatibleModelSettings(input)
// then
expect(result.reasoningEffort).toBeUndefined()
expect(result.changes).toEqual([
{
field: "reasoningEffort",
from: "high",
to: undefined,
reason: "unknown-model-family",
},
])
})
test("does not record case-only normalization as a compatibility downgrade", () => {
const result = resolveCompatibleModelSettings({
providerID: "openai",
modelID: "gpt-5.4",
desired: { variant: "HIGH", reasoningEffort: "HIGH" },
})
expect(result).toEqual({
variant: "high",
reasoningEffort: "high",
changes: [],
})
})
test("downgrades unsupported GPT reasoningEffort to nearest lower level", () => {
const result = resolveCompatibleModelSettings({
providerID: "openai",
modelID: "gpt-4.1",
desired: { reasoningEffort: "xhigh" },
})
expect(result.reasoningEffort).toBe("high")
expect(result.changes).toEqual([
{
field: "reasoningEffort",
from: "xhigh",
to: "high",
reason: "unsupported-by-model-family",
},
])
})
test("drops reasoningEffort for Claude family", () => {
const result = resolveCompatibleModelSettings({
providerID: "anthropic",
modelID: "claude-sonnet-4-6",
desired: { reasoningEffort: "high" },
})
expect(result.reasoningEffort).toBeUndefined()
expect(result.changes).toEqual([
{
field: "reasoningEffort",
from: "high",
to: undefined,
reason: "unsupported-by-model-family",
},
])
})
test("handles combined variant and reasoningEffort normalization", () => {
const result = resolveCompatibleModelSettings({
providerID: "anthropic",
modelID: "claude-sonnet-4-6",
desired: { variant: "max", reasoningEffort: "high" },
})
expect(result).toEqual({
variant: "high",
reasoningEffort: undefined,
changes: [
{
field: "variant",
from: "max",
to: "high",
reason: "unsupported-by-model-family",
},
{
field: "reasoningEffort",
from: "high",
to: undefined,
reason: "unsupported-by-model-family",
},
],
})
})
test("treats unknown model families conservatively by dropping unsupported settings", () => {
const result = resolveCompatibleModelSettings({
providerID: "mystery",
modelID: "mystery-model-1",
desired: { variant: "max", reasoningEffort: "high" },
})
expect(result).toEqual({
variant: undefined,
reasoningEffort: undefined,
changes: [
{
field: "variant",
from: "max",
to: undefined,
reason: "unknown-model-family",
},
{
field: "reasoningEffort",
from: "high",
to: undefined,
reason: "unknown-model-family",
},
],
})
})
})

View File

@@ -0,0 +1,260 @@
import { normalizeModelID } from "./model-normalization"
type CompatibilityField = "variant" | "reasoningEffort"
type DesiredModelSettings = {
variant?: string
reasoningEffort?: string
}
type VariantCapabilities = {
variants?: string[]
}
export type ModelSettingsCompatibilityInput = {
providerID: string
modelID: string
desired: DesiredModelSettings
capabilities?: VariantCapabilities
}
export type ModelSettingsCompatibilityChange = {
field: CompatibilityField
from: string
to?: string
reason: "unsupported-by-model-family" | "unknown-model-family" | "unsupported-by-model-metadata"
}
export type ModelSettingsCompatibilityResult = {
variant?: string
reasoningEffort?: string
changes: ModelSettingsCompatibilityChange[]
}
type ModelFamily = "claude-opus" | "claude-non-opus" | "openai-reasoning" | "gpt-5" | "gpt-5-mini" | "gpt-legacy" | "unknown"
const VARIANT_LADDER = ["low", "medium", "high", "xhigh", "max"]
const REASONING_LADDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"]
function detectModelFamily(providerID: string, modelID: string): ModelFamily {
const provider = providerID.toLowerCase()
const model = normalizeModelID(modelID).toLowerCase()
const isClaudeProvider = [
"anthropic",
"google-vertex-anthropic",
"aws-bedrock-anthropic",
].includes(provider)
|| (["github-copilot", "opencode", "aws-bedrock", "bedrock"].includes(provider) && model.includes("claude"))
if (isClaudeProvider) {
return /claude(?:-\d+(?:-\d+)*)?-opus/.test(model) ? "claude-opus" : "claude-non-opus"
}
const isOpenAiReasoningFamily = provider === "openai" && (/^o\d(?:$|-)/.test(model) || model.includes("reasoning"))
if (isOpenAiReasoningFamily) {
return "openai-reasoning"
}
if (/gpt-5.*-mini/.test(model)) {
return "gpt-5-mini"
}
if (model.includes("gpt-5")) {
return "gpt-5"
}
if (model.includes("gpt") || (provider === "openai" && /^o\d(?:$|-)/.test(model))) {
return "gpt-legacy"
}
return "unknown"
}
function downgradeWithinLadder(value: string, allowed: string[], ladder: string[]): string | undefined {
const requestedIndex = ladder.indexOf(value)
if (requestedIndex === -1) return undefined
for (let index = requestedIndex; index >= 0; index -= 1) {
const candidate = ladder[index]
if (allowed.includes(candidate)) {
return candidate
}
}
return undefined
}
function normalizeCapabilitiesVariants(capabilities: VariantCapabilities | undefined): string[] | undefined {
if (!capabilities?.variants || capabilities.variants.length === 0) {
return undefined
}
return capabilities.variants.map((variant) => variant.toLowerCase())
}
function resolveVariant(
modelFamily: ModelFamily,
variant: string,
capabilities?: VariantCapabilities,
): { value?: string; reason?: ModelSettingsCompatibilityChange["reason"] } {
const normalized = variant.toLowerCase()
const metadataVariants = normalizeCapabilitiesVariants(capabilities)
if (metadataVariants) {
if (metadataVariants.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, metadataVariants, VARIANT_LADDER),
reason: "unsupported-by-model-metadata",
}
}
if (modelFamily === "claude-opus") {
const allowed = ["low", "medium", "high", "max"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "claude-non-opus") {
const allowed = ["low", "medium", "high"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "gpt-5" || modelFamily === "gpt-5-mini") {
const allowed = ["low", "medium", "high", "xhigh", "max"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "openai-reasoning") {
const allowed = ["low", "medium", "high"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "gpt-legacy") {
const allowed = ["low", "medium", "high"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
reason: "unsupported-by-model-family",
}
}
return { value: undefined, reason: "unknown-model-family" }
}
function resolveReasoningEffort(modelFamily: ModelFamily, reasoningEffort: string): { value?: string; reason?: ModelSettingsCompatibilityChange["reason"] } {
const normalized = reasoningEffort.toLowerCase()
if (modelFamily === "gpt-5") {
const allowed = ["none", "minimal", "low", "medium", "high", "xhigh"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "gpt-5-mini") {
return { value: undefined, reason: "unsupported-by-model-family" }
}
if (modelFamily === "openai-reasoning") {
const allowed = ["none", "minimal", "low", "medium", "high"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "gpt-legacy") {
const allowed = ["none", "minimal", "low", "medium", "high"]
if (allowed.includes(normalized)) {
return { value: normalized }
}
return {
value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER),
reason: "unsupported-by-model-family",
}
}
if (modelFamily === "claude-opus" || modelFamily === "claude-non-opus") {
return { value: undefined, reason: "unsupported-by-model-family" }
}
return { value: undefined, reason: "unknown-model-family" }
}
export function resolveCompatibleModelSettings(
input: ModelSettingsCompatibilityInput,
): ModelSettingsCompatibilityResult {
const modelFamily = detectModelFamily(input.providerID, input.modelID)
const changes: ModelSettingsCompatibilityChange[] = []
let variant = input.desired.variant
if (variant !== undefined) {
const normalizedVariant = variant.toLowerCase()
const resolved = resolveVariant(modelFamily, normalizedVariant, input.capabilities)
if (resolved.value !== normalizedVariant && resolved.reason) {
changes.push({
field: "variant",
from: variant,
to: resolved.value,
reason: resolved.reason,
})
}
variant = resolved.value
}
let reasoningEffort = input.desired.reasoningEffort
if (reasoningEffort !== undefined) {
const normalizedReasoningEffort = reasoningEffort.toLowerCase()
const resolved = resolveReasoningEffort(modelFamily, normalizedReasoningEffort)
if (resolved.value !== normalizedReasoningEffort && resolved.reason) {
changes.push({
field: "reasoningEffort",
from: reasoningEffort,
to: resolved.value,
reason: resolved.reason,
})
}
reasoningEffort = resolved.value
}
return {
variant,
reasoningEffort,
changes,
}
}

View File

@@ -0,0 +1,31 @@
import { clearSessionPromptParams, setSessionPromptParams } from "./session-prompt-params-state"
type PromptParamModel = {
temperature?: number
top_p?: number
reasoningEffort?: string
maxTokens?: number
thinking?: { type: "enabled" | "disabled"; budgetTokens?: number }
}
export function applySessionPromptParams(
sessionID: string,
model: PromptParamModel | undefined,
): void {
if (!model) {
clearSessionPromptParams(sessionID)
return
}
const promptOptions: Record<string, unknown> = {
...(model.reasoningEffort ? { reasoningEffort: model.reasoningEffort } : {}),
...(model.thinking ? { thinking: model.thinking } : {}),
...(model.maxTokens !== undefined ? { maxTokens: model.maxTokens } : {}),
}
setSessionPromptParams(sessionID, {
...(model.temperature !== undefined ? { temperature: model.temperature } : {}),
...(model.top_p !== undefined ? { topP: model.top_p } : {}),
...(Object.keys(promptOptions).length > 0 ? { options: promptOptions } : {}),
})
}