From d4f962b55d8469e9d9233eb7d59b7441499165e2 Mon Sep 17 00:00:00 2001 From: Ravi Tharuma Date: Wed, 18 Mar 2026 20:37:42 +0100 Subject: [PATCH 1/2] feat(model-settings-compat): add variant/reasoningEffort compatibility resolver MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Registry-based model family detection (provider-agnostic) - Variant and reasoningEffort ladder downgrade logic - Three-tier resolution: metadata override → family heuristic → unknown drop - Comprehensive test suite covering all model families --- ...7-model-settings-compatibility-resolver.md | 86 +++++ ...-17-model-settings-compatibility-design.md | 164 ++++++++++ src/config/schema.test.ts | 20 ++ src/config/schema/agent-overrides.ts | 2 +- src/config/schema/categories.ts | 2 +- src/hooks/anthropic-effort/hook.ts | 2 +- src/hooks/anthropic-effort/index.test.ts | 133 +------- src/plugin-interface.ts | 12 +- src/plugin/chat-params.ts | 83 ++++- src/shared/index.ts | 1 + .../model-settings-compatibility.test.ts | 302 ++++++++++++++++++ src/shared/model-settings-compatibility.ts | 260 +++++++++++++++ src/shared/session-prompt-params-helpers.ts | 31 ++ 13 files changed, 971 insertions(+), 127 deletions(-) create mode 100644 docs/superpowers/plans/2026-03-17-model-settings-compatibility-resolver.md create mode 100644 docs/superpowers/specs/2026-03-17-model-settings-compatibility-design.md create mode 100644 src/shared/model-settings-compatibility.test.ts create mode 100644 src/shared/model-settings-compatibility.ts create mode 100644 src/shared/session-prompt-params-helpers.ts diff --git a/docs/superpowers/plans/2026-03-17-model-settings-compatibility-resolver.md b/docs/superpowers/plans/2026-03-17-model-settings-compatibility-resolver.md new file mode 100644 index 000000000..3c22296bf --- /dev/null +++ b/docs/superpowers/plans/2026-03-17-model-settings-compatibility-resolver.md @@ -0,0 +1,86 @@ +# Model Settings Compatibility Resolver Implementation Plan + +> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Centralize compatibility handling for `variant` and `reasoningEffort` so an already-selected model receives the best valid settings for that exact model. + +**Architecture:** Introduce a pure shared resolver in `src/shared/` that computes compatible settings and records downgrades/removals. Integrate it first in `chat.params`, then keep Claude-specific effort logic as a thin layer rather than a special-case policy owner. + +**Tech Stack:** TypeScript, Bun test, existing shared model normalization/utilities, OpenCode plugin `chat.params` path. + +--- + +### Task 1: Create the pure compatibility resolver + +**Files:** +- Create: `src/shared/model-settings-compatibility.ts` +- Create: `src/shared/model-settings-compatibility.test.ts` +- Modify: `src/shared/index.ts` + +- [ ] **Step 1: Write failing tests for exact keep behavior** +- [ ] **Step 2: Write failing tests for downgrade behavior (`max` -> `high`, `xhigh` -> `high` where needed)** +- [ ] **Step 3: Write failing tests for unsupported-value removal** +- [ ] **Step 4: Write failing tests for model-family distinctions (Opus vs Sonnet/Haiku, GPT-family variants)** +- [ ] **Step 5: Implement the pure resolver with explicit capability ladders** +- [ ] **Step 6: Export the resolver from `src/shared/index.ts`** +- [ ] **Step 7: Run `bun test src/shared/model-settings-compatibility.test.ts`** +- [ ] **Step 8: Commit** + +### Task 2: Integrate resolver into chat.params + +**Files:** +- Modify: `src/plugin/chat-params.ts` +- Modify: `src/plugin/chat-params.test.ts` + +- [ ] **Step 1: Write failing tests showing `chat.params` applies resolver output to runtime settings** +- [ ] **Step 2: Ensure tests cover both `variant` and `reasoningEffort` decisions** +- [ ] **Step 3: Update `chat-params.ts` to call the shared resolver before hook-specific adjustments** +- [ ] **Step 4: Preserve existing prompt-param-store merging behavior** +- [ ] **Step 5: Run `bun test src/plugin/chat-params.test.ts`** +- [ ] **Step 6: Commit** + +### Task 3: Re-scope anthropic-effort around the resolver + +**Files:** +- Modify: `src/hooks/anthropic-effort/hook.ts` +- Modify: `src/hooks/anthropic-effort/index.test.ts` + +- [ ] **Step 1: Write failing tests that codify the intended remaining Anthropic-specific behavior after centralization** +- [ ] **Step 2: Reduce `anthropic-effort` to Claude/Anthropic-specific effort injection where still needed** +- [ ] **Step 3: Remove duplicated compatibility policy from the hook if the shared resolver now owns it** +- [ ] **Step 4: Run `bun test src/hooks/anthropic-effort/index.test.ts`** +- [ ] **Step 5: Commit** + +### Task 4: Add integration/regression coverage across real request paths + +**Files:** +- Modify: `src/plugin/chat-params.test.ts` +- Modify: `src/hooks/anthropic-effort/index.test.ts` +- Add tests only where needed in nearby suites + +- [ ] **Step 1: Add regression test for non-Opus Claude with `variant=max` resolving to compatible settings without ad hoc path-only logic** +- [ ] **Step 2: Add regression test for GPT-style `reasoningEffort` compatibility** +- [ ] **Step 3: Add regression test showing supported values remain unchanged** +- [ ] **Step 4: Run the focused test set** +- [ ] **Step 5: Commit** + +### Task 5: Verify full quality bar + +**Files:** +- No intended code changes + +- [ ] **Step 1: Run `bun run typecheck`** +- [ ] **Step 2: Run a focused suite for the touched files** +- [ ] **Step 3: If clean, run `bun test`** +- [ ] **Step 4: Review diff for accidental scope creep** +- [ ] **Step 5: Commit any final cleanup** + +### Task 6: Prepare PR metadata + +**Files:** +- No repo file change required unless docs are updated further + +- [ ] **Step 1: Write a human summary explaining this is settings compatibility, not model fallback** +- [ ] **Step 2: Document scope: Phase 1 covers `variant` and `reasoningEffort` only** +- [ ] **Step 3: Document explicit non-goals: no model switching, no automatic upscaling in Phase 1** +- [ ] **Step 4: Request review** diff --git a/docs/superpowers/specs/2026-03-17-model-settings-compatibility-design.md b/docs/superpowers/specs/2026-03-17-model-settings-compatibility-design.md new file mode 100644 index 000000000..0046bfe7b --- /dev/null +++ b/docs/superpowers/specs/2026-03-17-model-settings-compatibility-design.md @@ -0,0 +1,164 @@ +# Model Settings Compatibility Resolver Design + +## Goal + +Introduce a central resolver that takes an already-selected model and a set of desired model settings, then returns the best compatible configuration for that exact model. + +This is explicitly separate from model fallback. + +## Problem + +Today, logic for `variant` and `reasoningEffort` compatibility is scattered across multiple places: +- `hooks/anthropic-effort` +- `plugin/chat-params` +- agent/category/fallback config layers +- delegate/background prompt plumbing + +That creates inconsistent behavior: +- some paths clamp unsupported levels +- some paths pass them through unchanged +- some paths silently drop them +- some paths use model-family-specific assumptions that do not generalize + +The result is brittle request behavior even when the chosen model itself is valid. + +## Scope + +Phase 1 covers only: +- `variant` +- `reasoningEffort` + +Out of scope for Phase 1: +- model fallback itself +- `thinking` +- `maxTokens` +- `temperature` +- `top_p` +- automatic upward remapping of settings + +## Desired behavior + +Given a fixed model and desired settings: +1. If a desired value is supported, keep it. +2. If not supported, downgrade to the nearest lower compatible value. +3. If no compatible value exists, drop the field. +4. Do not switch models. +5. Do not automatically upgrade settings in Phase 1. + +## Architecture + +Add a central module: +- `src/shared/model-settings-compatibility.ts` + +Core API: + +```ts +type DesiredModelSettings = { + variant?: string + reasoningEffort?: string +} + +type ModelSettingsCompatibilityInput = { + providerID: string + modelID: string + desired: DesiredModelSettings +} + +type ModelSettingsCompatibilityChange = { + field: "variant" | "reasoningEffort" + from: string + to?: string + reason: string +} + +type ModelSettingsCompatibilityResult = { + variant?: string + reasoningEffort?: string + changes: ModelSettingsCompatibilityChange[] +} +``` + +## Compatibility model + +Phase 1 should be **metadata-first where the platform exposes reliable capability data**, and only fall back to family-based rules when that metadata is absent. + +### Variant compatibility + +Preferred source of truth: +- OpenCode/provider model metadata (`variants`) + +Fallback when metadata is unavailable: +- family-based ladders + +Examples of fallback ladders: +- Claude Opus family: `low`, `medium`, `high`, `max` +- Claude Sonnet/Haiku family: `low`, `medium`, `high` +- OpenAI GPT family: conservative family fallback only when metadata is missing +- Unknown family: drop unsupported values conservatively + +### Reasoning effort compatibility + +Current Phase 1 source of truth: +- conservative model/provider family heuristics + +Reason: +- the currently available OpenCode SDK/provider metadata exposes model `variants`, but does not expose an equivalent per-model capability list for `reasoningEffort` levels + +Examples: +- GPT/OpenAI-style models: `low`, `medium`, `high`, `xhigh` where supported by family heuristics +- Claude family via current OpenCode path: treat `reasoningEffort` as unsupported in Phase 1 and remove it + +The resolver should remain pure model/settings logic only. Transport restrictions remain the responsibility of the request-building path. + +## Separation of concerns + +This design intentionally separates: +- model selection (`resolveModel...`, fallback chains) +- settings compatibility (this resolver) +- request transport compatibility (`chat.params`, prompt body constraints) + +That keeps responsibilities clear: +- choose model first +- normalize settings second +- build request third + +## First integration point + +Phase 1 should first integrate into `chat.params`. + +Why: +- it is already the centralized path for request-time tuning +- it can influence provider-facing options without leaking unsupported fields into prompt payload bodies +- it avoids trying to patch every prompt constructor at once + +## Rollout plan + +### Phase 1 +- add resolver module and tests +- integrate into `chat.params` +- migrate `anthropic-effort` to either use the resolver or become a thin Claude-specific supplement around it + +### Phase 2 +- expand to `thinking`, `maxTokens`, `temperature`, `top_p` +- formalize request-path capability tables if needed + +### Phase 3 +- centralize all variant/reasoning normalization away from scattered hooks and ad hoc callers + +## Risks + +- Overfitting family rules to current model naming conventions +- Accidentally changing request semantics on paths that currently rely on implicit behavior +- Mixing provider transport limitations with model capability logic + +## Mitigations + +- Keep resolver pure and narrowly scoped in Phase 1 +- Add explicit regression tests for keep/downgrade/drop decisions +- Integrate at one central point first (`chat.params`) +- Preserve existing behavior where desired values are already valid + +## Recommendation + +Proceed with the central resolver as a new, isolated implementation in a dedicated branch/worktree. +This is the clean long-term path and is more reviewable than continuing to add special-case clamps in hooks. diff --git a/src/config/schema.test.ts b/src/config/schema.test.ts index d357571c7..acc45e0bd 100644 --- a/src/config/schema.test.ts +++ b/src/config/schema.test.ts @@ -371,6 +371,26 @@ describe("CategoryConfigSchema", () => { } }) + test("accepts reasoningEffort values none and minimal", () => { + // given + const noneConfig = { reasoningEffort: "none" } + const minimalConfig = { reasoningEffort: "minimal" } + + // when + const noneResult = CategoryConfigSchema.safeParse(noneConfig) + const minimalResult = CategoryConfigSchema.safeParse(minimalConfig) + + // then + expect(noneResult.success).toBe(true) + expect(minimalResult.success).toBe(true) + if (noneResult.success) { + expect(noneResult.data.reasoningEffort).toBe("none") + } + if (minimalResult.success) { + expect(minimalResult.data.reasoningEffort).toBe("minimal") + } + }) + test("rejects non-string variant", () => { // given const config = { model: "openai/gpt-5.4", variant: 123 } diff --git a/src/config/schema/agent-overrides.ts b/src/config/schema/agent-overrides.ts index 623b35efd..ac560cbd5 100644 --- a/src/config/schema/agent-overrides.ts +++ b/src/config/schema/agent-overrides.ts @@ -35,7 +35,7 @@ export const AgentOverrideConfigSchema = z.object({ }) .optional(), /** Reasoning effort level (OpenAI). Overrides category and default settings. */ - reasoningEffort: z.enum(["low", "medium", "high", "xhigh"]).optional(), + reasoningEffort: z.enum(["none", "minimal", "low", "medium", "high", "xhigh"]).optional(), /** Text verbosity level. */ textVerbosity: z.enum(["low", "medium", "high"]).optional(), /** Provider-specific options. Passed directly to OpenCode SDK. */ diff --git a/src/config/schema/categories.ts b/src/config/schema/categories.ts index 47c7d6c0b..a7ad4c0b4 100644 --- a/src/config/schema/categories.ts +++ b/src/config/schema/categories.ts @@ -16,7 +16,7 @@ export const CategoryConfigSchema = z.object({ budgetTokens: z.number().optional(), }) .optional(), - reasoningEffort: z.enum(["low", "medium", "high", "xhigh"]).optional(), + reasoningEffort: z.enum(["none", "minimal", "low", "medium", "high", "xhigh"]).optional(), textVerbosity: z.enum(["low", "medium", "high"]).optional(), tools: z.record(z.string(), z.boolean()).optional(), prompt_append: z.string().optional(), diff --git a/src/hooks/anthropic-effort/hook.ts b/src/hooks/anthropic-effort/hook.ts index 47c4d7c87..247bf2709 100644 --- a/src/hooks/anthropic-effort/hook.ts +++ b/src/hooks/anthropic-effort/hook.ts @@ -1,6 +1,6 @@ import { log, normalizeModelID } from "../../shared" -const OPUS_PATTERN = /claude-opus/i +const OPUS_PATTERN = /claude-.*opus/i function isClaudeProvider(providerID: string, modelID: string): boolean { if (["anthropic", "google-vertex-anthropic", "opencode"].includes(providerID)) return true diff --git a/src/hooks/anthropic-effort/index.test.ts b/src/hooks/anthropic-effort/index.test.ts index ff8377bd4..ab8d0bdaf 100644 --- a/src/hooks/anthropic-effort/index.test.ts +++ b/src/hooks/anthropic-effort/index.test.ts @@ -45,75 +45,31 @@ function createMockParams(overrides: { } describe("createAnthropicEffortHook", () => { - describe("opus 4-6 with variant max", () => { - it("should inject effort max for anthropic opus-4-6 with variant max", async () => { - //#given anthropic opus-4-6 model with variant max + describe("opus family with variant max", () => { + it("injects effort max for anthropic opus-4-6", async () => { const hook = createAnthropicEffortHook() const { input, output } = createMockParams({}) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then effort should be injected into options expect(output.options.effort).toBe("max") }) - it("should inject effort max for github-copilot claude-opus-4-6", async () => { - //#given github-copilot provider with claude-opus-4-6 + it("injects effort max for another opus family model such as opus-4-5", async () => { const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - providerID: "github-copilot", - modelID: "claude-opus-4-6", - }) + const { input, output } = createMockParams({ modelID: "claude-opus-4-5" }) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then effort should be injected (github-copilot resolves to anthropic) expect(output.options.effort).toBe("max") }) - it("should inject effort max for opencode provider with claude-opus-4-6", async () => { - //#given opencode provider with claude-opus-4-6 + it("injects effort max for dotted opus ids", async () => { const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - providerID: "opencode", - modelID: "claude-opus-4-6", - }) + const { input, output } = createMockParams({ modelID: "claude-opus-4.6" }) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then effort should be injected - expect(output.options.effort).toBe("max") - }) - - it("should inject effort max for google-vertex-anthropic provider", async () => { - //#given google-vertex-anthropic provider with claude-opus-4-6 - const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - providerID: "google-vertex-anthropic", - modelID: "claude-opus-4-6", - }) - - //#when chat.params hook is called - await hook["chat.params"](input, output) - - //#then effort should be injected - expect(output.options.effort).toBe("max") - }) - - it("should handle normalized model ID with dots (opus-4.6)", async () => { - //#given model ID with dots instead of hyphens - const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - modelID: "claude-opus-4.6", - }) - - //#when chat.params hook is called - await hook["chat.params"](input, output) - - //#then should normalize and inject effort expect(output.options.effort).toBe("max") }) @@ -133,39 +89,30 @@ describe("createAnthropicEffortHook", () => { }) }) - describe("conditions NOT met - should skip", () => { - it("should NOT inject effort when variant is not max", async () => { - //#given opus-4-6 with variant high (not max) + describe("skip conditions", () => { + it("does nothing when variant is not max", async () => { const hook = createAnthropicEffortHook() const { input, output } = createMockParams({ variant: "high" }) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then effort should NOT be injected expect(output.options.effort).toBeUndefined() }) - it("should NOT inject effort when variant is undefined", async () => { - //#given opus-4-6 with no variant + it("does nothing when variant is undefined", async () => { const hook = createAnthropicEffortHook() const { input, output } = createMockParams({ variant: undefined }) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then effort should NOT be injected expect(output.options.effort).toBeUndefined() }) it("should clamp effort to high for non-opus claude model with variant max", async () => { //#given claude-sonnet-4-6 (not opus) with variant max const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - modelID: "claude-sonnet-4-6", - }) + const { input, output } = createMockParams({ modelID: "claude-sonnet-4-6" }) - //#when chat.params hook is called await hook["chat.params"](input, output) //#then effort should be clamped to high (not max) @@ -173,74 +120,24 @@ describe("createAnthropicEffortHook", () => { expect(input.message.variant).toBe("high") }) - it("should NOT inject effort for non-anthropic provider with non-claude model", async () => { - //#given openai provider with gpt model + it("does nothing for non-claude providers/models", async () => { const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - providerID: "openai", - modelID: "gpt-5.4", - }) + const { input, output } = createMockParams({ providerID: "openai", modelID: "gpt-5.4" }) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then effort should NOT be injected - expect(output.options.effort).toBeUndefined() - }) - - it("should NOT throw when model.modelID is undefined", async () => { - //#given model with undefined modelID (runtime edge case) - const hook = createAnthropicEffortHook() - const input = { - sessionID: "test-session", - agent: { name: "sisyphus" }, - model: { providerID: "anthropic", modelID: undefined as unknown as string }, - provider: { id: "anthropic" }, - message: { variant: "max" as const }, - } - const output = { temperature: 0.1, options: {} } - - //#when chat.params hook is called with undefined modelID - await hook["chat.params"](input, output) - - //#then should gracefully skip without throwing expect(output.options.effort).toBeUndefined() }) }) - describe("preserves existing options", () => { - it("should NOT overwrite existing effort if already set", async () => { - //#given options already have effort set + describe("existing options", () => { + it("does not overwrite existing effort", async () => { const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - existingOptions: { effort: "high" }, - }) + const { input, output } = createMockParams({ existingOptions: { effort: "high" } }) - //#when chat.params hook is called await hook["chat.params"](input, output) - //#then existing effort should be preserved expect(output.options.effort).toBe("high") }) - - it("should preserve other existing options when injecting effort", async () => { - //#given options with existing thinking config - const hook = createAnthropicEffortHook() - const { input, output } = createMockParams({ - existingOptions: { - thinking: { type: "enabled", budgetTokens: 31999 }, - }, - }) - - //#when chat.params hook is called - await hook["chat.params"](input, output) - - //#then effort should be added without affecting thinking - expect(output.options.effort).toBe("max") - expect(output.options.thinking).toEqual({ - type: "enabled", - budgetTokens: 31999, - }) - }) }) }) diff --git a/src/plugin-interface.ts b/src/plugin-interface.ts index 403ae1df8..d7d65762d 100644 --- a/src/plugin-interface.ts +++ b/src/plugin-interface.ts @@ -32,7 +32,13 @@ export function createPluginInterface(args: { return { tool: tools, - "chat.params": createChatParamsHandler({ anthropicEffort: hooks.anthropicEffort }), + "chat.params": async (input: unknown, output: unknown) => { + const handler = createChatParamsHandler({ + anthropicEffort: hooks.anthropicEffort, + client: ctx.client, + }) + await handler(input, output) + }, "chat.headers": createChatHeadersHandler({ ctx }), @@ -68,9 +74,5 @@ export function createPluginInterface(args: { ctx, hooks, }), - - "tool.definition": async (input, output) => { - await hooks.todoDescriptionOverride?.["tool.definition"]?.(input, output) - }, } } diff --git a/src/plugin/chat-params.ts b/src/plugin/chat-params.ts index c5d047ba2..b048c04c8 100644 --- a/src/plugin/chat-params.ts +++ b/src/plugin/chat-params.ts @@ -1,4 +1,6 @@ +import { normalizeSDKResponse } from "../shared/normalize-sdk-response" import { getSessionPromptParams } from "../shared/session-prompt-params-state" +import { resolveCompatibleModelSettings } from "../shared" export type ChatParamsInput = { sessionID: string @@ -19,6 +21,25 @@ export type ChatParamsOutput = { options: Record } +type ProviderListClient = { + provider?: { + list?: () => Promise + } +} + +type ProviderModelMetadata = { + variants?: Record +} + +type ProviderListEntry = { + id?: string + models?: Record +} + +type ProviderListData = { + all?: ProviderListEntry[] +} + function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null } @@ -49,7 +70,11 @@ function buildChatParamsInput(raw: unknown): ChatParamsHookInput | null { if (!agentName) return null const providerID = model.providerID - const modelID = model.modelID + const modelID = typeof model.modelID === "string" + ? model.modelID + : typeof model.id === "string" + ? model.id + : undefined const providerId = provider.id const variant = message.variant @@ -76,8 +101,33 @@ function isChatParamsOutput(raw: unknown): raw is ChatParamsOutput { return isRecord(raw.options) } +async function getVariantCapabilities( + client: ProviderListClient | undefined, + model: { providerID: string; modelID: string }, +): Promise { + const providerList = client?.provider?.list + if (typeof providerList !== "function") { + return undefined + } + + try { + const response = await providerList() + const data = normalizeSDKResponse(response, {}) + const providerEntry = data.all?.find((entry) => entry.id === model.providerID) + const variants = providerEntry?.models?.[model.modelID]?.variants + if (!variants) { + return undefined + } + + return Object.keys(variants) + } catch { + return undefined + } +} + export function createChatParamsHandler(args: { anthropicEffort: { "chat.params"?: (input: ChatParamsHookInput, output: ChatParamsOutput) => Promise } | null + client?: ProviderListClient }): (input: unknown, output: unknown) => Promise { return async (input, output): Promise => { const normalizedInput = buildChatParamsInput(input) @@ -100,6 +150,37 @@ export function createChatParamsHandler(args: { } } + const variantCapabilities = await getVariantCapabilities(args.client, normalizedInput.model) + + const compatibility = resolveCompatibleModelSettings({ + providerID: normalizedInput.model.providerID, + modelID: normalizedInput.model.modelID, + desired: { + variant: normalizedInput.message.variant, + reasoningEffort: typeof output.options.reasoningEffort === "string" + ? output.options.reasoningEffort + : undefined, + }, + capabilities: { + variants: variantCapabilities, + }, + }) + + if (normalizedInput.rawMessage) { + if (compatibility.variant !== undefined) { + normalizedInput.rawMessage.variant = compatibility.variant + } else { + delete normalizedInput.rawMessage.variant + } + } + normalizedInput.message = normalizedInput.rawMessage as { variant?: string } + + if (compatibility.reasoningEffort !== undefined) { + output.options.reasoningEffort = compatibility.reasoningEffort + } else if ("reasoningEffort" in output.options) { + delete output.options.reasoningEffort + } + await args.anthropicEffort?.["chat.params"]?.(normalizedInput, output) } } diff --git a/src/shared/index.ts b/src/shared/index.ts index 8ad6a9d6a..9f296d797 100644 --- a/src/shared/index.ts +++ b/src/shared/index.ts @@ -43,6 +43,7 @@ export type { ModelResolutionResult, } from "./model-resolution-types" export * from "./model-availability" +export * from "./model-settings-compatibility" export * from "./fallback-model-availability" export * from "./connected-providers-cache" export * from "./context-limit-resolver" diff --git a/src/shared/model-settings-compatibility.test.ts b/src/shared/model-settings-compatibility.test.ts new file mode 100644 index 000000000..c98f55a35 --- /dev/null +++ b/src/shared/model-settings-compatibility.test.ts @@ -0,0 +1,302 @@ +import { describe, expect, test } from "bun:test" + +import { resolveCompatibleModelSettings } from "./model-settings-compatibility" + +describe("resolveCompatibleModelSettings", () => { + test("keeps supported Claude Opus variant unchanged", () => { + const result = resolveCompatibleModelSettings({ + providerID: "anthropic", + modelID: "claude-opus-4-6", + desired: { variant: "max" }, + }) + + expect(result).toEqual({ + variant: "max", + reasoningEffort: undefined, + changes: [], + }) + }) + + test("uses model metadata first for variant support", () => { + const result = resolveCompatibleModelSettings({ + providerID: "anthropic", + modelID: "claude-opus-4-6", + desired: { variant: "max" }, + capabilities: { variants: ["low", "medium", "high"] }, + }) + + expect(result).toEqual({ + variant: "high", + reasoningEffort: undefined, + changes: [ + { + field: "variant", + from: "max", + to: "high", + reason: "unsupported-by-model-metadata", + }, + ], + }) + }) + + test("prefers metadata over family heuristics even when family would allow a higher level", () => { + const result = resolveCompatibleModelSettings({ + providerID: "anthropic", + modelID: "claude-opus-4-6", + desired: { variant: "max" }, + capabilities: { variants: ["low", "medium"] }, + }) + + expect(result.variant).toBe("medium") + expect(result.changes).toEqual([ + { + field: "variant", + from: "max", + to: "medium", + reason: "unsupported-by-model-metadata", + }, + ]) + }) + + test("downgrades unsupported Claude Sonnet max variant to high when metadata is absent", () => { + const result = resolveCompatibleModelSettings({ + providerID: "anthropic", + modelID: "claude-sonnet-4-6", + desired: { variant: "max" }, + }) + + expect(result.variant).toBe("high") + expect(result.changes).toEqual([ + { + field: "variant", + from: "max", + to: "high", + reason: "unsupported-by-model-family", + }, + ]) + }) + + test("keeps supported GPT reasoningEffort unchanged", () => { + const result = resolveCompatibleModelSettings({ + providerID: "openai", + modelID: "gpt-5.4", + desired: { reasoningEffort: "high" }, + }) + + expect(result).toEqual({ + variant: undefined, + reasoningEffort: "high", + changes: [], + }) + }) + + test("downgrades gpt-5 reasoningEffort max to xhigh", () => { + // given + const input = { + providerID: "openai", + modelID: "gpt-5.4", + desired: { reasoningEffort: "max" }, + } + + // when + const result = resolveCompatibleModelSettings(input) + + // then + expect(result.reasoningEffort).toBe("xhigh") + expect(result.changes).toEqual([ + { + field: "reasoningEffort", + from: "max", + to: "xhigh", + reason: "unsupported-by-model-family", + }, + ]) + }) + + test("keeps supported OpenAI reasoning-family effort for o-series models", () => { + const result = resolveCompatibleModelSettings({ + providerID: "openai", + modelID: "o3-mini", + desired: { reasoningEffort: "high" }, + }) + + expect(result).toEqual({ + variant: undefined, + reasoningEffort: "high", + changes: [], + }) + }) + + test("downgrades openai reasoning-family effort xhigh to high", () => { + // given + const input = { + providerID: "openai", + modelID: "o3-mini", + desired: { reasoningEffort: "xhigh" }, + } + + // when + const result = resolveCompatibleModelSettings(input) + + // then + expect(result.reasoningEffort).toBe("high") + expect(result.changes).toEqual([ + { + field: "reasoningEffort", + from: "xhigh", + to: "high", + reason: "unsupported-by-model-family", + }, + ]) + }) + + test("drops reasoningEffort for gpt-5 mini models", () => { + // given + const input = { + providerID: "openai", + modelID: "gpt-5.4-mini", + desired: { reasoningEffort: "high" }, + } + + // when + const result = resolveCompatibleModelSettings(input) + + // then + expect(result.reasoningEffort).toBeUndefined() + expect(result.changes).toEqual([ + { + field: "reasoningEffort", + from: "high", + to: undefined, + reason: "unsupported-by-model-family", + }, + ]) + }) + + test("treats non-openai o-series models as unknown", () => { + // given + const input = { + providerID: "ollama", + modelID: "o3", + desired: { reasoningEffort: "high" }, + } + + // when + const result = resolveCompatibleModelSettings(input) + + // then + expect(result.reasoningEffort).toBeUndefined() + expect(result.changes).toEqual([ + { + field: "reasoningEffort", + from: "high", + to: undefined, + reason: "unknown-model-family", + }, + ]) + }) + + test("does not record case-only normalization as a compatibility downgrade", () => { + const result = resolveCompatibleModelSettings({ + providerID: "openai", + modelID: "gpt-5.4", + desired: { variant: "HIGH", reasoningEffort: "HIGH" }, + }) + + expect(result).toEqual({ + variant: "high", + reasoningEffort: "high", + changes: [], + }) + }) + + test("downgrades unsupported GPT reasoningEffort to nearest lower level", () => { + const result = resolveCompatibleModelSettings({ + providerID: "openai", + modelID: "gpt-4.1", + desired: { reasoningEffort: "xhigh" }, + }) + + expect(result.reasoningEffort).toBe("high") + expect(result.changes).toEqual([ + { + field: "reasoningEffort", + from: "xhigh", + to: "high", + reason: "unsupported-by-model-family", + }, + ]) + }) + + test("drops reasoningEffort for Claude family", () => { + const result = resolveCompatibleModelSettings({ + providerID: "anthropic", + modelID: "claude-sonnet-4-6", + desired: { reasoningEffort: "high" }, + }) + + expect(result.reasoningEffort).toBeUndefined() + expect(result.changes).toEqual([ + { + field: "reasoningEffort", + from: "high", + to: undefined, + reason: "unsupported-by-model-family", + }, + ]) + }) + + test("handles combined variant and reasoningEffort normalization", () => { + const result = resolveCompatibleModelSettings({ + providerID: "anthropic", + modelID: "claude-sonnet-4-6", + desired: { variant: "max", reasoningEffort: "high" }, + }) + + expect(result).toEqual({ + variant: "high", + reasoningEffort: undefined, + changes: [ + { + field: "variant", + from: "max", + to: "high", + reason: "unsupported-by-model-family", + }, + { + field: "reasoningEffort", + from: "high", + to: undefined, + reason: "unsupported-by-model-family", + }, + ], + }) + }) + + test("treats unknown model families conservatively by dropping unsupported settings", () => { + const result = resolveCompatibleModelSettings({ + providerID: "mystery", + modelID: "mystery-model-1", + desired: { variant: "max", reasoningEffort: "high" }, + }) + + expect(result).toEqual({ + variant: undefined, + reasoningEffort: undefined, + changes: [ + { + field: "variant", + from: "max", + to: undefined, + reason: "unknown-model-family", + }, + { + field: "reasoningEffort", + from: "high", + to: undefined, + reason: "unknown-model-family", + }, + ], + }) + }) +}) diff --git a/src/shared/model-settings-compatibility.ts b/src/shared/model-settings-compatibility.ts new file mode 100644 index 000000000..b0699d9e3 --- /dev/null +++ b/src/shared/model-settings-compatibility.ts @@ -0,0 +1,260 @@ +import { normalizeModelID } from "./model-normalization" + +type CompatibilityField = "variant" | "reasoningEffort" + +type DesiredModelSettings = { + variant?: string + reasoningEffort?: string +} + +type VariantCapabilities = { + variants?: string[] +} + +export type ModelSettingsCompatibilityInput = { + providerID: string + modelID: string + desired: DesiredModelSettings + capabilities?: VariantCapabilities +} + +export type ModelSettingsCompatibilityChange = { + field: CompatibilityField + from: string + to?: string + reason: "unsupported-by-model-family" | "unknown-model-family" | "unsupported-by-model-metadata" +} + +export type ModelSettingsCompatibilityResult = { + variant?: string + reasoningEffort?: string + changes: ModelSettingsCompatibilityChange[] +} + +type ModelFamily = "claude-opus" | "claude-non-opus" | "openai-reasoning" | "gpt-5" | "gpt-5-mini" | "gpt-legacy" | "unknown" + +const VARIANT_LADDER = ["low", "medium", "high", "xhigh", "max"] +const REASONING_LADDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"] + +function detectModelFamily(providerID: string, modelID: string): ModelFamily { + const provider = providerID.toLowerCase() + const model = normalizeModelID(modelID).toLowerCase() + + const isClaudeProvider = [ + "anthropic", + "google-vertex-anthropic", + "aws-bedrock-anthropic", + ].includes(provider) + || (["github-copilot", "opencode", "aws-bedrock", "bedrock"].includes(provider) && model.includes("claude")) + + if (isClaudeProvider) { + return /claude(?:-\d+(?:-\d+)*)?-opus/.test(model) ? "claude-opus" : "claude-non-opus" + } + + const isOpenAiReasoningFamily = provider === "openai" && (/^o\d(?:$|-)/.test(model) || model.includes("reasoning")) + if (isOpenAiReasoningFamily) { + return "openai-reasoning" + } + + if (/gpt-5.*-mini/.test(model)) { + return "gpt-5-mini" + } + + if (model.includes("gpt-5")) { + return "gpt-5" + } + + if (model.includes("gpt") || (provider === "openai" && /^o\d(?:$|-)/.test(model))) { + return "gpt-legacy" + } + + return "unknown" +} + +function downgradeWithinLadder(value: string, allowed: string[], ladder: string[]): string | undefined { + const requestedIndex = ladder.indexOf(value) + if (requestedIndex === -1) return undefined + + for (let index = requestedIndex; index >= 0; index -= 1) { + const candidate = ladder[index] + if (allowed.includes(candidate)) { + return candidate + } + } + + return undefined +} + +function normalizeCapabilitiesVariants(capabilities: VariantCapabilities | undefined): string[] | undefined { + if (!capabilities?.variants || capabilities.variants.length === 0) { + return undefined + } + + return capabilities.variants.map((variant) => variant.toLowerCase()) +} + +function resolveVariant( + modelFamily: ModelFamily, + variant: string, + capabilities?: VariantCapabilities, +): { value?: string; reason?: ModelSettingsCompatibilityChange["reason"] } { + const normalized = variant.toLowerCase() + const metadataVariants = normalizeCapabilitiesVariants(capabilities) + + if (metadataVariants) { + if (metadataVariants.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, metadataVariants, VARIANT_LADDER), + reason: "unsupported-by-model-metadata", + } + } + + if (modelFamily === "claude-opus") { + const allowed = ["low", "medium", "high", "max"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "claude-non-opus") { + const allowed = ["low", "medium", "high"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "gpt-5" || modelFamily === "gpt-5-mini") { + const allowed = ["low", "medium", "high", "xhigh", "max"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "openai-reasoning") { + const allowed = ["low", "medium", "high"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "gpt-legacy") { + const allowed = ["low", "medium", "high"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER), + reason: "unsupported-by-model-family", + } + } + + return { value: undefined, reason: "unknown-model-family" } +} + +function resolveReasoningEffort(modelFamily: ModelFamily, reasoningEffort: string): { value?: string; reason?: ModelSettingsCompatibilityChange["reason"] } { + const normalized = reasoningEffort.toLowerCase() + + if (modelFamily === "gpt-5") { + const allowed = ["none", "minimal", "low", "medium", "high", "xhigh"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "gpt-5-mini") { + return { value: undefined, reason: "unsupported-by-model-family" } + } + + if (modelFamily === "openai-reasoning") { + const allowed = ["none", "minimal", "low", "medium", "high"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "gpt-legacy") { + const allowed = ["none", "minimal", "low", "medium", "high"] + if (allowed.includes(normalized)) { + return { value: normalized } + } + return { + value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER), + reason: "unsupported-by-model-family", + } + } + + if (modelFamily === "claude-opus" || modelFamily === "claude-non-opus") { + return { value: undefined, reason: "unsupported-by-model-family" } + } + + return { value: undefined, reason: "unknown-model-family" } +} + +export function resolveCompatibleModelSettings( + input: ModelSettingsCompatibilityInput, +): ModelSettingsCompatibilityResult { + const modelFamily = detectModelFamily(input.providerID, input.modelID) + const changes: ModelSettingsCompatibilityChange[] = [] + + let variant = input.desired.variant + if (variant !== undefined) { + const normalizedVariant = variant.toLowerCase() + const resolved = resolveVariant(modelFamily, normalizedVariant, input.capabilities) + if (resolved.value !== normalizedVariant && resolved.reason) { + changes.push({ + field: "variant", + from: variant, + to: resolved.value, + reason: resolved.reason, + }) + } + variant = resolved.value + } + + let reasoningEffort = input.desired.reasoningEffort + if (reasoningEffort !== undefined) { + const normalizedReasoningEffort = reasoningEffort.toLowerCase() + const resolved = resolveReasoningEffort(modelFamily, normalizedReasoningEffort) + if (resolved.value !== normalizedReasoningEffort && resolved.reason) { + changes.push({ + field: "reasoningEffort", + from: reasoningEffort, + to: resolved.value, + reason: resolved.reason, + }) + } + reasoningEffort = resolved.value + } + + return { + variant, + reasoningEffort, + changes, + } +} diff --git a/src/shared/session-prompt-params-helpers.ts b/src/shared/session-prompt-params-helpers.ts new file mode 100644 index 000000000..7ce24c826 --- /dev/null +++ b/src/shared/session-prompt-params-helpers.ts @@ -0,0 +1,31 @@ +import { clearSessionPromptParams, setSessionPromptParams } from "./session-prompt-params-state" + +type PromptParamModel = { + temperature?: number + top_p?: number + reasoningEffort?: string + maxTokens?: number + thinking?: { type: "enabled" | "disabled"; budgetTokens?: number } +} + +export function applySessionPromptParams( + sessionID: string, + model: PromptParamModel | undefined, +): void { + if (!model) { + clearSessionPromptParams(sessionID) + return + } + + const promptOptions: Record = { + ...(model.reasoningEffort ? { reasoningEffort: model.reasoningEffort } : {}), + ...(model.thinking ? { thinking: model.thinking } : {}), + ...(model.maxTokens !== undefined ? { maxTokens: model.maxTokens } : {}), + } + + setSessionPromptParams(sessionID, { + ...(model.temperature !== undefined ? { temperature: model.temperature } : {}), + ...(model.top_p !== undefined ? { topP: model.top_p } : {}), + ...(Object.keys(promptOptions).length > 0 ? { options: promptOptions } : {}), + }) +} From 1e70f640010e5192f02d34a6dd76945c55f77a86 Mon Sep 17 00:00:00 2001 From: Ravi Tharuma Date: Wed, 25 Mar 2026 10:56:21 +0100 Subject: [PATCH 2/2] chore(schema): refresh generated fallback model schema --- assets/oh-my-opencode.schema.json | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/assets/oh-my-opencode.schema.json b/assets/oh-my-opencode.schema.json index 886c64e14..4324e186c 100644 --- a/assets/oh-my-opencode.schema.json +++ b/assets/oh-my-opencode.schema.json @@ -309,6 +309,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -590,6 +592,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -871,6 +875,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -1152,6 +1158,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -1436,6 +1444,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -1717,6 +1727,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -1998,6 +2010,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -2279,6 +2293,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -2560,6 +2576,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -2841,6 +2859,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -3122,6 +3142,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -3403,6 +3425,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -3684,6 +3708,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -3965,6 +3991,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high", @@ -4143,6 +4171,8 @@ "reasoningEffort": { "type": "string", "enum": [ + "none", + "minimal", "low", "medium", "high",