feat(model-settings-compat): add variant/reasoningEffort compatibility resolver
- Registry-based model family detection (provider-agnostic) - Variant and reasoningEffort ladder downgrade logic - Three-tier resolution: metadata override → family heuristic → unknown drop - Comprehensive test suite covering all model families
This commit is contained in:
@@ -0,0 +1,86 @@
|
|||||||
|
# Model Settings Compatibility Resolver Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Centralize compatibility handling for `variant` and `reasoningEffort` so an already-selected model receives the best valid settings for that exact model.
|
||||||
|
|
||||||
|
**Architecture:** Introduce a pure shared resolver in `src/shared/` that computes compatible settings and records downgrades/removals. Integrate it first in `chat.params`, then keep Claude-specific effort logic as a thin layer rather than a special-case policy owner.
|
||||||
|
|
||||||
|
**Tech Stack:** TypeScript, Bun test, existing shared model normalization/utilities, OpenCode plugin `chat.params` path.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Create the pure compatibility resolver
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `src/shared/model-settings-compatibility.ts`
|
||||||
|
- Create: `src/shared/model-settings-compatibility.test.ts`
|
||||||
|
- Modify: `src/shared/index.ts`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing tests for exact keep behavior**
|
||||||
|
- [ ] **Step 2: Write failing tests for downgrade behavior (`max` -> `high`, `xhigh` -> `high` where needed)**
|
||||||
|
- [ ] **Step 3: Write failing tests for unsupported-value removal**
|
||||||
|
- [ ] **Step 4: Write failing tests for model-family distinctions (Opus vs Sonnet/Haiku, GPT-family variants)**
|
||||||
|
- [ ] **Step 5: Implement the pure resolver with explicit capability ladders**
|
||||||
|
- [ ] **Step 6: Export the resolver from `src/shared/index.ts`**
|
||||||
|
- [ ] **Step 7: Run `bun test src/shared/model-settings-compatibility.test.ts`**
|
||||||
|
- [ ] **Step 8: Commit**
|
||||||
|
|
||||||
|
### Task 2: Integrate resolver into chat.params
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/plugin/chat-params.ts`
|
||||||
|
- Modify: `src/plugin/chat-params.test.ts`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing tests showing `chat.params` applies resolver output to runtime settings**
|
||||||
|
- [ ] **Step 2: Ensure tests cover both `variant` and `reasoningEffort` decisions**
|
||||||
|
- [ ] **Step 3: Update `chat-params.ts` to call the shared resolver before hook-specific adjustments**
|
||||||
|
- [ ] **Step 4: Preserve existing prompt-param-store merging behavior**
|
||||||
|
- [ ] **Step 5: Run `bun test src/plugin/chat-params.test.ts`**
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
### Task 3: Re-scope anthropic-effort around the resolver
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/hooks/anthropic-effort/hook.ts`
|
||||||
|
- Modify: `src/hooks/anthropic-effort/index.test.ts`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing tests that codify the intended remaining Anthropic-specific behavior after centralization**
|
||||||
|
- [ ] **Step 2: Reduce `anthropic-effort` to Claude/Anthropic-specific effort injection where still needed**
|
||||||
|
- [ ] **Step 3: Remove duplicated compatibility policy from the hook if the shared resolver now owns it**
|
||||||
|
- [ ] **Step 4: Run `bun test src/hooks/anthropic-effort/index.test.ts`**
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
### Task 4: Add integration/regression coverage across real request paths
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/plugin/chat-params.test.ts`
|
||||||
|
- Modify: `src/hooks/anthropic-effort/index.test.ts`
|
||||||
|
- Add tests only where needed in nearby suites
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add regression test for non-Opus Claude with `variant=max` resolving to compatible settings without ad hoc path-only logic**
|
||||||
|
- [ ] **Step 2: Add regression test for GPT-style `reasoningEffort` compatibility**
|
||||||
|
- [ ] **Step 3: Add regression test showing supported values remain unchanged**
|
||||||
|
- [ ] **Step 4: Run the focused test set**
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
### Task 5: Verify full quality bar
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- No intended code changes
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run `bun run typecheck`**
|
||||||
|
- [ ] **Step 2: Run a focused suite for the touched files**
|
||||||
|
- [ ] **Step 3: If clean, run `bun test`**
|
||||||
|
- [ ] **Step 4: Review diff for accidental scope creep**
|
||||||
|
- [ ] **Step 5: Commit any final cleanup**
|
||||||
|
|
||||||
|
### Task 6: Prepare PR metadata
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- No repo file change required unless docs are updated further
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write a human summary explaining this is settings compatibility, not model fallback**
|
||||||
|
- [ ] **Step 2: Document scope: Phase 1 covers `variant` and `reasoningEffort` only**
|
||||||
|
- [ ] **Step 3: Document explicit non-goals: no model switching, no automatic upscaling in Phase 1**
|
||||||
|
- [ ] **Step 4: Request review**
|
||||||
@@ -0,0 +1,164 @@
|
|||||||
|
# Model Settings Compatibility Resolver Design
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Introduce a central resolver that takes an already-selected model and a set of desired model settings, then returns the best compatible configuration for that exact model.
|
||||||
|
|
||||||
|
This is explicitly separate from model fallback.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Today, logic for `variant` and `reasoningEffort` compatibility is scattered across multiple places:
|
||||||
|
- `hooks/anthropic-effort`
|
||||||
|
- `plugin/chat-params`
|
||||||
|
- agent/category/fallback config layers
|
||||||
|
- delegate/background prompt plumbing
|
||||||
|
|
||||||
|
That creates inconsistent behavior:
|
||||||
|
- some paths clamp unsupported levels
|
||||||
|
- some paths pass them through unchanged
|
||||||
|
- some paths silently drop them
|
||||||
|
- some paths use model-family-specific assumptions that do not generalize
|
||||||
|
|
||||||
|
The result is brittle request behavior even when the chosen model itself is valid.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
Phase 1 covers only:
|
||||||
|
- `variant`
|
||||||
|
- `reasoningEffort`
|
||||||
|
|
||||||
|
Out of scope for Phase 1:
|
||||||
|
- model fallback itself
|
||||||
|
- `thinking`
|
||||||
|
- `maxTokens`
|
||||||
|
- `temperature`
|
||||||
|
- `top_p`
|
||||||
|
- automatic upward remapping of settings
|
||||||
|
|
||||||
|
## Desired behavior
|
||||||
|
|
||||||
|
Given a fixed model and desired settings:
|
||||||
|
1. If a desired value is supported, keep it.
|
||||||
|
2. If not supported, downgrade to the nearest lower compatible value.
|
||||||
|
3. If no compatible value exists, drop the field.
|
||||||
|
4. Do not switch models.
|
||||||
|
5. Do not automatically upgrade settings in Phase 1.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Add a central module:
|
||||||
|
- `src/shared/model-settings-compatibility.ts`
|
||||||
|
|
||||||
|
Core API:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
type DesiredModelSettings = {
|
||||||
|
variant?: string
|
||||||
|
reasoningEffort?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
type ModelSettingsCompatibilityInput = {
|
||||||
|
providerID: string
|
||||||
|
modelID: string
|
||||||
|
desired: DesiredModelSettings
|
||||||
|
}
|
||||||
|
|
||||||
|
type ModelSettingsCompatibilityChange = {
|
||||||
|
field: "variant" | "reasoningEffort"
|
||||||
|
from: string
|
||||||
|
to?: string
|
||||||
|
reason: string
|
||||||
|
}
|
||||||
|
|
||||||
|
type ModelSettingsCompatibilityResult = {
|
||||||
|
variant?: string
|
||||||
|
reasoningEffort?: string
|
||||||
|
changes: ModelSettingsCompatibilityChange[]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Compatibility model
|
||||||
|
|
||||||
|
Phase 1 should be **metadata-first where the platform exposes reliable capability data**, and only fall back to family-based rules when that metadata is absent.
|
||||||
|
|
||||||
|
### Variant compatibility
|
||||||
|
|
||||||
|
Preferred source of truth:
|
||||||
|
- OpenCode/provider model metadata (`variants`)
|
||||||
|
|
||||||
|
Fallback when metadata is unavailable:
|
||||||
|
- family-based ladders
|
||||||
|
|
||||||
|
Examples of fallback ladders:
|
||||||
|
- Claude Opus family: `low`, `medium`, `high`, `max`
|
||||||
|
- Claude Sonnet/Haiku family: `low`, `medium`, `high`
|
||||||
|
- OpenAI GPT family: conservative family fallback only when metadata is missing
|
||||||
|
- Unknown family: drop unsupported values conservatively
|
||||||
|
|
||||||
|
### Reasoning effort compatibility
|
||||||
|
|
||||||
|
Current Phase 1 source of truth:
|
||||||
|
- conservative model/provider family heuristics
|
||||||
|
|
||||||
|
Reason:
|
||||||
|
- the currently available OpenCode SDK/provider metadata exposes model `variants`, but does not expose an equivalent per-model capability list for `reasoningEffort` levels
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
- GPT/OpenAI-style models: `low`, `medium`, `high`, `xhigh` where supported by family heuristics
|
||||||
|
- Claude family via current OpenCode path: treat `reasoningEffort` as unsupported in Phase 1 and remove it
|
||||||
|
|
||||||
|
The resolver should remain pure model/settings logic only. Transport restrictions remain the responsibility of the request-building path.
|
||||||
|
|
||||||
|
## Separation of concerns
|
||||||
|
|
||||||
|
This design intentionally separates:
|
||||||
|
- model selection (`resolveModel...`, fallback chains)
|
||||||
|
- settings compatibility (this resolver)
|
||||||
|
- request transport compatibility (`chat.params`, prompt body constraints)
|
||||||
|
|
||||||
|
That keeps responsibilities clear:
|
||||||
|
- choose model first
|
||||||
|
- normalize settings second
|
||||||
|
- build request third
|
||||||
|
|
||||||
|
## First integration point
|
||||||
|
|
||||||
|
Phase 1 should first integrate into `chat.params`.
|
||||||
|
|
||||||
|
Why:
|
||||||
|
- it is already the centralized path for request-time tuning
|
||||||
|
- it can influence provider-facing options without leaking unsupported fields into prompt payload bodies
|
||||||
|
- it avoids trying to patch every prompt constructor at once
|
||||||
|
|
||||||
|
## Rollout plan
|
||||||
|
|
||||||
|
### Phase 1
|
||||||
|
- add resolver module and tests
|
||||||
|
- integrate into `chat.params`
|
||||||
|
- migrate `anthropic-effort` to either use the resolver or become a thin Claude-specific supplement around it
|
||||||
|
|
||||||
|
### Phase 2
|
||||||
|
- expand to `thinking`, `maxTokens`, `temperature`, `top_p`
|
||||||
|
- formalize request-path capability tables if needed
|
||||||
|
|
||||||
|
### Phase 3
|
||||||
|
- centralize all variant/reasoning normalization away from scattered hooks and ad hoc callers
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
|
||||||
|
- Overfitting family rules to current model naming conventions
|
||||||
|
- Accidentally changing request semantics on paths that currently rely on implicit behavior
|
||||||
|
- Mixing provider transport limitations with model capability logic
|
||||||
|
|
||||||
|
## Mitigations
|
||||||
|
|
||||||
|
- Keep resolver pure and narrowly scoped in Phase 1
|
||||||
|
- Add explicit regression tests for keep/downgrade/drop decisions
|
||||||
|
- Integrate at one central point first (`chat.params`)
|
||||||
|
- Preserve existing behavior where desired values are already valid
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
Proceed with the central resolver as a new, isolated implementation in a dedicated branch/worktree.
|
||||||
|
This is the clean long-term path and is more reviewable than continuing to add special-case clamps in hooks.
|
||||||
@@ -371,6 +371,26 @@ describe("CategoryConfigSchema", () => {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
test("accepts reasoningEffort values none and minimal", () => {
|
||||||
|
// given
|
||||||
|
const noneConfig = { reasoningEffort: "none" }
|
||||||
|
const minimalConfig = { reasoningEffort: "minimal" }
|
||||||
|
|
||||||
|
// when
|
||||||
|
const noneResult = CategoryConfigSchema.safeParse(noneConfig)
|
||||||
|
const minimalResult = CategoryConfigSchema.safeParse(minimalConfig)
|
||||||
|
|
||||||
|
// then
|
||||||
|
expect(noneResult.success).toBe(true)
|
||||||
|
expect(minimalResult.success).toBe(true)
|
||||||
|
if (noneResult.success) {
|
||||||
|
expect(noneResult.data.reasoningEffort).toBe("none")
|
||||||
|
}
|
||||||
|
if (minimalResult.success) {
|
||||||
|
expect(minimalResult.data.reasoningEffort).toBe("minimal")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
test("rejects non-string variant", () => {
|
test("rejects non-string variant", () => {
|
||||||
// given
|
// given
|
||||||
const config = { model: "openai/gpt-5.4", variant: 123 }
|
const config = { model: "openai/gpt-5.4", variant: 123 }
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ export const AgentOverrideConfigSchema = z.object({
|
|||||||
})
|
})
|
||||||
.optional(),
|
.optional(),
|
||||||
/** Reasoning effort level (OpenAI). Overrides category and default settings. */
|
/** Reasoning effort level (OpenAI). Overrides category and default settings. */
|
||||||
reasoningEffort: z.enum(["low", "medium", "high", "xhigh"]).optional(),
|
reasoningEffort: z.enum(["none", "minimal", "low", "medium", "high", "xhigh"]).optional(),
|
||||||
/** Text verbosity level. */
|
/** Text verbosity level. */
|
||||||
textVerbosity: z.enum(["low", "medium", "high"]).optional(),
|
textVerbosity: z.enum(["low", "medium", "high"]).optional(),
|
||||||
/** Provider-specific options. Passed directly to OpenCode SDK. */
|
/** Provider-specific options. Passed directly to OpenCode SDK. */
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ export const CategoryConfigSchema = z.object({
|
|||||||
budgetTokens: z.number().optional(),
|
budgetTokens: z.number().optional(),
|
||||||
})
|
})
|
||||||
.optional(),
|
.optional(),
|
||||||
reasoningEffort: z.enum(["low", "medium", "high", "xhigh"]).optional(),
|
reasoningEffort: z.enum(["none", "minimal", "low", "medium", "high", "xhigh"]).optional(),
|
||||||
textVerbosity: z.enum(["low", "medium", "high"]).optional(),
|
textVerbosity: z.enum(["low", "medium", "high"]).optional(),
|
||||||
tools: z.record(z.string(), z.boolean()).optional(),
|
tools: z.record(z.string(), z.boolean()).optional(),
|
||||||
prompt_append: z.string().optional(),
|
prompt_append: z.string().optional(),
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import { log, normalizeModelID } from "../../shared"
|
import { log, normalizeModelID } from "../../shared"
|
||||||
|
|
||||||
const OPUS_PATTERN = /claude-opus/i
|
const OPUS_PATTERN = /claude-.*opus/i
|
||||||
|
|
||||||
function isClaudeProvider(providerID: string, modelID: string): boolean {
|
function isClaudeProvider(providerID: string, modelID: string): boolean {
|
||||||
if (["anthropic", "google-vertex-anthropic", "opencode"].includes(providerID)) return true
|
if (["anthropic", "google-vertex-anthropic", "opencode"].includes(providerID)) return true
|
||||||
|
|||||||
@@ -45,75 +45,31 @@ function createMockParams(overrides: {
|
|||||||
}
|
}
|
||||||
|
|
||||||
describe("createAnthropicEffortHook", () => {
|
describe("createAnthropicEffortHook", () => {
|
||||||
describe("opus 4-6 with variant max", () => {
|
describe("opus family with variant max", () => {
|
||||||
it("should inject effort max for anthropic opus-4-6 with variant max", async () => {
|
it("injects effort max for anthropic opus-4-6", async () => {
|
||||||
//#given anthropic opus-4-6 model with variant max
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({})
|
const { input, output } = createMockParams({})
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should be injected into options
|
|
||||||
expect(output.options.effort).toBe("max")
|
expect(output.options.effort).toBe("max")
|
||||||
})
|
})
|
||||||
|
|
||||||
it("should inject effort max for github-copilot claude-opus-4-6", async () => {
|
it("injects effort max for another opus family model such as opus-4-5", async () => {
|
||||||
//#given github-copilot provider with claude-opus-4-6
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({
|
const { input, output } = createMockParams({ modelID: "claude-opus-4-5" })
|
||||||
providerID: "github-copilot",
|
|
||||||
modelID: "claude-opus-4-6",
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should be injected (github-copilot resolves to anthropic)
|
|
||||||
expect(output.options.effort).toBe("max")
|
expect(output.options.effort).toBe("max")
|
||||||
})
|
})
|
||||||
|
|
||||||
it("should inject effort max for opencode provider with claude-opus-4-6", async () => {
|
it("injects effort max for dotted opus ids", async () => {
|
||||||
//#given opencode provider with claude-opus-4-6
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({
|
const { input, output } = createMockParams({ modelID: "claude-opus-4.6" })
|
||||||
providerID: "opencode",
|
|
||||||
modelID: "claude-opus-4-6",
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should be injected
|
|
||||||
expect(output.options.effort).toBe("max")
|
|
||||||
})
|
|
||||||
|
|
||||||
it("should inject effort max for google-vertex-anthropic provider", async () => {
|
|
||||||
//#given google-vertex-anthropic provider with claude-opus-4-6
|
|
||||||
const hook = createAnthropicEffortHook()
|
|
||||||
const { input, output } = createMockParams({
|
|
||||||
providerID: "google-vertex-anthropic",
|
|
||||||
modelID: "claude-opus-4-6",
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
|
||||||
|
|
||||||
//#then effort should be injected
|
|
||||||
expect(output.options.effort).toBe("max")
|
|
||||||
})
|
|
||||||
|
|
||||||
it("should handle normalized model ID with dots (opus-4.6)", async () => {
|
|
||||||
//#given model ID with dots instead of hyphens
|
|
||||||
const hook = createAnthropicEffortHook()
|
|
||||||
const { input, output } = createMockParams({
|
|
||||||
modelID: "claude-opus-4.6",
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
|
||||||
|
|
||||||
//#then should normalize and inject effort
|
|
||||||
expect(output.options.effort).toBe("max")
|
expect(output.options.effort).toBe("max")
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -133,39 +89,30 @@ describe("createAnthropicEffortHook", () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
describe("conditions NOT met - should skip", () => {
|
describe("skip conditions", () => {
|
||||||
it("should NOT inject effort when variant is not max", async () => {
|
it("does nothing when variant is not max", async () => {
|
||||||
//#given opus-4-6 with variant high (not max)
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({ variant: "high" })
|
const { input, output } = createMockParams({ variant: "high" })
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should NOT be injected
|
|
||||||
expect(output.options.effort).toBeUndefined()
|
expect(output.options.effort).toBeUndefined()
|
||||||
})
|
})
|
||||||
|
|
||||||
it("should NOT inject effort when variant is undefined", async () => {
|
it("does nothing when variant is undefined", async () => {
|
||||||
//#given opus-4-6 with no variant
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({ variant: undefined })
|
const { input, output } = createMockParams({ variant: undefined })
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should NOT be injected
|
|
||||||
expect(output.options.effort).toBeUndefined()
|
expect(output.options.effort).toBeUndefined()
|
||||||
})
|
})
|
||||||
|
|
||||||
it("should clamp effort to high for non-opus claude model with variant max", async () => {
|
it("should clamp effort to high for non-opus claude model with variant max", async () => {
|
||||||
//#given claude-sonnet-4-6 (not opus) with variant max
|
//#given claude-sonnet-4-6 (not opus) with variant max
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({
|
const { input, output } = createMockParams({ modelID: "claude-sonnet-4-6" })
|
||||||
modelID: "claude-sonnet-4-6",
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should be clamped to high (not max)
|
//#then effort should be clamped to high (not max)
|
||||||
@@ -173,74 +120,24 @@ describe("createAnthropicEffortHook", () => {
|
|||||||
expect(input.message.variant).toBe("high")
|
expect(input.message.variant).toBe("high")
|
||||||
})
|
})
|
||||||
|
|
||||||
it("should NOT inject effort for non-anthropic provider with non-claude model", async () => {
|
it("does nothing for non-claude providers/models", async () => {
|
||||||
//#given openai provider with gpt model
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({
|
const { input, output } = createMockParams({ providerID: "openai", modelID: "gpt-5.4" })
|
||||||
providerID: "openai",
|
|
||||||
modelID: "gpt-5.4",
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then effort should NOT be injected
|
|
||||||
expect(output.options.effort).toBeUndefined()
|
|
||||||
})
|
|
||||||
|
|
||||||
it("should NOT throw when model.modelID is undefined", async () => {
|
|
||||||
//#given model with undefined modelID (runtime edge case)
|
|
||||||
const hook = createAnthropicEffortHook()
|
|
||||||
const input = {
|
|
||||||
sessionID: "test-session",
|
|
||||||
agent: { name: "sisyphus" },
|
|
||||||
model: { providerID: "anthropic", modelID: undefined as unknown as string },
|
|
||||||
provider: { id: "anthropic" },
|
|
||||||
message: { variant: "max" as const },
|
|
||||||
}
|
|
||||||
const output = { temperature: 0.1, options: {} }
|
|
||||||
|
|
||||||
//#when chat.params hook is called with undefined modelID
|
|
||||||
await hook["chat.params"](input, output)
|
|
||||||
|
|
||||||
//#then should gracefully skip without throwing
|
|
||||||
expect(output.options.effort).toBeUndefined()
|
expect(output.options.effort).toBeUndefined()
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
describe("preserves existing options", () => {
|
describe("existing options", () => {
|
||||||
it("should NOT overwrite existing effort if already set", async () => {
|
it("does not overwrite existing effort", async () => {
|
||||||
//#given options already have effort set
|
|
||||||
const hook = createAnthropicEffortHook()
|
const hook = createAnthropicEffortHook()
|
||||||
const { input, output } = createMockParams({
|
const { input, output } = createMockParams({ existingOptions: { effort: "high" } })
|
||||||
existingOptions: { effort: "high" },
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
await hook["chat.params"](input, output)
|
||||||
|
|
||||||
//#then existing effort should be preserved
|
|
||||||
expect(output.options.effort).toBe("high")
|
expect(output.options.effort).toBe("high")
|
||||||
})
|
})
|
||||||
|
|
||||||
it("should preserve other existing options when injecting effort", async () => {
|
|
||||||
//#given options with existing thinking config
|
|
||||||
const hook = createAnthropicEffortHook()
|
|
||||||
const { input, output } = createMockParams({
|
|
||||||
existingOptions: {
|
|
||||||
thinking: { type: "enabled", budgetTokens: 31999 },
|
|
||||||
},
|
|
||||||
})
|
|
||||||
|
|
||||||
//#when chat.params hook is called
|
|
||||||
await hook["chat.params"](input, output)
|
|
||||||
|
|
||||||
//#then effort should be added without affecting thinking
|
|
||||||
expect(output.options.effort).toBe("max")
|
|
||||||
expect(output.options.thinking).toEqual({
|
|
||||||
type: "enabled",
|
|
||||||
budgetTokens: 31999,
|
|
||||||
})
|
|
||||||
})
|
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -32,7 +32,13 @@ export function createPluginInterface(args: {
|
|||||||
return {
|
return {
|
||||||
tool: tools,
|
tool: tools,
|
||||||
|
|
||||||
"chat.params": createChatParamsHandler({ anthropicEffort: hooks.anthropicEffort }),
|
"chat.params": async (input: unknown, output: unknown) => {
|
||||||
|
const handler = createChatParamsHandler({
|
||||||
|
anthropicEffort: hooks.anthropicEffort,
|
||||||
|
client: ctx.client,
|
||||||
|
})
|
||||||
|
await handler(input, output)
|
||||||
|
},
|
||||||
|
|
||||||
"chat.headers": createChatHeadersHandler({ ctx }),
|
"chat.headers": createChatHeadersHandler({ ctx }),
|
||||||
|
|
||||||
@@ -68,9 +74,5 @@ export function createPluginInterface(args: {
|
|||||||
ctx,
|
ctx,
|
||||||
hooks,
|
hooks,
|
||||||
}),
|
}),
|
||||||
|
|
||||||
"tool.definition": async (input, output) => {
|
|
||||||
await hooks.todoDescriptionOverride?.["tool.definition"]?.(input, output)
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
|
import { normalizeSDKResponse } from "../shared/normalize-sdk-response"
|
||||||
import { getSessionPromptParams } from "../shared/session-prompt-params-state"
|
import { getSessionPromptParams } from "../shared/session-prompt-params-state"
|
||||||
|
import { resolveCompatibleModelSettings } from "../shared"
|
||||||
|
|
||||||
export type ChatParamsInput = {
|
export type ChatParamsInput = {
|
||||||
sessionID: string
|
sessionID: string
|
||||||
@@ -19,6 +21,25 @@ export type ChatParamsOutput = {
|
|||||||
options: Record<string, unknown>
|
options: Record<string, unknown>
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type ProviderListClient = {
|
||||||
|
provider?: {
|
||||||
|
list?: () => Promise<unknown>
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type ProviderModelMetadata = {
|
||||||
|
variants?: Record<string, unknown>
|
||||||
|
}
|
||||||
|
|
||||||
|
type ProviderListEntry = {
|
||||||
|
id?: string
|
||||||
|
models?: Record<string, ProviderModelMetadata>
|
||||||
|
}
|
||||||
|
|
||||||
|
type ProviderListData = {
|
||||||
|
all?: ProviderListEntry[]
|
||||||
|
}
|
||||||
|
|
||||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||||
return typeof value === "object" && value !== null
|
return typeof value === "object" && value !== null
|
||||||
}
|
}
|
||||||
@@ -49,7 +70,11 @@ function buildChatParamsInput(raw: unknown): ChatParamsHookInput | null {
|
|||||||
if (!agentName) return null
|
if (!agentName) return null
|
||||||
|
|
||||||
const providerID = model.providerID
|
const providerID = model.providerID
|
||||||
const modelID = model.modelID
|
const modelID = typeof model.modelID === "string"
|
||||||
|
? model.modelID
|
||||||
|
: typeof model.id === "string"
|
||||||
|
? model.id
|
||||||
|
: undefined
|
||||||
const providerId = provider.id
|
const providerId = provider.id
|
||||||
const variant = message.variant
|
const variant = message.variant
|
||||||
|
|
||||||
@@ -76,8 +101,33 @@ function isChatParamsOutput(raw: unknown): raw is ChatParamsOutput {
|
|||||||
return isRecord(raw.options)
|
return isRecord(raw.options)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function getVariantCapabilities(
|
||||||
|
client: ProviderListClient | undefined,
|
||||||
|
model: { providerID: string; modelID: string },
|
||||||
|
): Promise<string[] | undefined> {
|
||||||
|
const providerList = client?.provider?.list
|
||||||
|
if (typeof providerList !== "function") {
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const response = await providerList()
|
||||||
|
const data = normalizeSDKResponse<ProviderListData>(response, {})
|
||||||
|
const providerEntry = data.all?.find((entry) => entry.id === model.providerID)
|
||||||
|
const variants = providerEntry?.models?.[model.modelID]?.variants
|
||||||
|
if (!variants) {
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
return Object.keys(variants)
|
||||||
|
} catch {
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export function createChatParamsHandler(args: {
|
export function createChatParamsHandler(args: {
|
||||||
anthropicEffort: { "chat.params"?: (input: ChatParamsHookInput, output: ChatParamsOutput) => Promise<void> } | null
|
anthropicEffort: { "chat.params"?: (input: ChatParamsHookInput, output: ChatParamsOutput) => Promise<void> } | null
|
||||||
|
client?: ProviderListClient
|
||||||
}): (input: unknown, output: unknown) => Promise<void> {
|
}): (input: unknown, output: unknown) => Promise<void> {
|
||||||
return async (input, output): Promise<void> => {
|
return async (input, output): Promise<void> => {
|
||||||
const normalizedInput = buildChatParamsInput(input)
|
const normalizedInput = buildChatParamsInput(input)
|
||||||
@@ -100,6 +150,37 @@ export function createChatParamsHandler(args: {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const variantCapabilities = await getVariantCapabilities(args.client, normalizedInput.model)
|
||||||
|
|
||||||
|
const compatibility = resolveCompatibleModelSettings({
|
||||||
|
providerID: normalizedInput.model.providerID,
|
||||||
|
modelID: normalizedInput.model.modelID,
|
||||||
|
desired: {
|
||||||
|
variant: normalizedInput.message.variant,
|
||||||
|
reasoningEffort: typeof output.options.reasoningEffort === "string"
|
||||||
|
? output.options.reasoningEffort
|
||||||
|
: undefined,
|
||||||
|
},
|
||||||
|
capabilities: {
|
||||||
|
variants: variantCapabilities,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
if (normalizedInput.rawMessage) {
|
||||||
|
if (compatibility.variant !== undefined) {
|
||||||
|
normalizedInput.rawMessage.variant = compatibility.variant
|
||||||
|
} else {
|
||||||
|
delete normalizedInput.rawMessage.variant
|
||||||
|
}
|
||||||
|
}
|
||||||
|
normalizedInput.message = normalizedInput.rawMessage as { variant?: string }
|
||||||
|
|
||||||
|
if (compatibility.reasoningEffort !== undefined) {
|
||||||
|
output.options.reasoningEffort = compatibility.reasoningEffort
|
||||||
|
} else if ("reasoningEffort" in output.options) {
|
||||||
|
delete output.options.reasoningEffort
|
||||||
|
}
|
||||||
|
|
||||||
await args.anthropicEffort?.["chat.params"]?.(normalizedInput, output)
|
await args.anthropicEffort?.["chat.params"]?.(normalizedInput, output)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -43,6 +43,7 @@ export type {
|
|||||||
ModelResolutionResult,
|
ModelResolutionResult,
|
||||||
} from "./model-resolution-types"
|
} from "./model-resolution-types"
|
||||||
export * from "./model-availability"
|
export * from "./model-availability"
|
||||||
|
export * from "./model-settings-compatibility"
|
||||||
export * from "./fallback-model-availability"
|
export * from "./fallback-model-availability"
|
||||||
export * from "./connected-providers-cache"
|
export * from "./connected-providers-cache"
|
||||||
export * from "./context-limit-resolver"
|
export * from "./context-limit-resolver"
|
||||||
|
|||||||
302
src/shared/model-settings-compatibility.test.ts
Normal file
302
src/shared/model-settings-compatibility.test.ts
Normal file
@@ -0,0 +1,302 @@
|
|||||||
|
import { describe, expect, test } from "bun:test"
|
||||||
|
|
||||||
|
import { resolveCompatibleModelSettings } from "./model-settings-compatibility"
|
||||||
|
|
||||||
|
describe("resolveCompatibleModelSettings", () => {
|
||||||
|
test("keeps supported Claude Opus variant unchanged", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "anthropic",
|
||||||
|
modelID: "claude-opus-4-6",
|
||||||
|
desired: { variant: "max" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: "max",
|
||||||
|
reasoningEffort: undefined,
|
||||||
|
changes: [],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
test("uses model metadata first for variant support", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "anthropic",
|
||||||
|
modelID: "claude-opus-4-6",
|
||||||
|
desired: { variant: "max" },
|
||||||
|
capabilities: { variants: ["low", "medium", "high"] },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: "high",
|
||||||
|
reasoningEffort: undefined,
|
||||||
|
changes: [
|
||||||
|
{
|
||||||
|
field: "variant",
|
||||||
|
from: "max",
|
||||||
|
to: "high",
|
||||||
|
reason: "unsupported-by-model-metadata",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
test("prefers metadata over family heuristics even when family would allow a higher level", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "anthropic",
|
||||||
|
modelID: "claude-opus-4-6",
|
||||||
|
desired: { variant: "max" },
|
||||||
|
capabilities: { variants: ["low", "medium"] },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result.variant).toBe("medium")
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "variant",
|
||||||
|
from: "max",
|
||||||
|
to: "medium",
|
||||||
|
reason: "unsupported-by-model-metadata",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("downgrades unsupported Claude Sonnet max variant to high when metadata is absent", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "anthropic",
|
||||||
|
modelID: "claude-sonnet-4-6",
|
||||||
|
desired: { variant: "max" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result.variant).toBe("high")
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "variant",
|
||||||
|
from: "max",
|
||||||
|
to: "high",
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("keeps supported GPT reasoningEffort unchanged", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "gpt-5.4",
|
||||||
|
desired: { reasoningEffort: "high" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: undefined,
|
||||||
|
reasoningEffort: "high",
|
||||||
|
changes: [],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
test("downgrades gpt-5 reasoningEffort max to xhigh", () => {
|
||||||
|
// given
|
||||||
|
const input = {
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "gpt-5.4",
|
||||||
|
desired: { reasoningEffort: "max" },
|
||||||
|
}
|
||||||
|
|
||||||
|
// when
|
||||||
|
const result = resolveCompatibleModelSettings(input)
|
||||||
|
|
||||||
|
// then
|
||||||
|
expect(result.reasoningEffort).toBe("xhigh")
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "max",
|
||||||
|
to: "xhigh",
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("keeps supported OpenAI reasoning-family effort for o-series models", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "o3-mini",
|
||||||
|
desired: { reasoningEffort: "high" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: undefined,
|
||||||
|
reasoningEffort: "high",
|
||||||
|
changes: [],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
test("downgrades openai reasoning-family effort xhigh to high", () => {
|
||||||
|
// given
|
||||||
|
const input = {
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "o3-mini",
|
||||||
|
desired: { reasoningEffort: "xhigh" },
|
||||||
|
}
|
||||||
|
|
||||||
|
// when
|
||||||
|
const result = resolveCompatibleModelSettings(input)
|
||||||
|
|
||||||
|
// then
|
||||||
|
expect(result.reasoningEffort).toBe("high")
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "xhigh",
|
||||||
|
to: "high",
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("drops reasoningEffort for gpt-5 mini models", () => {
|
||||||
|
// given
|
||||||
|
const input = {
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "gpt-5.4-mini",
|
||||||
|
desired: { reasoningEffort: "high" },
|
||||||
|
}
|
||||||
|
|
||||||
|
// when
|
||||||
|
const result = resolveCompatibleModelSettings(input)
|
||||||
|
|
||||||
|
// then
|
||||||
|
expect(result.reasoningEffort).toBeUndefined()
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "high",
|
||||||
|
to: undefined,
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("treats non-openai o-series models as unknown", () => {
|
||||||
|
// given
|
||||||
|
const input = {
|
||||||
|
providerID: "ollama",
|
||||||
|
modelID: "o3",
|
||||||
|
desired: { reasoningEffort: "high" },
|
||||||
|
}
|
||||||
|
|
||||||
|
// when
|
||||||
|
const result = resolveCompatibleModelSettings(input)
|
||||||
|
|
||||||
|
// then
|
||||||
|
expect(result.reasoningEffort).toBeUndefined()
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "high",
|
||||||
|
to: undefined,
|
||||||
|
reason: "unknown-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("does not record case-only normalization as a compatibility downgrade", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "gpt-5.4",
|
||||||
|
desired: { variant: "HIGH", reasoningEffort: "HIGH" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: "high",
|
||||||
|
reasoningEffort: "high",
|
||||||
|
changes: [],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
test("downgrades unsupported GPT reasoningEffort to nearest lower level", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "openai",
|
||||||
|
modelID: "gpt-4.1",
|
||||||
|
desired: { reasoningEffort: "xhigh" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result.reasoningEffort).toBe("high")
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "xhigh",
|
||||||
|
to: "high",
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("drops reasoningEffort for Claude family", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "anthropic",
|
||||||
|
modelID: "claude-sonnet-4-6",
|
||||||
|
desired: { reasoningEffort: "high" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result.reasoningEffort).toBeUndefined()
|
||||||
|
expect(result.changes).toEqual([
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "high",
|
||||||
|
to: undefined,
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("handles combined variant and reasoningEffort normalization", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "anthropic",
|
||||||
|
modelID: "claude-sonnet-4-6",
|
||||||
|
desired: { variant: "max", reasoningEffort: "high" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: "high",
|
||||||
|
reasoningEffort: undefined,
|
||||||
|
changes: [
|
||||||
|
{
|
||||||
|
field: "variant",
|
||||||
|
from: "max",
|
||||||
|
to: "high",
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "high",
|
||||||
|
to: undefined,
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
test("treats unknown model families conservatively by dropping unsupported settings", () => {
|
||||||
|
const result = resolveCompatibleModelSettings({
|
||||||
|
providerID: "mystery",
|
||||||
|
modelID: "mystery-model-1",
|
||||||
|
desired: { variant: "max", reasoningEffort: "high" },
|
||||||
|
})
|
||||||
|
|
||||||
|
expect(result).toEqual({
|
||||||
|
variant: undefined,
|
||||||
|
reasoningEffort: undefined,
|
||||||
|
changes: [
|
||||||
|
{
|
||||||
|
field: "variant",
|
||||||
|
from: "max",
|
||||||
|
to: undefined,
|
||||||
|
reason: "unknown-model-family",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: "high",
|
||||||
|
to: undefined,
|
||||||
|
reason: "unknown-model-family",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
})
|
||||||
|
})
|
||||||
260
src/shared/model-settings-compatibility.ts
Normal file
260
src/shared/model-settings-compatibility.ts
Normal file
@@ -0,0 +1,260 @@
|
|||||||
|
import { normalizeModelID } from "./model-normalization"
|
||||||
|
|
||||||
|
type CompatibilityField = "variant" | "reasoningEffort"
|
||||||
|
|
||||||
|
type DesiredModelSettings = {
|
||||||
|
variant?: string
|
||||||
|
reasoningEffort?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
type VariantCapabilities = {
|
||||||
|
variants?: string[]
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ModelSettingsCompatibilityInput = {
|
||||||
|
providerID: string
|
||||||
|
modelID: string
|
||||||
|
desired: DesiredModelSettings
|
||||||
|
capabilities?: VariantCapabilities
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ModelSettingsCompatibilityChange = {
|
||||||
|
field: CompatibilityField
|
||||||
|
from: string
|
||||||
|
to?: string
|
||||||
|
reason: "unsupported-by-model-family" | "unknown-model-family" | "unsupported-by-model-metadata"
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ModelSettingsCompatibilityResult = {
|
||||||
|
variant?: string
|
||||||
|
reasoningEffort?: string
|
||||||
|
changes: ModelSettingsCompatibilityChange[]
|
||||||
|
}
|
||||||
|
|
||||||
|
type ModelFamily = "claude-opus" | "claude-non-opus" | "openai-reasoning" | "gpt-5" | "gpt-5-mini" | "gpt-legacy" | "unknown"
|
||||||
|
|
||||||
|
const VARIANT_LADDER = ["low", "medium", "high", "xhigh", "max"]
|
||||||
|
const REASONING_LADDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"]
|
||||||
|
|
||||||
|
function detectModelFamily(providerID: string, modelID: string): ModelFamily {
|
||||||
|
const provider = providerID.toLowerCase()
|
||||||
|
const model = normalizeModelID(modelID).toLowerCase()
|
||||||
|
|
||||||
|
const isClaudeProvider = [
|
||||||
|
"anthropic",
|
||||||
|
"google-vertex-anthropic",
|
||||||
|
"aws-bedrock-anthropic",
|
||||||
|
].includes(provider)
|
||||||
|
|| (["github-copilot", "opencode", "aws-bedrock", "bedrock"].includes(provider) && model.includes("claude"))
|
||||||
|
|
||||||
|
if (isClaudeProvider) {
|
||||||
|
return /claude(?:-\d+(?:-\d+)*)?-opus/.test(model) ? "claude-opus" : "claude-non-opus"
|
||||||
|
}
|
||||||
|
|
||||||
|
const isOpenAiReasoningFamily = provider === "openai" && (/^o\d(?:$|-)/.test(model) || model.includes("reasoning"))
|
||||||
|
if (isOpenAiReasoningFamily) {
|
||||||
|
return "openai-reasoning"
|
||||||
|
}
|
||||||
|
|
||||||
|
if (/gpt-5.*-mini/.test(model)) {
|
||||||
|
return "gpt-5-mini"
|
||||||
|
}
|
||||||
|
|
||||||
|
if (model.includes("gpt-5")) {
|
||||||
|
return "gpt-5"
|
||||||
|
}
|
||||||
|
|
||||||
|
if (model.includes("gpt") || (provider === "openai" && /^o\d(?:$|-)/.test(model))) {
|
||||||
|
return "gpt-legacy"
|
||||||
|
}
|
||||||
|
|
||||||
|
return "unknown"
|
||||||
|
}
|
||||||
|
|
||||||
|
function downgradeWithinLadder(value: string, allowed: string[], ladder: string[]): string | undefined {
|
||||||
|
const requestedIndex = ladder.indexOf(value)
|
||||||
|
if (requestedIndex === -1) return undefined
|
||||||
|
|
||||||
|
for (let index = requestedIndex; index >= 0; index -= 1) {
|
||||||
|
const candidate = ladder[index]
|
||||||
|
if (allowed.includes(candidate)) {
|
||||||
|
return candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeCapabilitiesVariants(capabilities: VariantCapabilities | undefined): string[] | undefined {
|
||||||
|
if (!capabilities?.variants || capabilities.variants.length === 0) {
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
return capabilities.variants.map((variant) => variant.toLowerCase())
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveVariant(
|
||||||
|
modelFamily: ModelFamily,
|
||||||
|
variant: string,
|
||||||
|
capabilities?: VariantCapabilities,
|
||||||
|
): { value?: string; reason?: ModelSettingsCompatibilityChange["reason"] } {
|
||||||
|
const normalized = variant.toLowerCase()
|
||||||
|
const metadataVariants = normalizeCapabilitiesVariants(capabilities)
|
||||||
|
|
||||||
|
if (metadataVariants) {
|
||||||
|
if (metadataVariants.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, metadataVariants, VARIANT_LADDER),
|
||||||
|
reason: "unsupported-by-model-metadata",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "claude-opus") {
|
||||||
|
const allowed = ["low", "medium", "high", "max"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "claude-non-opus") {
|
||||||
|
const allowed = ["low", "medium", "high"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "gpt-5" || modelFamily === "gpt-5-mini") {
|
||||||
|
const allowed = ["low", "medium", "high", "xhigh", "max"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "openai-reasoning") {
|
||||||
|
const allowed = ["low", "medium", "high"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "gpt-legacy") {
|
||||||
|
const allowed = ["low", "medium", "high"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, VARIANT_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return { value: undefined, reason: "unknown-model-family" }
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveReasoningEffort(modelFamily: ModelFamily, reasoningEffort: string): { value?: string; reason?: ModelSettingsCompatibilityChange["reason"] } {
|
||||||
|
const normalized = reasoningEffort.toLowerCase()
|
||||||
|
|
||||||
|
if (modelFamily === "gpt-5") {
|
||||||
|
const allowed = ["none", "minimal", "low", "medium", "high", "xhigh"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "gpt-5-mini") {
|
||||||
|
return { value: undefined, reason: "unsupported-by-model-family" }
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "openai-reasoning") {
|
||||||
|
const allowed = ["none", "minimal", "low", "medium", "high"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "gpt-legacy") {
|
||||||
|
const allowed = ["none", "minimal", "low", "medium", "high"]
|
||||||
|
if (allowed.includes(normalized)) {
|
||||||
|
return { value: normalized }
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
value: downgradeWithinLadder(normalized, allowed, REASONING_LADDER),
|
||||||
|
reason: "unsupported-by-model-family",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (modelFamily === "claude-opus" || modelFamily === "claude-non-opus") {
|
||||||
|
return { value: undefined, reason: "unsupported-by-model-family" }
|
||||||
|
}
|
||||||
|
|
||||||
|
return { value: undefined, reason: "unknown-model-family" }
|
||||||
|
}
|
||||||
|
|
||||||
|
export function resolveCompatibleModelSettings(
|
||||||
|
input: ModelSettingsCompatibilityInput,
|
||||||
|
): ModelSettingsCompatibilityResult {
|
||||||
|
const modelFamily = detectModelFamily(input.providerID, input.modelID)
|
||||||
|
const changes: ModelSettingsCompatibilityChange[] = []
|
||||||
|
|
||||||
|
let variant = input.desired.variant
|
||||||
|
if (variant !== undefined) {
|
||||||
|
const normalizedVariant = variant.toLowerCase()
|
||||||
|
const resolved = resolveVariant(modelFamily, normalizedVariant, input.capabilities)
|
||||||
|
if (resolved.value !== normalizedVariant && resolved.reason) {
|
||||||
|
changes.push({
|
||||||
|
field: "variant",
|
||||||
|
from: variant,
|
||||||
|
to: resolved.value,
|
||||||
|
reason: resolved.reason,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
variant = resolved.value
|
||||||
|
}
|
||||||
|
|
||||||
|
let reasoningEffort = input.desired.reasoningEffort
|
||||||
|
if (reasoningEffort !== undefined) {
|
||||||
|
const normalizedReasoningEffort = reasoningEffort.toLowerCase()
|
||||||
|
const resolved = resolveReasoningEffort(modelFamily, normalizedReasoningEffort)
|
||||||
|
if (resolved.value !== normalizedReasoningEffort && resolved.reason) {
|
||||||
|
changes.push({
|
||||||
|
field: "reasoningEffort",
|
||||||
|
from: reasoningEffort,
|
||||||
|
to: resolved.value,
|
||||||
|
reason: resolved.reason,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
reasoningEffort = resolved.value
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
variant,
|
||||||
|
reasoningEffort,
|
||||||
|
changes,
|
||||||
|
}
|
||||||
|
}
|
||||||
31
src/shared/session-prompt-params-helpers.ts
Normal file
31
src/shared/session-prompt-params-helpers.ts
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
import { clearSessionPromptParams, setSessionPromptParams } from "./session-prompt-params-state"
|
||||||
|
|
||||||
|
type PromptParamModel = {
|
||||||
|
temperature?: number
|
||||||
|
top_p?: number
|
||||||
|
reasoningEffort?: string
|
||||||
|
maxTokens?: number
|
||||||
|
thinking?: { type: "enabled" | "disabled"; budgetTokens?: number }
|
||||||
|
}
|
||||||
|
|
||||||
|
export function applySessionPromptParams(
|
||||||
|
sessionID: string,
|
||||||
|
model: PromptParamModel | undefined,
|
||||||
|
): void {
|
||||||
|
if (!model) {
|
||||||
|
clearSessionPromptParams(sessionID)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const promptOptions: Record<string, unknown> = {
|
||||||
|
...(model.reasoningEffort ? { reasoningEffort: model.reasoningEffort } : {}),
|
||||||
|
...(model.thinking ? { thinking: model.thinking } : {}),
|
||||||
|
...(model.maxTokens !== undefined ? { maxTokens: model.maxTokens } : {}),
|
||||||
|
}
|
||||||
|
|
||||||
|
setSessionPromptParams(sessionID, {
|
||||||
|
...(model.temperature !== undefined ? { temperature: model.temperature } : {}),
|
||||||
|
...(model.top_p !== undefined ? { topP: model.top_p } : {}),
|
||||||
|
...(Object.keys(promptOptions).length > 0 ? { options: promptOptions } : {}),
|
||||||
|
})
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user