mirror of https://github.com/openclaw/openclaw.git
fix(openrouter): silently dropped images for new OpenRouter models — runtime capability detection (#45824)
* fix: fetch OpenRouter model capabilities at runtime for unknown models When an OpenRouter model is not in the built-in static snapshot from pi-ai, the fallback hardcodes input: ["text"], silently dropping images. Query the OpenRouter API at runtime to detect actual capabilities (image support, reasoning, context window) for models not in the built-in list. Results are cached in memory for 1 hour. On API failure/timeout, falls back to text-only (no regression). * feat(openrouter): add disk cache for OpenRouter model capabilities Persist the OpenRouter model catalog to ~/.openclaw/cache/openrouter-models.json so it survives process restarts. Cache lookup order: 1. In-memory Map (instant) 2. On-disk JSON file (avoids network on restart) 3. OpenRouter API fetch (populates both layers) Also triggers a background refresh when a model is not found in the cache, in case it was newly added to OpenRouter. * refactor(openrouter): remove pre-warm, use pure lazy-load with disk cache - Remove eager ensureOpenRouterModelCache() from run.ts - Remove TTL — model capabilities are stable, no periodic re-fetching - Cache lookup: in-memory → disk → API fetch (only when needed) - API is only called when no cache exists or a model is not found - Disk cache persists across gateway restarts * fix(openrouter): address review feedback - Fix timer leak: move clearTimeout to finally block - Fix modality check: only check input side of "->" separator to avoid matching image-generation models (text->image) - Use resolveStateDir() instead of hardcoded homedir()/.openclaw - Separate cache dir and filename constants - Add utf-8 encoding to writeFileSync for consistency - Add data validation when reading disk cache * ci: retrigger checks * fix: preload unknown OpenRouter model capabilities before resolve * fix: accept top-level OpenRouter max token metadata * fix: update changelog for OpenRouter runtime capability lookup (#45824) (thanks @DJjjjhao) * fix: avoid redundant OpenRouter refetches and preserve suppression guards --------- Co-authored-by: Ayaan Zaidi <hi@obviy.us>
This commit is contained in:
Jinhao Dong
GitHub
parent
a97b9014a2
commit
8e4a1d87e2
|
|
@ -22,6 +22,7 @@ Docs: https://docs.openclaw.ai
|
|||
- Control UI/dashboard: preserve structured gateway shutdown reasons across restart disconnects so config-triggered restarts no longer fall back to `disconnected (1006): no reason`. (#46532) Thanks @vincentkoc.
|
||||
- Android/chat: theme the thinking dropdown and TLS trust dialogs explicitly so popup surfaces match the active app theme instead of falling back to mismatched Material defaults.
|
||||
- Z.AI/onboarding: detect a working default model even for explicit `zai-coding-*` endpoint choices, so Coding Plan setup can keep the selected endpoint while defaulting to `glm-5` when available or `glm-4.7` as fallback. (#45969)
|
||||
- Models/OpenRouter runtime capabilities: fetch uncatalogued OpenRouter model metadata on first use so newly added vision models keep image input instead of silently degrading to text-only, with top-level capability field fallbacks for `/api/v1/models`. (#45824) Thanks @DJjjjhao.
|
||||
- Z.AI/onboarding: add `glm-5-turbo` to the default Z.AI provider catalog so onboarding-generated configs expose the new model alongside the existing GLM defaults. (#46670) Thanks @tomsun28.
|
||||
- Zalo Personal/group gating: stop reapplying `dmPolicy.allowFrom` as a sender gate for already-allowlisted groups when `groupAllowFrom` is unset, so any member of an allowed group can trigger replies while DMs stay restricted. (#40146)
|
||||
- Plugins/install precedence: keep bundled plugins ahead of auto-discovered globals by default, but let an explicitly installed plugin record win its own duplicate-id tie so installed channel plugins load from `~/.openclaw/extensions` after `openclaw plugins install`.
|
||||
|
|
|
|||
|
|
@ -87,7 +87,7 @@ import {
|
|||
import { getDmHistoryLimitFromSessionKey, limitHistoryTurns } from "./history.js";
|
||||
import { resolveGlobalLane, resolveSessionLane } from "./lanes.js";
|
||||
import { log } from "./logger.js";
|
||||
import { buildModelAliasLines, resolveModel } from "./model.js";
|
||||
import { buildModelAliasLines, resolveModelAsync } from "./model.js";
|
||||
import { buildEmbeddedSandboxInfo } from "./sandbox-info.js";
|
||||
import { prewarmSessionFile, trackSessionManagerAccess } from "./session-manager-cache.js";
|
||||
import { resolveEmbeddedRunSkillEntries } from "./skills-runtime.js";
|
||||
|
|
@ -423,7 +423,7 @@ export async function compactEmbeddedPiSessionDirect(
|
|||
};
|
||||
const agentDir = params.agentDir ?? resolveOpenClawAgentDir();
|
||||
await ensureOpenClawModelsJson(params.config, agentDir);
|
||||
const { model, error, authStorage, modelRegistry } = resolveModel(
|
||||
const { model, error, authStorage, modelRegistry } = await resolveModelAsync(
|
||||
provider,
|
||||
modelId,
|
||||
agentDir,
|
||||
|
|
@ -1064,7 +1064,12 @@ export async function compactEmbeddedPiSession(
|
|||
const ceProvider = (params.provider ?? DEFAULT_PROVIDER).trim() || DEFAULT_PROVIDER;
|
||||
const ceModelId = (params.model ?? DEFAULT_MODEL).trim() || DEFAULT_MODEL;
|
||||
const agentDir = params.agentDir ?? resolveOpenClawAgentDir();
|
||||
const { model: ceModel } = resolveModel(ceProvider, ceModelId, agentDir, params.config);
|
||||
const { model: ceModel } = await resolveModelAsync(
|
||||
ceProvider,
|
||||
ceModelId,
|
||||
agentDir,
|
||||
params.config,
|
||||
);
|
||||
const ceCtxInfo = resolveContextWindowInfo({
|
||||
cfg: params.config,
|
||||
provider: ceProvider,
|
||||
|
|
|
|||
|
|
@ -5,8 +5,22 @@ vi.mock("../pi-model-discovery.js", () => ({
|
|||
discoverModels: vi.fn(() => ({ find: vi.fn(() => null) })),
|
||||
}));
|
||||
|
||||
import type { OpenRouterModelCapabilities } from "./openrouter-model-capabilities.js";
|
||||
|
||||
const mockGetOpenRouterModelCapabilities = vi.fn<
|
||||
(modelId: string) => OpenRouterModelCapabilities | undefined
|
||||
>(() => undefined);
|
||||
const mockLoadOpenRouterModelCapabilities = vi.fn<(modelId: string) => Promise<void>>(
|
||||
async () => {},
|
||||
);
|
||||
vi.mock("./openrouter-model-capabilities.js", () => ({
|
||||
getOpenRouterModelCapabilities: (modelId: string) => mockGetOpenRouterModelCapabilities(modelId),
|
||||
loadOpenRouterModelCapabilities: (modelId: string) =>
|
||||
mockLoadOpenRouterModelCapabilities(modelId),
|
||||
}));
|
||||
|
||||
import type { OpenClawConfig } from "../../config/config.js";
|
||||
import { buildInlineProviderModels, resolveModel } from "./model.js";
|
||||
import { buildInlineProviderModels, resolveModel, resolveModelAsync } from "./model.js";
|
||||
import {
|
||||
buildOpenAICodexForwardCompatExpectation,
|
||||
makeModel,
|
||||
|
|
@ -17,6 +31,10 @@ import {
|
|||
|
||||
beforeEach(() => {
|
||||
resetMockDiscoverModels();
|
||||
mockGetOpenRouterModelCapabilities.mockReset();
|
||||
mockGetOpenRouterModelCapabilities.mockReturnValue(undefined);
|
||||
mockLoadOpenRouterModelCapabilities.mockReset();
|
||||
mockLoadOpenRouterModelCapabilities.mockResolvedValue();
|
||||
});
|
||||
|
||||
function buildForwardCompatTemplate(params: {
|
||||
|
|
@ -416,6 +434,107 @@ describe("resolveModel", () => {
|
|||
});
|
||||
});
|
||||
|
||||
it("uses OpenRouter API capabilities for unknown models when cache is populated", () => {
|
||||
mockGetOpenRouterModelCapabilities.mockReturnValue({
|
||||
name: "Healer Alpha",
|
||||
input: ["text", "image"],
|
||||
reasoning: true,
|
||||
contextWindow: 262144,
|
||||
maxTokens: 65536,
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
});
|
||||
|
||||
const result = resolveModel("openrouter", "openrouter/healer-alpha", "/tmp/agent");
|
||||
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.model).toMatchObject({
|
||||
provider: "openrouter",
|
||||
id: "openrouter/healer-alpha",
|
||||
name: "Healer Alpha",
|
||||
reasoning: true,
|
||||
input: ["text", "image"],
|
||||
contextWindow: 262144,
|
||||
maxTokens: 65536,
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back to text-only when OpenRouter API cache is empty", () => {
|
||||
mockGetOpenRouterModelCapabilities.mockReturnValue(undefined);
|
||||
|
||||
const result = resolveModel("openrouter", "openrouter/healer-alpha", "/tmp/agent");
|
||||
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.model).toMatchObject({
|
||||
provider: "openrouter",
|
||||
id: "openrouter/healer-alpha",
|
||||
reasoning: false,
|
||||
input: ["text"],
|
||||
});
|
||||
});
|
||||
|
||||
it("preloads OpenRouter capabilities before first async resolve of an unknown model", async () => {
|
||||
mockLoadOpenRouterModelCapabilities.mockImplementation(async (modelId) => {
|
||||
if (modelId === "google/gemini-3.1-flash-image-preview") {
|
||||
mockGetOpenRouterModelCapabilities.mockReturnValue({
|
||||
name: "Google: Nano Banana 2 (Gemini 3.1 Flash Image Preview)",
|
||||
input: ["text", "image"],
|
||||
reasoning: true,
|
||||
contextWindow: 65536,
|
||||
maxTokens: 65536,
|
||||
cost: { input: 0.5, output: 3, cacheRead: 0, cacheWrite: 0 },
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
const result = await resolveModelAsync(
|
||||
"openrouter",
|
||||
"google/gemini-3.1-flash-image-preview",
|
||||
"/tmp/agent",
|
||||
);
|
||||
|
||||
expect(mockLoadOpenRouterModelCapabilities).toHaveBeenCalledWith(
|
||||
"google/gemini-3.1-flash-image-preview",
|
||||
);
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.model).toMatchObject({
|
||||
provider: "openrouter",
|
||||
id: "google/gemini-3.1-flash-image-preview",
|
||||
reasoning: true,
|
||||
input: ["text", "image"],
|
||||
contextWindow: 65536,
|
||||
maxTokens: 65536,
|
||||
});
|
||||
});
|
||||
|
||||
it("skips OpenRouter preload for models already present in the registry", async () => {
|
||||
mockDiscoveredModel({
|
||||
provider: "openrouter",
|
||||
modelId: "openrouter/healer-alpha",
|
||||
templateModel: {
|
||||
id: "openrouter/healer-alpha",
|
||||
name: "Healer Alpha",
|
||||
api: "openai-completions",
|
||||
provider: "openrouter",
|
||||
baseUrl: "https://openrouter.ai/api/v1",
|
||||
reasoning: true,
|
||||
input: ["text", "image"],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 262144,
|
||||
maxTokens: 65536,
|
||||
},
|
||||
});
|
||||
|
||||
const result = await resolveModelAsync("openrouter", "openrouter/healer-alpha", "/tmp/agent");
|
||||
|
||||
expect(mockLoadOpenRouterModelCapabilities).not.toHaveBeenCalled();
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.model).toMatchObject({
|
||||
provider: "openrouter",
|
||||
id: "openrouter/healer-alpha",
|
||||
input: ["text", "image"],
|
||||
});
|
||||
});
|
||||
|
||||
it("prefers configured provider api metadata over discovered registry model", () => {
|
||||
mockDiscoveredModel({
|
||||
provider: "onehub",
|
||||
|
|
@ -788,6 +907,27 @@ describe("resolveModel", () => {
|
|||
);
|
||||
});
|
||||
|
||||
it("keeps suppressed openai gpt-5.3-codex-spark from falling through provider fallback", () => {
|
||||
const cfg = {
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
api: "openai-responses",
|
||||
models: [{ ...makeModel("gpt-4.1"), api: "openai-responses" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
|
||||
const result = resolveModel("openai", "gpt-5.3-codex-spark", "/tmp/agent", cfg);
|
||||
|
||||
expect(result.model).toBeUndefined();
|
||||
expect(result.error).toBe(
|
||||
"Unknown model: openai/gpt-5.3-codex-spark. gpt-5.3-codex-spark is only supported via openai-codex OAuth. Use openai-codex/gpt-5.3-codex-spark.",
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects azure openai gpt-5.3-codex-spark with a codex-only hint", () => {
|
||||
const result = resolveModel("azure-openai-responses", "gpt-5.3-codex-spark", "/tmp/agent");
|
||||
|
||||
|
|
|
|||
|
|
@ -14,6 +14,10 @@ import {
|
|||
} from "../model-suppression.js";
|
||||
import { discoverAuthStorage, discoverModels } from "../pi-model-discovery.js";
|
||||
import { normalizeResolvedProviderModel } from "./model.provider-normalization.js";
|
||||
import {
|
||||
getOpenRouterModelCapabilities,
|
||||
loadOpenRouterModelCapabilities,
|
||||
} from "./openrouter-model-capabilities.js";
|
||||
|
||||
type InlineModelEntry = ModelDefinitionConfig & {
|
||||
provider: string;
|
||||
|
|
@ -156,28 +160,31 @@ export function buildInlineProviderModels(
|
|||
});
|
||||
}
|
||||
|
||||
export function resolveModelWithRegistry(params: {
|
||||
function resolveExplicitModelWithRegistry(params: {
|
||||
provider: string;
|
||||
modelId: string;
|
||||
modelRegistry: ModelRegistry;
|
||||
cfg?: OpenClawConfig;
|
||||
}): Model<Api> | undefined {
|
||||
}): { kind: "resolved"; model: Model<Api> } | { kind: "suppressed" } | undefined {
|
||||
const { provider, modelId, modelRegistry, cfg } = params;
|
||||
if (shouldSuppressBuiltInModel({ provider, id: modelId })) {
|
||||
return undefined;
|
||||
return { kind: "suppressed" };
|
||||
}
|
||||
const providerConfig = resolveConfiguredProviderConfig(cfg, provider);
|
||||
const model = modelRegistry.find(provider, modelId) as Model<Api> | null;
|
||||
|
||||
if (model) {
|
||||
return normalizeResolvedModel({
|
||||
provider,
|
||||
model: applyConfiguredProviderOverrides({
|
||||
discoveredModel: model,
|
||||
providerConfig,
|
||||
modelId,
|
||||
return {
|
||||
kind: "resolved",
|
||||
model: normalizeResolvedModel({
|
||||
provider,
|
||||
model: applyConfiguredProviderOverrides({
|
||||
discoveredModel: model,
|
||||
providerConfig,
|
||||
modelId,
|
||||
}),
|
||||
}),
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
const providers = cfg?.models?.providers ?? {};
|
||||
|
|
@ -187,40 +194,70 @@ export function resolveModelWithRegistry(params: {
|
|||
(entry) => normalizeProviderId(entry.provider) === normalizedProvider && entry.id === modelId,
|
||||
);
|
||||
if (inlineMatch?.api) {
|
||||
return normalizeResolvedModel({ provider, model: inlineMatch as Model<Api> });
|
||||
return {
|
||||
kind: "resolved",
|
||||
model: normalizeResolvedModel({ provider, model: inlineMatch as Model<Api> }),
|
||||
};
|
||||
}
|
||||
|
||||
// Forward-compat fallbacks must be checked BEFORE the generic providerCfg fallback.
|
||||
// Otherwise, configured providers can default to a generic API and break specific transports.
|
||||
const forwardCompat = resolveForwardCompatModel(provider, modelId, modelRegistry);
|
||||
if (forwardCompat) {
|
||||
return normalizeResolvedModel({
|
||||
provider,
|
||||
model: applyConfiguredProviderOverrides({
|
||||
discoveredModel: forwardCompat,
|
||||
providerConfig,
|
||||
modelId,
|
||||
return {
|
||||
kind: "resolved",
|
||||
model: normalizeResolvedModel({
|
||||
provider,
|
||||
model: applyConfiguredProviderOverrides({
|
||||
discoveredModel: forwardCompat,
|
||||
providerConfig,
|
||||
modelId,
|
||||
}),
|
||||
}),
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
return undefined;
|
||||
}
|
||||
|
||||
export function resolveModelWithRegistry(params: {
|
||||
provider: string;
|
||||
modelId: string;
|
||||
modelRegistry: ModelRegistry;
|
||||
cfg?: OpenClawConfig;
|
||||
}): Model<Api> | undefined {
|
||||
const explicitModel = resolveExplicitModelWithRegistry(params);
|
||||
if (explicitModel?.kind === "suppressed") {
|
||||
return undefined;
|
||||
}
|
||||
if (explicitModel?.kind === "resolved") {
|
||||
return explicitModel.model;
|
||||
}
|
||||
|
||||
const { provider, modelId, cfg } = params;
|
||||
const normalizedProvider = normalizeProviderId(provider);
|
||||
const providerConfig = resolveConfiguredProviderConfig(cfg, provider);
|
||||
|
||||
// OpenRouter is a pass-through proxy - any model ID available on OpenRouter
|
||||
// should work without being pre-registered in the local catalog.
|
||||
// Try to fetch actual capabilities from the OpenRouter API so that new models
|
||||
// (not yet in the static pi-ai snapshot) get correct image/reasoning support.
|
||||
if (normalizedProvider === "openrouter") {
|
||||
const capabilities = getOpenRouterModelCapabilities(modelId);
|
||||
return normalizeResolvedModel({
|
||||
provider,
|
||||
model: {
|
||||
id: modelId,
|
||||
name: modelId,
|
||||
name: capabilities?.name ?? modelId,
|
||||
api: "openai-completions",
|
||||
provider,
|
||||
baseUrl: "https://openrouter.ai/api/v1",
|
||||
reasoning: false,
|
||||
input: ["text"],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: DEFAULT_CONTEXT_TOKENS,
|
||||
reasoning: capabilities?.reasoning ?? false,
|
||||
input: capabilities?.input ?? ["text"],
|
||||
cost: capabilities?.cost ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: capabilities?.contextWindow ?? DEFAULT_CONTEXT_TOKENS,
|
||||
// Align with OPENROUTER_DEFAULT_MAX_TOKENS in models-config.providers.ts
|
||||
maxTokens: 8192,
|
||||
maxTokens: capabilities?.maxTokens ?? 8192,
|
||||
} as Model<Api>,
|
||||
});
|
||||
}
|
||||
|
|
@ -287,6 +324,46 @@ export function resolveModel(
|
|||
};
|
||||
}
|
||||
|
||||
export async function resolveModelAsync(
|
||||
provider: string,
|
||||
modelId: string,
|
||||
agentDir?: string,
|
||||
cfg?: OpenClawConfig,
|
||||
): Promise<{
|
||||
model?: Model<Api>;
|
||||
error?: string;
|
||||
authStorage: AuthStorage;
|
||||
modelRegistry: ModelRegistry;
|
||||
}> {
|
||||
const resolvedAgentDir = agentDir ?? resolveOpenClawAgentDir();
|
||||
const authStorage = discoverAuthStorage(resolvedAgentDir);
|
||||
const modelRegistry = discoverModels(authStorage, resolvedAgentDir);
|
||||
const explicitModel = resolveExplicitModelWithRegistry({ provider, modelId, modelRegistry, cfg });
|
||||
if (explicitModel?.kind === "suppressed") {
|
||||
return {
|
||||
error: buildUnknownModelError(provider, modelId),
|
||||
authStorage,
|
||||
modelRegistry,
|
||||
};
|
||||
}
|
||||
if (!explicitModel && normalizeProviderId(provider) === "openrouter") {
|
||||
await loadOpenRouterModelCapabilities(modelId);
|
||||
}
|
||||
const model =
|
||||
explicitModel?.kind === "resolved"
|
||||
? explicitModel.model
|
||||
: resolveModelWithRegistry({ provider, modelId, modelRegistry, cfg });
|
||||
if (model) {
|
||||
return { model, authStorage, modelRegistry };
|
||||
}
|
||||
|
||||
return {
|
||||
error: buildUnknownModelError(provider, modelId),
|
||||
authStorage,
|
||||
modelRegistry,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a more helpful error when the model is not found.
|
||||
*
|
||||
|
|
|
|||
|
|
@ -0,0 +1,111 @@
|
|||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
describe("openrouter-model-capabilities", () => {
|
||||
afterEach(() => {
|
||||
vi.resetModules();
|
||||
vi.unstubAllGlobals();
|
||||
delete process.env.OPENCLAW_STATE_DIR;
|
||||
});
|
||||
|
||||
it("uses top-level OpenRouter max token fields when top_provider is absent", async () => {
|
||||
const stateDir = mkdtempSync(join(tmpdir(), "openclaw-openrouter-capabilities-"));
|
||||
process.env.OPENCLAW_STATE_DIR = stateDir;
|
||||
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(
|
||||
async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
data: [
|
||||
{
|
||||
id: "acme/top-level-max-completion",
|
||||
name: "Top Level Max Completion",
|
||||
architecture: { modality: "text+image->text" },
|
||||
supported_parameters: ["reasoning"],
|
||||
context_length: 65432,
|
||||
max_completion_tokens: 12345,
|
||||
pricing: { prompt: "0.000001", completion: "0.000002" },
|
||||
},
|
||||
{
|
||||
id: "acme/top-level-max-output",
|
||||
name: "Top Level Max Output",
|
||||
modality: "text+image->text",
|
||||
context_length: 54321,
|
||||
max_output_tokens: 23456,
|
||||
pricing: { prompt: "0.000003", completion: "0.000004" },
|
||||
},
|
||||
],
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
},
|
||||
),
|
||||
),
|
||||
);
|
||||
|
||||
const module = await import("./openrouter-model-capabilities.js");
|
||||
|
||||
try {
|
||||
await module.loadOpenRouterModelCapabilities("acme/top-level-max-completion");
|
||||
|
||||
expect(module.getOpenRouterModelCapabilities("acme/top-level-max-completion")).toMatchObject({
|
||||
input: ["text", "image"],
|
||||
reasoning: true,
|
||||
contextWindow: 65432,
|
||||
maxTokens: 12345,
|
||||
});
|
||||
expect(module.getOpenRouterModelCapabilities("acme/top-level-max-output")).toMatchObject({
|
||||
input: ["text", "image"],
|
||||
reasoning: false,
|
||||
contextWindow: 54321,
|
||||
maxTokens: 23456,
|
||||
});
|
||||
} finally {
|
||||
rmSync(stateDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("does not refetch immediately after an awaited miss for the same model id", async () => {
|
||||
const stateDir = mkdtempSync(join(tmpdir(), "openclaw-openrouter-capabilities-"));
|
||||
process.env.OPENCLAW_STATE_DIR = stateDir;
|
||||
|
||||
const fetchSpy = vi.fn(
|
||||
async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
data: [
|
||||
{
|
||||
id: "acme/known-model",
|
||||
name: "Known Model",
|
||||
architecture: { modality: "text->text" },
|
||||
context_length: 1234,
|
||||
},
|
||||
],
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
},
|
||||
),
|
||||
);
|
||||
vi.stubGlobal("fetch", fetchSpy);
|
||||
|
||||
const module = await import("./openrouter-model-capabilities.js");
|
||||
|
||||
try {
|
||||
await module.loadOpenRouterModelCapabilities("acme/missing-model");
|
||||
expect(module.getOpenRouterModelCapabilities("acme/missing-model")).toBeUndefined();
|
||||
expect(fetchSpy).toHaveBeenCalledTimes(1);
|
||||
|
||||
expect(module.getOpenRouterModelCapabilities("acme/missing-model")).toBeUndefined();
|
||||
expect(fetchSpy).toHaveBeenCalledTimes(2);
|
||||
} finally {
|
||||
rmSync(stateDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,301 @@
|
|||
/**
|
||||
* Runtime OpenRouter model capability detection.
|
||||
*
|
||||
* When an OpenRouter model is not in the built-in static list, we look up its
|
||||
* actual capabilities from a cached copy of the OpenRouter model catalog.
|
||||
*
|
||||
* Cache layers (checked in order):
|
||||
* 1. In-memory Map (instant, cleared on process restart)
|
||||
* 2. On-disk JSON file (<stateDir>/cache/openrouter-models.json)
|
||||
* 3. OpenRouter API fetch (populates both layers)
|
||||
*
|
||||
* Model capabilities are assumed stable — the cache has no TTL expiry.
|
||||
* A background refresh is triggered only when a model is not found in
|
||||
* the cache (i.e. a newly added model on OpenRouter).
|
||||
*
|
||||
* Sync callers can read whatever is already cached. Async callers can await a
|
||||
* one-time fetch so the first unknown-model lookup resolves with real
|
||||
* capabilities instead of the text-only fallback.
|
||||
*/
|
||||
|
||||
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { resolveStateDir } from "../../config/paths.js";
|
||||
import { resolveProxyFetchFromEnv } from "../../infra/net/proxy-fetch.js";
|
||||
import { createSubsystemLogger } from "../../logging/subsystem.js";
|
||||
|
||||
const log = createSubsystemLogger("openrouter-model-capabilities");
|
||||
|
||||
const OPENROUTER_MODELS_URL = "https://openrouter.ai/api/v1/models";
|
||||
const FETCH_TIMEOUT_MS = 10_000;
|
||||
const DISK_CACHE_FILENAME = "openrouter-models.json";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface OpenRouterApiModel {
|
||||
id: string;
|
||||
name?: string;
|
||||
modality?: string;
|
||||
architecture?: {
|
||||
modality?: string;
|
||||
};
|
||||
supported_parameters?: string[];
|
||||
context_length?: number;
|
||||
max_completion_tokens?: number;
|
||||
max_output_tokens?: number;
|
||||
top_provider?: {
|
||||
max_completion_tokens?: number;
|
||||
};
|
||||
pricing?: {
|
||||
prompt?: string;
|
||||
completion?: string;
|
||||
input_cache_read?: string;
|
||||
input_cache_write?: string;
|
||||
};
|
||||
}
|
||||
|
||||
export interface OpenRouterModelCapabilities {
|
||||
name: string;
|
||||
input: Array<"text" | "image">;
|
||||
reasoning: boolean;
|
||||
contextWindow: number;
|
||||
maxTokens: number;
|
||||
cost: {
|
||||
input: number;
|
||||
output: number;
|
||||
cacheRead: number;
|
||||
cacheWrite: number;
|
||||
};
|
||||
}
|
||||
|
||||
interface DiskCachePayload {
|
||||
models: Record<string, OpenRouterModelCapabilities>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Disk cache
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function resolveDiskCacheDir(): string {
|
||||
return join(resolveStateDir(), "cache");
|
||||
}
|
||||
|
||||
function resolveDiskCachePath(): string {
|
||||
return join(resolveDiskCacheDir(), DISK_CACHE_FILENAME);
|
||||
}
|
||||
|
||||
function writeDiskCache(map: Map<string, OpenRouterModelCapabilities>): void {
|
||||
try {
|
||||
const cacheDir = resolveDiskCacheDir();
|
||||
if (!existsSync(cacheDir)) {
|
||||
mkdirSync(cacheDir, { recursive: true });
|
||||
}
|
||||
const payload: DiskCachePayload = {
|
||||
models: Object.fromEntries(map),
|
||||
};
|
||||
writeFileSync(resolveDiskCachePath(), JSON.stringify(payload), "utf-8");
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
log.debug(`Failed to write OpenRouter disk cache: ${message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function isValidCapabilities(value: unknown): value is OpenRouterModelCapabilities {
|
||||
if (!value || typeof value !== "object") {
|
||||
return false;
|
||||
}
|
||||
const record = value as Record<string, unknown>;
|
||||
return (
|
||||
typeof record.name === "string" &&
|
||||
Array.isArray(record.input) &&
|
||||
typeof record.reasoning === "boolean" &&
|
||||
typeof record.contextWindow === "number" &&
|
||||
typeof record.maxTokens === "number"
|
||||
);
|
||||
}
|
||||
|
||||
function readDiskCache(): Map<string, OpenRouterModelCapabilities> | undefined {
|
||||
try {
|
||||
const cachePath = resolveDiskCachePath();
|
||||
if (!existsSync(cachePath)) {
|
||||
return undefined;
|
||||
}
|
||||
const raw = readFileSync(cachePath, "utf-8");
|
||||
const payload = JSON.parse(raw) as unknown;
|
||||
if (!payload || typeof payload !== "object") {
|
||||
return undefined;
|
||||
}
|
||||
const models = (payload as DiskCachePayload).models;
|
||||
if (!models || typeof models !== "object") {
|
||||
return undefined;
|
||||
}
|
||||
const map = new Map<string, OpenRouterModelCapabilities>();
|
||||
for (const [id, caps] of Object.entries(models)) {
|
||||
if (isValidCapabilities(caps)) {
|
||||
map.set(id, caps);
|
||||
}
|
||||
}
|
||||
return map.size > 0 ? map : undefined;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// In-memory cache state
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
let cache: Map<string, OpenRouterModelCapabilities> | undefined;
|
||||
let fetchInFlight: Promise<void> | undefined;
|
||||
const skipNextMissRefresh = new Set<string>();
|
||||
|
||||
function parseModel(model: OpenRouterApiModel): OpenRouterModelCapabilities {
|
||||
const input: Array<"text" | "image"> = ["text"];
|
||||
const modality = model.architecture?.modality ?? model.modality ?? "";
|
||||
const inputModalities = modality.split("->")[0] ?? "";
|
||||
if (inputModalities.includes("image")) {
|
||||
input.push("image");
|
||||
}
|
||||
|
||||
return {
|
||||
name: model.name || model.id,
|
||||
input,
|
||||
reasoning: model.supported_parameters?.includes("reasoning") ?? false,
|
||||
contextWindow: model.context_length || 128_000,
|
||||
maxTokens:
|
||||
model.top_provider?.max_completion_tokens ??
|
||||
model.max_completion_tokens ??
|
||||
model.max_output_tokens ??
|
||||
8192,
|
||||
cost: {
|
||||
input: parseFloat(model.pricing?.prompt || "0") * 1_000_000,
|
||||
output: parseFloat(model.pricing?.completion || "0") * 1_000_000,
|
||||
cacheRead: parseFloat(model.pricing?.input_cache_read || "0") * 1_000_000,
|
||||
cacheWrite: parseFloat(model.pricing?.input_cache_write || "0") * 1_000_000,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// API fetch
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function doFetch(): Promise<void> {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS);
|
||||
try {
|
||||
const fetchFn = resolveProxyFetchFromEnv() ?? globalThis.fetch;
|
||||
|
||||
const response = await fetchFn(OPENROUTER_MODELS_URL, {
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
log.warn(`OpenRouter models API returned ${response.status}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const data = (await response.json()) as { data?: OpenRouterApiModel[] };
|
||||
const models = data.data ?? [];
|
||||
const map = new Map<string, OpenRouterModelCapabilities>();
|
||||
|
||||
for (const model of models) {
|
||||
if (!model.id) {
|
||||
continue;
|
||||
}
|
||||
map.set(model.id, parseModel(model));
|
||||
}
|
||||
|
||||
cache = map;
|
||||
writeDiskCache(map);
|
||||
log.debug(`Cached ${map.size} OpenRouter models from API`);
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
log.warn(`Failed to fetch OpenRouter models: ${message}`);
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
function triggerFetch(): void {
|
||||
if (fetchInFlight) {
|
||||
return;
|
||||
}
|
||||
fetchInFlight = doFetch().finally(() => {
|
||||
fetchInFlight = undefined;
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public API
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Ensure the cache is populated. Checks in-memory first, then disk, then
|
||||
* triggers a background API fetch as a last resort.
|
||||
* Does not block — returns immediately.
|
||||
*/
|
||||
export function ensureOpenRouterModelCache(): void {
|
||||
if (cache) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Try loading from disk before hitting the network.
|
||||
const disk = readDiskCache();
|
||||
if (disk) {
|
||||
cache = disk;
|
||||
log.debug(`Loaded ${disk.size} OpenRouter models from disk cache`);
|
||||
return;
|
||||
}
|
||||
|
||||
triggerFetch();
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure capabilities for a specific model are available before first use.
|
||||
*
|
||||
* Known cached entries return immediately. Unknown entries wait for at most
|
||||
* one catalog fetch, then leave sync resolution to read from the populated
|
||||
* cache on the same request.
|
||||
*/
|
||||
export async function loadOpenRouterModelCapabilities(modelId: string): Promise<void> {
|
||||
ensureOpenRouterModelCache();
|
||||
if (cache?.has(modelId)) {
|
||||
return;
|
||||
}
|
||||
let fetchPromise = fetchInFlight;
|
||||
if (!fetchPromise) {
|
||||
triggerFetch();
|
||||
fetchPromise = fetchInFlight;
|
||||
}
|
||||
await fetchPromise;
|
||||
if (!cache?.has(modelId)) {
|
||||
skipNextMissRefresh.add(modelId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Synchronously look up model capabilities from the cache.
|
||||
*
|
||||
* If a model is not found but the cache exists, a background refresh is
|
||||
* triggered in case it's a newly added model not yet in the cache.
|
||||
*/
|
||||
export function getOpenRouterModelCapabilities(
|
||||
modelId: string,
|
||||
): OpenRouterModelCapabilities | undefined {
|
||||
ensureOpenRouterModelCache();
|
||||
const result = cache?.get(modelId);
|
||||
|
||||
// Model not found but cache exists — may be a newly added model.
|
||||
// Trigger a refresh so the next call picks it up.
|
||||
if (!result && skipNextMissRefresh.delete(modelId)) {
|
||||
return undefined;
|
||||
}
|
||||
if (!result && cache && !fetchInFlight) {
|
||||
triggerFetch();
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
|
@ -66,7 +66,7 @@ import { derivePromptTokens, normalizeUsage, type UsageLike } from "../usage.js"
|
|||
import { redactRunIdentifier, resolveRunWorkspaceDir } from "../workspace-run.js";
|
||||
import { resolveGlobalLane, resolveSessionLane } from "./lanes.js";
|
||||
import { log } from "./logger.js";
|
||||
import { resolveModel } from "./model.js";
|
||||
import { resolveModelAsync } from "./model.js";
|
||||
import { runEmbeddedAttempt } from "./run/attempt.js";
|
||||
import { createFailoverDecisionLogger } from "./run/failover-observation.js";
|
||||
import type { RunEmbeddedPiAgentParams } from "./run/params.js";
|
||||
|
|
@ -367,7 +367,7 @@ export async function runEmbeddedPiAgent(
|
|||
log.info(`[hooks] model overridden to ${modelId}`);
|
||||
}
|
||||
|
||||
const { model, error, authStorage, modelRegistry } = resolveModel(
|
||||
const { model, error, authStorage, modelRegistry } = await resolveModelAsync(
|
||||
provider,
|
||||
modelId,
|
||||
agentDir,
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ import {
|
|||
type ModelRef,
|
||||
} from "../agents/model-selection.js";
|
||||
import { createConfiguredOllamaStreamFn } from "../agents/ollama-stream.js";
|
||||
import { resolveModel } from "../agents/pi-embedded-runner/model.js";
|
||||
import { resolveModelAsync } from "../agents/pi-embedded-runner/model.js";
|
||||
import type { OpenClawConfig } from "../config/config.js";
|
||||
import type {
|
||||
ResolvedTtsConfig,
|
||||
|
|
@ -456,7 +456,7 @@ export async function summarizeText(params: {
|
|||
|
||||
const startTime = Date.now();
|
||||
const { ref } = resolveSummaryModelRef(cfg, config);
|
||||
const resolved = resolveModel(ref.provider, ref.model, undefined, cfg);
|
||||
const resolved = await resolveModelAsync(ref.provider, ref.model, undefined, cfg);
|
||||
if (!resolved.model) {
|
||||
throw new Error(resolved.error ?? `Unknown summary model: ${ref.provider}/${ref.model}`);
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue