fix(browser): add browser session selection

This commit is contained in:
Peter Steinberger 2026-03-14 03:46:34 +00:00
parent b857a8d8bc
commit 5c40c1c78a
No known key found for this signature in database
19 changed files with 575 additions and 36 deletions

View File

@ -11,6 +11,7 @@ Docs: https://docs.openclaw.ai
- Android/chat settings: redesign the chat settings sheet with grouped device and media sections, refresh the Connect and Voice tabs, and tighten the chat composer/session header for a denser mobile layout. (#44894) Thanks @obviyus. - Android/chat settings: redesign the chat settings sheet with grouped device and media sections, refresh the Connect and Voice tabs, and tighten the chat composer/session header for a denser mobile layout. (#44894) Thanks @obviyus.
- iOS/onboarding: add a first-run welcome pager before gateway setup, stop auto-opening the QR scanner, and show `/pair qr` instructions on the connect step. (#45054) Thanks @ngutman. - iOS/onboarding: add a first-run welcome pager before gateway setup, stop auto-opening the QR scanner, and show `/pair qr` instructions on the connect step. (#45054) Thanks @ngutman.
- Docker/timezone override: add `OPENCLAW_TZ` so `docker-setup.sh` can pin gateway and CLI containers to a chosen IANA timezone instead of inheriting the daemon default. (#34119) Thanks @Lanfei. - Docker/timezone override: add `OPENCLAW_TZ` so `docker-setup.sh` can pin gateway and CLI containers to a chosen IANA timezone instead of inheriting the daemon default. (#34119) Thanks @Lanfei.
- Browser/agents: add `browserSession="agent" | "user"` so agent browser calls can explicitly choose the isolated OpenClaw browser or a logged-in user browser, with docs for when user presence and attach approval are required.
### Fixes ### Fixes
@ -139,6 +140,7 @@ Docs: https://docs.openclaw.ai
- Mattermost/reply media delivery: pass agent-scoped `mediaLocalRoots` through shared reply delivery so allowed local files upload correctly from button, slash-command, and model-picker replies. (#44021) Thanks @LyleLiu666. - Mattermost/reply media delivery: pass agent-scoped `mediaLocalRoots` through shared reply delivery so allowed local files upload correctly from button, slash-command, and model-picker replies. (#44021) Thanks @LyleLiu666.
- Plugins/env-scoped roots: fix plugin discovery/load caches and provenance tracking so same-process `HOME`/`OPENCLAW_HOME` changes no longer reuse stale plugin state or misreport `~/...` plugins as untracked. (#44046) thanks @gumadeiras. - Plugins/env-scoped roots: fix plugin discovery/load caches and provenance tracking so same-process `HOME`/`OPENCLAW_HOME` changes no longer reuse stale plugin state or misreport `~/...` plugins as untracked. (#44046) thanks @gumadeiras.
- Gateway/session discovery: discover disk-only and retired ACP session stores under custom templated `session.store` roots so ACP reconciliation, session-id/session-label targeting, and run-id fallback keep working after restart. (#44176) thanks @gumadeiras. - Gateway/session discovery: discover disk-only and retired ACP session stores under custom templated `session.store` roots so ACP reconciliation, session-id/session-label targeting, and run-id fallback keep working after restart. (#44176) thanks @gumadeiras.
- Browser/existing-session: stop reporting fake CDP ports/URLs for live attached Chrome sessions, render `transport: chrome-mcp` in CLI/status output instead of `port: 0`, and keep timeout diagnostics transport-aware when no direct CDP URL exists.
- Models/OpenRouter native ids: canonicalize native OpenRouter model keys across config writes, runtime lookups, fallback management, and `models list --plain`, and migrate legacy duplicated `openrouter/openrouter/...` config entries forward on write. - Models/OpenRouter native ids: canonicalize native OpenRouter model keys across config writes, runtime lookups, fallback management, and `models list --plain`, and migrate legacy duplicated `openrouter/openrouter/...` config entries forward on write.
- Feishu/event dedupe: keep early duplicate suppression aligned with the shared Feishu message-id contract and release the pre-queue dedupe marker after failed dispatch so retried events can recover instead of being dropped until the short TTL expires. (#43762) Thanks @yunweibang. - Feishu/event dedupe: keep early duplicate suppression aligned with the shared Feishu message-id contract and release the pre-queue dedupe marker after failed dispatch so retried events can recover instead of being dropped until the short TTL expires. (#43762) Thanks @yunweibang.
- Gateway/hooks: bucket hook auth failures by forwarded client IP behind trusted proxies and warn when `hooks.allowedAgentIds` leaves hook routing unrestricted. - Gateway/hooks: bucket hook auth failures by forwarded client IP behind trusted proxies and warn when `hooks.allowedAgentIds` leaves hook routing unrestricted.

View File

@ -20,6 +20,13 @@ Back to the main browser docs: [Browser](/tools/browser).
OpenClaw controls a **dedicated Chrome profile** (named `openclaw`, orangetinted UI). This is separate from your daily browser profile. OpenClaw controls a **dedicated Chrome profile** (named `openclaw`, orangetinted UI). This is separate from your daily browser profile.
For agent browser tool calls:
- Default choice: the agent should use its isolated `openclaw` browser.
- Use the **user browser** only when existing logged-in sessions matter and the user is at the computer to click/approve any attach prompt.
- If you need to force the choice, use `browserSession="agent"` or `browserSession="user"`.
- If you have multiple user-browser profiles, specify the profile explicitly instead of guessing.
Two easy ways to access it: Two easy ways to access it:
1. **Ask the agent to open the browser** and then log in yourself. 1. **Ask the agent to open the browser** and then log in yourself.

View File

@ -51,6 +51,15 @@ Gateway.
- `existing-session`: official Chrome MCP attach flow for a running Chrome - `existing-session`: official Chrome MCP attach flow for a running Chrome
profile. profile.
For agent browser tool calls:
- Default: use the isolated `openclaw` browser.
- Use the **user browser** only when existing logged-in sessions matter and the
user is at the computer to click/approve any attach prompt.
- If you need to force the choice, use `browserSession="agent"` or
`browserSession="user"`.
- `profile` is the explicit override when you already know which profile to use.
Set `browser.defaultProfile: "openclaw"` if you want managed mode by default. Set `browser.defaultProfile: "openclaw"` if you want managed mode by default.
## Configuration ## Configuration
@ -70,7 +79,7 @@ Browser settings live in `~/.openclaw/openclaw.json`.
// cdpUrl: "http://127.0.0.1:18792", // legacy single-profile override // cdpUrl: "http://127.0.0.1:18792", // legacy single-profile override
remoteCdpTimeoutMs: 1500, // remote CDP HTTP timeout (ms) remoteCdpTimeoutMs: 1500, // remote CDP HTTP timeout (ms)
remoteCdpHandshakeTimeoutMs: 3000, // remote CDP WebSocket handshake timeout (ms) remoteCdpHandshakeTimeoutMs: 3000, // remote CDP WebSocket handshake timeout (ms)
defaultProfile: "chrome", defaultProfile: "openclaw",
color: "#FF4500", color: "#FF4500",
headless: false, headless: false,
noSandbox: false, noSandbox: false,
@ -79,8 +88,7 @@ Browser settings live in `~/.openclaw/openclaw.json`.
profiles: { profiles: {
openclaw: { cdpPort: 18800, color: "#FF4500" }, openclaw: { cdpPort: 18800, color: "#FF4500" },
work: { cdpPort: 18801, color: "#0066CC" }, work: { cdpPort: 18801, color: "#0066CC" },
chromeLive: { "chrome-live": {
cdpPort: 18802,
driver: "existing-session", driver: "existing-session",
attachOnly: true, attachOnly: true,
color: "#00AA00", color: "#00AA00",
@ -324,7 +332,7 @@ openclaw browser extension install
2. Use it: 2. Use it:
- CLI: `openclaw browser --browser-profile chrome tabs` - CLI: `openclaw browser --browser-profile chrome tabs`
- Agent tool: `browser` with `profile="chrome"` - Agent tool: `browser` with `browserSession="user"` (or `profile="chrome"`)
Optional: if you want a different name or relay port, create your own profile: Optional: if you want a different name or relay port, create your own profile:
@ -340,6 +348,8 @@ Notes:
- This mode relies on Playwright-on-CDP for most operations (screenshots/snapshots/actions). - This mode relies on Playwright-on-CDP for most operations (screenshots/snapshots/actions).
- Detach by clicking the extension icon again. - Detach by clicking the extension icon again.
- Agent use: prefer `browserSession="user"` for logged-in sites. The user must be
present to click the extension and attach the tab.
## Chrome existing-session via MCP ## Chrome existing-session via MCP
@ -379,6 +389,7 @@ openclaw browser --browser-profile chrome-live snapshot --format ai
What success looks like: What success looks like:
- `status` shows `driver: existing-session` - `status` shows `driver: existing-session`
- `status` shows `transport: chrome-mcp`
- `status` shows `running: true` - `status` shows `running: true`
- `tabs` lists your already-open Chrome tabs - `tabs` lists your already-open Chrome tabs
- `snapshot` returns refs from the selected live tab - `snapshot` returns refs from the selected live tab
@ -388,6 +399,14 @@ What to check if attach does not work:
- Chrome is version `144+` - Chrome is version `144+`
- remote debugging is enabled at `chrome://inspect/#remote-debugging` - remote debugging is enabled at `chrome://inspect/#remote-debugging`
- Chrome showed and you accepted the attach consent prompt - Chrome showed and you accepted the attach consent prompt
Agent use:
- Use `browserSession="user"` when you need the users logged-in browser state.
- If you know the profile name, pass `profile="chrome-live"` (or your custom
existing-session profile).
- Only choose this mode when the user is at the computer to approve the attach
prompt.
- the Gateway or node host can spawn `npx chrome-devtools-mcp@latest --autoConnect` - the Gateway or node host can spawn `npx chrome-devtools-mcp@latest --autoConnect`
Notes: Notes:

View File

@ -310,13 +310,18 @@ Profile management:
Common parameters: Common parameters:
- `browserSession` (`agent` | `user`)
- `profile` (optional; defaults to `browser.defaultProfile`) - `profile` (optional; defaults to `browser.defaultProfile`)
- `target` (`sandbox` | `host` | `node`) - `target` (`sandbox` | `host` | `node`)
- `node` (optional; picks a specific node id/name) - `node` (optional; picks a specific node id/name)
Notes: Notes:
- Requires `browser.enabled=true` (default is `true`; set `false` to disable). - Requires `browser.enabled=true` (default is `true`; set `false` to disable).
- `browserSession="agent"` is the safe default: isolated OpenClaw-managed browser.
- `browserSession="user"` means the real local host browser. Use it only when existing logins/cookies matter and the user is present to click/approve any attach prompt.
- `browserSession="user"` is host-only; do not combine it with sandbox/node targets.
- All actions accept optional `profile` parameter for multi-instance support. - All actions accept optional `profile` parameter for multi-instance support.
- When `profile` is omitted, uses `browser.defaultProfile` (defaults to "chrome"). - `profile` overrides `browserSession` when both are supplied.
- When `profile` is omitted, uses `browser.defaultProfile` (defaults to `openclaw`).
- Profile names: lowercase alphanumeric + hyphens only (max 64 chars). - Profile names: lowercase alphanumeric + hyphens only (max 64 chars).
- Port range: 18800-18899 (~100 profiles max). - Port range: 18800-18899 (~100 profiles max).
- Remote profiles are attach-only (no start/stop/reset). - Remote profiles are attach-only (no start/stop/reset).

View File

@ -160,6 +160,8 @@ describe("createOpenClawCodingTools", () => {
it("mentions Chrome extension relay in browser tool description", () => { it("mentions Chrome extension relay in browser tool description", () => {
const browser = createBrowserTool(); const browser = createBrowserTool();
expect(browser.description).toMatch(/Chrome extension/i); expect(browser.description).toMatch(/Chrome extension/i);
expect(browser.description).toMatch(/browserSession="agent"/i);
expect(browser.description).toMatch(/browserSession="user"/i);
expect(browser.description).toMatch(/profile="chrome"/i); expect(browser.description).toMatch(/profile="chrome"/i);
}); });
it("keeps browser tool schema properties after normalization", () => { it("keeps browser tool schema properties after normalization", () => {
@ -172,6 +174,7 @@ describe("createOpenClawCodingTools", () => {
}; };
expect(parameters.properties?.action).toBeDefined(); expect(parameters.properties?.action).toBeDefined();
expect(parameters.properties?.target).toBeDefined(); expect(parameters.properties?.target).toBeDefined();
expect(parameters.properties?.browserSession).toBeDefined();
expect(parameters.properties?.targetUrl).toBeDefined(); expect(parameters.properties?.targetUrl).toBeDefined();
expect(parameters.properties?.request).toBeDefined(); expect(parameters.properties?.request).toBeDefined();
expect(parameters.required ?? []).toContain("action"); expect(parameters.required ?? []).toContain("action");

View File

@ -35,6 +35,7 @@ const BROWSER_TOOL_ACTIONS = [
] as const; ] as const;
const BROWSER_TARGETS = ["sandbox", "host", "node"] as const; const BROWSER_TARGETS = ["sandbox", "host", "node"] as const;
const BROWSER_SESSION_CHOICES = ["agent", "user"] as const;
const BROWSER_SNAPSHOT_FORMATS = ["aria", "ai"] as const; const BROWSER_SNAPSHOT_FORMATS = ["aria", "ai"] as const;
const BROWSER_SNAPSHOT_MODES = ["efficient"] as const; const BROWSER_SNAPSHOT_MODES = ["efficient"] as const;
@ -88,6 +89,7 @@ const BrowserActSchema = Type.Object({
export const BrowserToolSchema = Type.Object({ export const BrowserToolSchema = Type.Object({
action: stringEnum(BROWSER_TOOL_ACTIONS), action: stringEnum(BROWSER_TOOL_ACTIONS),
target: optionalStringEnum(BROWSER_TARGETS), target: optionalStringEnum(BROWSER_TARGETS),
browserSession: optionalStringEnum(BROWSER_SESSION_CHOICES),
node: Type.Optional(Type.String()), node: Type.Optional(Type.String()),
profile: Type.Optional(Type.String()), profile: Type.Optional(Type.String()),
targetUrl: Type.Optional(Type.String()), targetUrl: Type.Optional(Type.String()),

View File

@ -54,7 +54,45 @@ const browserConfigMocks = vi.hoisted(() => ({
resolveBrowserConfig: vi.fn(() => ({ resolveBrowserConfig: vi.fn(() => ({
enabled: true, enabled: true,
controlPort: 18791, controlPort: 18791,
profiles: {},
defaultProfile: "openclaw",
})), })),
resolveProfile: vi.fn((resolved: Record<string, unknown>, name: string) => {
const profile = (resolved.profiles as Record<string, Record<string, unknown>> | undefined)?.[
name
];
if (!profile) {
return null;
}
const driver =
profile.driver === "extension"
? "extension"
: profile.driver === "existing-session"
? "existing-session"
: "openclaw";
if (driver === "existing-session") {
return {
name,
driver,
cdpPort: 0,
cdpUrl: "",
cdpHost: "",
cdpIsLoopback: true,
color: typeof profile.color === "string" ? profile.color : "#FF4500",
attachOnly: true,
};
}
return {
name,
driver,
cdpPort: typeof profile.cdpPort === "number" ? profile.cdpPort : 18792,
cdpUrl: typeof profile.cdpUrl === "string" ? profile.cdpUrl : "http://127.0.0.1:18792",
cdpHost: "127.0.0.1",
cdpIsLoopback: true,
color: typeof profile.color === "string" ? profile.color : "#FF4500",
attachOnly: profile.attachOnly === true,
};
}),
})); }));
vi.mock("../../browser/config.js", () => browserConfigMocks); vi.mock("../../browser/config.js", () => browserConfigMocks);
@ -117,9 +155,27 @@ function mockSingleBrowserProxyNode() {
function resetBrowserToolMocks() { function resetBrowserToolMocks() {
vi.clearAllMocks(); vi.clearAllMocks();
configMocks.loadConfig.mockReturnValue({ browser: {} }); configMocks.loadConfig.mockReturnValue({ browser: {} });
browserConfigMocks.resolveBrowserConfig.mockReturnValue({
enabled: true,
controlPort: 18791,
profiles: {},
defaultProfile: "openclaw",
});
nodesUtilsMocks.listNodes.mockResolvedValue([]); nodesUtilsMocks.listNodes.mockResolvedValue([]);
} }
function setResolvedBrowserProfiles(
profiles: Record<string, Record<string, unknown>>,
defaultProfile = "openclaw",
) {
browserConfigMocks.resolveBrowserConfig.mockReturnValue({
enabled: true,
controlPort: 18791,
profiles,
defaultProfile,
});
}
function registerBrowserToolAfterEachReset() { function registerBrowserToolAfterEachReset() {
afterEach(() => { afterEach(() => {
resetBrowserToolMocks(); resetBrowserToolMocks();
@ -131,6 +187,7 @@ async function runSnapshotToolCall(params: {
refs?: "aria" | "dom"; refs?: "aria" | "dom";
maxChars?: number; maxChars?: number;
profile?: string; profile?: string;
browserSession?: "agent" | "user";
}) { }) {
const tool = createBrowserTool(); const tool = createBrowserTool();
await tool.execute?.("call-1", { action: "snapshot", ...params }); await tool.execute?.("call-1", { action: "snapshot", ...params });
@ -243,6 +300,90 @@ describe("browser tool snapshot maxChars", () => {
); );
}); });
it('uses the isolated openclaw profile for browserSession="agent"', async () => {
await runSnapshotToolCall({ browserSession: "agent", snapshotFormat: "ai" });
expect(browserClientMocks.browserSnapshot).toHaveBeenCalledWith(
undefined,
expect.objectContaining({
profile: "openclaw",
}),
);
});
it('uses the host user browser for browserSession="user"', async () => {
setResolvedBrowserProfiles({
openclaw: { cdpPort: 18800, color: "#FF4500" },
chrome: { driver: "extension", cdpUrl: "http://127.0.0.1:18792", color: "#0066CC" },
});
const tool = createBrowserTool({ sandboxBridgeUrl: "http://127.0.0.1:9999" });
await tool.execute?.("call-1", {
action: "snapshot",
browserSession: "user",
snapshotFormat: "ai",
});
expect(browserClientMocks.browserSnapshot).toHaveBeenCalledWith(
undefined,
expect.objectContaining({
profile: "chrome",
}),
);
});
it('uses a sole existing-session profile for browserSession="user"', async () => {
setResolvedBrowserProfiles({
openclaw: { cdpPort: 18800, color: "#FF4500" },
"chrome-live": { driver: "existing-session", attachOnly: true, color: "#00AA00" },
});
const tool = createBrowserTool({ sandboxBridgeUrl: "http://127.0.0.1:9999" });
await tool.execute?.("call-1", {
action: "snapshot",
browserSession: "user",
snapshotFormat: "ai",
});
expect(browserClientMocks.browserSnapshot).toHaveBeenCalledWith(
undefined,
expect.objectContaining({
profile: "chrome-live",
}),
);
});
it('fails when browserSession="user" is ambiguous', async () => {
setResolvedBrowserProfiles({
openclaw: { cdpPort: 18800, color: "#FF4500" },
personal: { driver: "existing-session", attachOnly: true, color: "#00AA00" },
work: { driver: "existing-session", attachOnly: true, color: "#0066CC" },
});
const tool = createBrowserTool();
await expect(
tool.execute?.("call-1", {
action: "snapshot",
browserSession: "user",
snapshotFormat: "ai",
}),
).rejects.toThrow(/Multiple user-browser profiles are configured/);
});
it('rejects browserSession="user" with target="sandbox"', async () => {
setResolvedBrowserProfiles({
chrome: { driver: "extension", cdpUrl: "http://127.0.0.1:18792", color: "#0066CC" },
});
const tool = createBrowserTool({ sandboxBridgeUrl: "http://127.0.0.1:9999" });
await expect(
tool.execute?.("call-1", {
action: "snapshot",
browserSession: "user",
target: "sandbox",
snapshotFormat: "ai",
}),
).rejects.toThrow(/cannot use the sandbox browser/);
});
it("lets the server choose snapshot format when the user does not request one", async () => { it("lets the server choose snapshot format when the user does not request one", async () => {
const tool = createBrowserTool(); const tool = createBrowserTool();
await tool.execute?.("call-1", { action: "snapshot", profile: "chrome" }); await tool.execute?.("call-1", { action: "snapshot", profile: "chrome" });

View File

@ -16,8 +16,10 @@ import {
browserStatus, browserStatus,
browserStop, browserStop,
} from "../../browser/client.js"; } from "../../browser/client.js";
import { resolveBrowserConfig } from "../../browser/config.js"; import { resolveBrowserConfig, resolveProfile } from "../../browser/config.js";
import { DEFAULT_OPENCLAW_BROWSER_PROFILE_NAME } from "../../browser/constants.js";
import { DEFAULT_UPLOAD_DIR, resolveExistingPathsWithinRoot } from "../../browser/paths.js"; import { DEFAULT_UPLOAD_DIR, resolveExistingPathsWithinRoot } from "../../browser/paths.js";
import { getBrowserProfileCapabilities } from "../../browser/profile-capabilities.js";
import { applyBrowserProxyPaths, persistBrowserProxyFiles } from "../../browser/proxy-files.js"; import { applyBrowserProxyPaths, persistBrowserProxyFiles } from "../../browser/proxy-files.js";
import { import {
trackSessionBrowserTab, trackSessionBrowserTab,
@ -278,6 +280,60 @@ function resolveBrowserBaseUrl(params: {
return undefined; return undefined;
} }
function listUserBrowserProfiles() {
const cfg = loadConfig();
const resolved = resolveBrowserConfig(cfg.browser, cfg);
return Object.keys(resolved.profiles ?? {})
.map((name) => resolveProfile(resolved, name))
.filter((profile): profile is NonNullable<typeof profile> => Boolean(profile))
.filter((profile) => {
const capabilities = getBrowserProfileCapabilities(profile);
return capabilities.requiresRelay || capabilities.usesChromeMcp;
});
}
function resolveBrowserToolProfile(params: {
profile?: string;
browserSession?: "agent" | "user";
}): string | undefined {
if (params.profile) {
return params.profile;
}
if (!params.browserSession) {
return undefined;
}
if (params.browserSession === "agent") {
return DEFAULT_OPENCLAW_BROWSER_PROFILE_NAME;
}
const userProfiles = listUserBrowserProfiles();
const defaultUserProfile = userProfiles.find(
(profile) => profile.name !== DEFAULT_OPENCLAW_BROWSER_PROFILE_NAME,
);
if (defaultUserProfile?.name === "chrome") {
return defaultUserProfile.name;
}
const chromeRelay = userProfiles.find((profile) => profile.name === "chrome");
if (chromeRelay) {
return chromeRelay.name;
}
if (userProfiles.length === 1) {
return userProfiles[0]?.name;
}
const chromeLive = userProfiles.find((profile) => profile.name === "chrome-live");
if (chromeLive) {
return chromeLive.name;
}
if (userProfiles.length === 0) {
throw new Error(
'No user-browser profile is configured. Use profile="chrome" for the extension relay or create an existing-session profile first.',
);
}
throw new Error(
`Multiple user-browser profiles are configured (${userProfiles.map((profile) => profile.name).join(", ")}). Pass profile="<name>".`,
);
}
export function createBrowserTool(opts?: { export function createBrowserTool(opts?: {
sandboxBridgeUrl?: string; sandboxBridgeUrl?: string;
allowHostControl?: boolean; allowHostControl?: boolean;
@ -291,10 +347,12 @@ export function createBrowserTool(opts?: {
name: "browser", name: "browser",
description: [ description: [
"Control the browser via OpenClaw's browser control server (status/start/stop/profiles/tabs/open/snapshot/screenshot/actions).", "Control the browser via OpenClaw's browser control server (status/start/stop/profiles/tabs/open/snapshot/screenshot/actions).",
'Profiles: use profile="chrome" for Chrome extension relay takeover (your existing Chrome tabs). Use profile="openclaw" for the isolated openclaw-managed browser.', 'Browser choice: use browserSession="agent" by default for the isolated OpenClaw browser. Use browserSession="user" only when logged-in browser state matters and the user is present to click/approve browser attach prompts.',
'If the user mentions the Chrome extension / Browser Relay / toolbar button / “attach tab”, ALWAYS use profile="chrome" (do not ask which profile).', 'browserSession="user" means the real local user browser on the host, not sandbox/node browsers. If user presence is unclear, ask first.',
'profile remains the explicit override. Use profile="chrome" for Chrome extension relay takeover (existing Chrome tabs). Use profile="openclaw" for the isolated OpenClaw-managed browser.',
'If the user mentions the Chrome extension / Browser Relay / toolbar button / “attach tab”, ALWAYS use browserSession="user" and prefer profile="chrome" (do not ask which profile unless ambiguous).',
'When a node-hosted browser proxy is available, the tool may auto-route to it. Pin a node with node=<id|name> or target="node".', 'When a node-hosted browser proxy is available, the tool may auto-route to it. Pin a node with node=<id|name> or target="node".',
"Chrome extension relay needs an attached tab: user must click the OpenClaw Browser Relay toolbar icon on the tab (badge ON). If no tab is connected, ask them to attach it.", "User-browser flows need user interaction: Chrome extension relay needs the user to click the OpenClaw Browser Relay toolbar icon on the tab (badge ON); existing-session may require approving a browser attach prompt.",
"When using refs from snapshot (e.g. e12), keep the same tab: prefer passing targetId from the snapshot response into subsequent actions (act/click/type/etc).", "When using refs from snapshot (e.g. e12), keep the same tab: prefer passing targetId from the snapshot response into subsequent actions (act/click/type/etc).",
'For stable, self-resolving refs across calls, use snapshot with refs="aria" (Playwright aria-ref ids). Default refs="role" are role+name-based.', 'For stable, self-resolving refs across calls, use snapshot with refs="aria" (Playwright aria-ref ids). Default refs="role" are role+name-based.',
"Use snapshot+act for UI automation. Avoid act:wait by default; use only in exceptional cases when no reliable UI state exists.", "Use snapshot+act for UI automation. Avoid act:wait by default; use only in exceptional cases when no reliable UI state exists.",
@ -305,13 +363,33 @@ export function createBrowserTool(opts?: {
execute: async (_toolCallId, args) => { execute: async (_toolCallId, args) => {
const params = args as Record<string, unknown>; const params = args as Record<string, unknown>;
const action = readStringParam(params, "action", { required: true }); const action = readStringParam(params, "action", { required: true });
const profile = readStringParam(params, "profile"); const browserSession = readStringParam(params, "browserSession") as
| "agent"
| "user"
| undefined;
const profile = resolveBrowserToolProfile({
profile: readStringParam(params, "profile"),
browserSession,
});
const requestedNode = readStringParam(params, "node"); const requestedNode = readStringParam(params, "node");
let target = readStringParam(params, "target") as "sandbox" | "host" | "node" | undefined; let target = readStringParam(params, "target") as "sandbox" | "host" | "node" | undefined;
if (requestedNode && target && target !== "node") { if (requestedNode && target && target !== "node") {
throw new Error('node is only supported with target="node".'); throw new Error('node is only supported with target="node".');
} }
if (browserSession === "user") {
if (requestedNode || target === "node") {
throw new Error('browserSession="user" only supports the local host browser.');
}
if (target === "sandbox") {
throw new Error(
'browserSession="user" cannot use the sandbox browser; use target="host" or omit target.',
);
}
}
if (!target && !requestedNode && browserSession === "user") {
target = "host";
}
if (!target && !requestedNode && profile === "chrome") { if (!target && !requestedNode && profile === "chrome") {
// Chrome extension relay takeover is a host Chrome feature; prefer host unless explicitly targeting a node. // Chrome extension relay takeover is a host Chrome feature; prefer host unless explicitly targeting a node.

View File

@ -1,15 +1,18 @@
import { fetchBrowserJson } from "./client-fetch.js"; import { fetchBrowserJson } from "./client-fetch.js";
export type BrowserTransport = "cdp" | "chrome-mcp";
export type BrowserStatus = { export type BrowserStatus = {
enabled: boolean; enabled: boolean;
profile?: string; profile?: string;
driver?: "openclaw" | "extension" | "existing-session"; driver?: "openclaw" | "extension" | "existing-session";
transport?: BrowserTransport;
running: boolean; running: boolean;
cdpReady?: boolean; cdpReady?: boolean;
cdpHttp?: boolean; cdpHttp?: boolean;
pid: number | null; pid: number | null;
cdpPort: number; cdpPort: number | null;
cdpUrl?: string; cdpUrl?: string | null;
chosenBrowser: string | null; chosenBrowser: string | null;
detectedBrowser?: string | null; detectedBrowser?: string | null;
detectedExecutablePath?: string | null; detectedExecutablePath?: string | null;
@ -24,8 +27,9 @@ export type BrowserStatus = {
export type ProfileStatus = { export type ProfileStatus = {
name: string; name: string;
cdpPort: number; transport?: BrowserTransport;
cdpUrl: string; cdpPort: number | null;
cdpUrl: string | null;
color: string; color: string;
driver: "openclaw" | "extension" | "existing-session"; driver: "openclaw" | "extension" | "existing-session";
running: boolean; running: boolean;
@ -155,8 +159,9 @@ export async function browserResetProfile(
export type BrowserCreateProfileResult = { export type BrowserCreateProfileResult = {
ok: true; ok: true;
profile: string; profile: string;
cdpPort: number; transport?: BrowserTransport;
cdpUrl: string; cdpPort: number | null;
cdpUrl: string | null;
color: string; color: string;
isRemote: boolean; isRemote: boolean;
}; };

View File

@ -178,7 +178,9 @@ describe("BrowserProfilesService", () => {
driver: "existing-session", driver: "existing-session",
}); });
expect(result.cdpPort).toBe(0); expect(result.transport).toBe("chrome-mcp");
expect(result.cdpPort).toBeNull();
expect(result.cdpUrl).toBeNull();
expect(result.isRemote).toBe(false); expect(result.isRemote).toBe(false);
expect(state.resolved.profiles["chrome-live"]).toEqual({ expect(state.resolved.profiles["chrome-live"]).toEqual({
driver: "existing-session", driver: "existing-session",

View File

@ -12,6 +12,7 @@ import {
BrowserResourceExhaustedError, BrowserResourceExhaustedError,
BrowserValidationError, BrowserValidationError,
} from "./errors.js"; } from "./errors.js";
import { getBrowserProfileCapabilities } from "./profile-capabilities.js";
import { import {
allocateCdpPort, allocateCdpPort,
allocateColor, allocateColor,
@ -32,8 +33,9 @@ export type CreateProfileParams = {
export type CreateProfileResult = { export type CreateProfileResult = {
ok: true; ok: true;
profile: string; profile: string;
cdpPort: number; transport: "cdp" | "chrome-mcp";
cdpUrl: string; cdpPort: number | null;
cdpUrl: string | null;
color: string; color: string;
isRemote: boolean; isRemote: boolean;
}; };
@ -181,12 +183,14 @@ export function createBrowserProfilesService(ctx: BrowserRouteContext) {
if (!resolved) { if (!resolved) {
throw new BrowserProfileNotFoundError(`profile "${name}" not found after creation`); throw new BrowserProfileNotFoundError(`profile "${name}" not found after creation`);
} }
const capabilities = getBrowserProfileCapabilities(resolved);
return { return {
ok: true, ok: true,
profile: name, profile: name,
cdpPort: resolved.cdpPort, transport: capabilities.usesChromeMcp ? "chrome-mcp" : "cdp",
cdpUrl: resolved.cdpUrl, cdpPort: capabilities.usesChromeMcp ? null : resolved.cdpPort,
cdpUrl: capabilities.usesChromeMcp ? null : resolved.cdpUrl,
color: resolved.color, color: resolved.color,
isRemote: !resolved.cdpIsLoopback, isRemote: !resolved.cdpIsLoopback,
}; };

View File

@ -1,8 +1,12 @@
import { describe, expect, it } from "vitest"; import { describe, expect, it, vi } from "vitest";
import { BrowserProfileUnavailableError } from "../errors.js"; import { BrowserProfileUnavailableError } from "../errors.js";
import { registerBrowserBasicRoutes } from "./basic.js"; import { registerBrowserBasicRoutes } from "./basic.js";
import { createBrowserRouteApp, createBrowserRouteResponse } from "./test-helpers.js"; import { createBrowserRouteApp, createBrowserRouteResponse } from "./test-helpers.js";
vi.mock("../chrome-mcp.js", () => ({
getChromeMcpPid: vi.fn(() => 4321),
}));
describe("basic browser routes", () => { describe("basic browser routes", () => {
it("maps existing-session status failures to JSON browser errors", async () => { it("maps existing-session status failures to JSON browser errors", async () => {
const { app, getHandlers } = createBrowserRouteApp(); const { app, getHandlers } = createBrowserRouteApp();
@ -21,8 +25,8 @@ describe("basic browser routes", () => {
profile: { profile: {
name: "chrome-live", name: "chrome-live",
driver: "existing-session", driver: "existing-session",
cdpPort: 18802, cdpPort: 0,
cdpUrl: "http://127.0.0.1:18802", cdpUrl: "",
color: "#00AA00", color: "#00AA00",
attachOnly: true, attachOnly: true,
}, },
@ -42,4 +46,49 @@ describe("basic browser routes", () => {
expect(response.statusCode).toBe(409); expect(response.statusCode).toBe(409);
expect(response.body).toMatchObject({ error: "attach failed" }); expect(response.body).toMatchObject({ error: "attach failed" });
}); });
it("reports Chrome MCP transport without fake CDP fields", async () => {
const { app, getHandlers } = createBrowserRouteApp();
registerBrowserBasicRoutes(app, {
state: () => ({
resolved: {
enabled: true,
headless: false,
noSandbox: false,
executablePath: undefined,
},
profiles: new Map(),
}),
forProfile: () =>
({
profile: {
name: "chrome-live",
driver: "existing-session",
cdpPort: 0,
cdpUrl: "",
color: "#00AA00",
attachOnly: true,
},
isHttpReachable: async () => true,
isReachable: async () => true,
}) as never,
} as never);
const handler = getHandlers.get("/");
expect(handler).toBeTypeOf("function");
const response = createBrowserRouteResponse();
await handler?.({ params: {}, query: { profile: "chrome-live" } }, response.res);
expect(response.statusCode).toBe(200);
expect(response.body).toMatchObject({
profile: "chrome-live",
driver: "existing-session",
transport: "chrome-mcp",
running: true,
cdpPort: null,
cdpUrl: null,
pid: 4321,
});
});
}); });

View File

@ -80,6 +80,7 @@ export function registerBrowserBasicRoutes(app: BrowserRouteRegistrar, ctx: Brow
]); ]);
const profileState = current.profiles.get(profileCtx.profile.name); const profileState = current.profiles.get(profileCtx.profile.name);
const capabilities = getBrowserProfileCapabilities(profileCtx.profile);
let detectedBrowser: string | null = null; let detectedBrowser: string | null = null;
let detectedExecutablePath: string | null = null; let detectedExecutablePath: string | null = null;
let detectError: string | null = null; let detectError: string | null = null;
@ -98,14 +99,15 @@ export function registerBrowserBasicRoutes(app: BrowserRouteRegistrar, ctx: Brow
enabled: current.resolved.enabled, enabled: current.resolved.enabled,
profile: profileCtx.profile.name, profile: profileCtx.profile.name,
driver: profileCtx.profile.driver, driver: profileCtx.profile.driver,
transport: capabilities.usesChromeMcp ? "chrome-mcp" : "cdp",
running: cdpReady, running: cdpReady,
cdpReady, cdpReady,
cdpHttp, cdpHttp,
pid: getBrowserProfileCapabilities(profileCtx.profile).usesChromeMcp pid: capabilities.usesChromeMcp
? getChromeMcpPid(profileCtx.profile.name) ? getChromeMcpPid(profileCtx.profile.name)
: (profileState?.running?.pid ?? null), : (profileState?.running?.pid ?? null),
cdpPort: profileCtx.profile.cdpPort, cdpPort: capabilities.usesChromeMcp ? null : profileCtx.profile.cdpPort,
cdpUrl: profileCtx.profile.cdpUrl, cdpUrl: capabilities.usesChromeMcp ? null : profileCtx.profile.cdpUrl,
chosenBrowser: profileState?.running?.exe.kind ?? null, chosenBrowser: profileState?.running?.exe.kind ?? null,
detectedBrowser, detectedBrowser,
detectedExecutablePath, detectedExecutablePath,

View File

@ -160,12 +160,13 @@ export function createBrowserRouteContext(opts: ContextOptions): BrowserRouteCon
if (!profile) { if (!profile) {
continue; continue;
} }
const capabilities = getBrowserProfileCapabilities(profile);
let tabCount = 0; let tabCount = 0;
let running = false; let running = false;
const profileCtx = createProfileContext(opts, profile); const profileCtx = createProfileContext(opts, profile);
if (getBrowserProfileCapabilities(profile).usesChromeMcp) { if (capabilities.usesChromeMcp) {
try { try {
running = await profileCtx.isReachable(300); running = await profileCtx.isReachable(300);
if (running) { if (running) {
@ -199,8 +200,9 @@ export function createBrowserRouteContext(opts: ContextOptions): BrowserRouteCon
result.push({ result.push({
name, name,
cdpPort: profile.cdpPort, transport: capabilities.usesChromeMcp ? "chrome-mcp" : "cdp",
cdpUrl: profile.cdpUrl, cdpPort: capabilities.usesChromeMcp ? null : profile.cdpPort,
cdpUrl: capabilities.usesChromeMcp ? null : profile.cdpUrl,
color: profile.color, color: profile.color,
driver: profile.driver, driver: profile.driver,
running, running,

View File

@ -1,5 +1,6 @@
import type { Server } from "node:http"; import type { Server } from "node:http";
import type { RunningChrome } from "./chrome.js"; import type { RunningChrome } from "./chrome.js";
import type { BrowserTransport } from "./client.js";
import type { BrowserTab } from "./client.js"; import type { BrowserTab } from "./client.js";
import type { ResolvedBrowserConfig, ResolvedBrowserProfile } from "./config.js"; import type { ResolvedBrowserConfig, ResolvedBrowserProfile } from "./config.js";
@ -53,8 +54,9 @@ export type ProfileContext = {
export type ProfileStatus = { export type ProfileStatus = {
name: string; name: string;
cdpPort: number; transport: BrowserTransport;
cdpUrl: string; cdpPort: number | null;
cdpUrl: string | null;
color: string; color: string;
driver: ResolvedBrowserProfile["driver"]; driver: ResolvedBrowserProfile["driver"];
running: boolean; running: boolean;

View File

@ -0,0 +1,151 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { registerBrowserManageCommands } from "./browser-cli-manage.js";
import { createBrowserProgram } from "./browser-cli-test-helpers.js";
const mocks = vi.hoisted(() => {
const runtimeLog = vi.fn();
const runtimeError = vi.fn();
const runtimeExit = vi.fn();
return {
callBrowserRequest: vi.fn<
(
opts: unknown,
req: { path?: string },
runtimeOpts?: { timeoutMs?: number },
) => Promise<Record<string, unknown>>
>(async () => ({})),
runtimeLog,
runtimeError,
runtimeExit,
runtime: {
log: runtimeLog,
error: runtimeError,
exit: runtimeExit,
},
};
});
vi.mock("./browser-cli-shared.js", () => ({
callBrowserRequest: mocks.callBrowserRequest,
}));
vi.mock("./cli-utils.js", () => ({
runCommandWithRuntime: async (
_runtime: unknown,
action: () => Promise<void>,
onError: (err: unknown) => void,
) => await action().catch(onError),
}));
vi.mock("../runtime.js", () => ({
defaultRuntime: mocks.runtime,
}));
function createProgram() {
const { program, browser, parentOpts } = createBrowserProgram();
registerBrowserManageCommands(browser, parentOpts);
return program;
}
describe("browser manage output", () => {
beforeEach(() => {
mocks.callBrowserRequest.mockClear();
mocks.runtimeLog.mockClear();
mocks.runtimeError.mockClear();
mocks.runtimeExit.mockClear();
});
it("shows chrome-mcp transport for existing-session status without fake CDP fields", async () => {
mocks.callBrowserRequest.mockImplementation(async (_opts: unknown, req: { path?: string }) =>
req.path === "/"
? {
enabled: true,
profile: "chrome-live",
driver: "existing-session",
transport: "chrome-mcp",
running: true,
cdpReady: true,
cdpHttp: true,
pid: 4321,
cdpPort: null,
cdpUrl: null,
chosenBrowser: null,
userDataDir: null,
color: "#00AA00",
headless: false,
noSandbox: false,
executablePath: null,
attachOnly: true,
}
: {},
);
const program = createProgram();
await program.parseAsync(["browser", "--browser-profile", "chrome-live", "status"], {
from: "user",
});
const output = mocks.runtimeLog.mock.calls.at(-1)?.[0] as string;
expect(output).toContain("transport: chrome-mcp");
expect(output).not.toContain("cdpPort:");
expect(output).not.toContain("cdpUrl:");
});
it("shows chrome-mcp transport in browser profiles output", async () => {
mocks.callBrowserRequest.mockImplementation(async (_opts: unknown, req: { path?: string }) =>
req.path === "/profiles"
? {
profiles: [
{
name: "chrome-live",
driver: "existing-session",
transport: "chrome-mcp",
running: true,
tabCount: 2,
isDefault: false,
isRemote: false,
cdpPort: null,
cdpUrl: null,
color: "#00AA00",
},
],
}
: {},
);
const program = createProgram();
await program.parseAsync(["browser", "profiles"], { from: "user" });
const output = mocks.runtimeLog.mock.calls.at(-1)?.[0] as string;
expect(output).toContain("chrome-live: running (2 tabs) [existing-session]");
expect(output).toContain("transport: chrome-mcp");
expect(output).not.toContain("port: 0");
});
it("shows chrome-mcp transport after creating an existing-session profile", async () => {
mocks.callBrowserRequest.mockImplementation(async (_opts: unknown, req: { path?: string }) =>
req.path === "/profiles/create"
? {
ok: true,
profile: "chrome-live",
transport: "chrome-mcp",
cdpPort: null,
cdpUrl: null,
color: "#00AA00",
isRemote: false,
}
: {},
);
const program = createProgram();
await program.parseAsync(
["browser", "create-profile", "--name", "chrome-live", "--driver", "existing-session"],
{ from: "user" },
);
const output = mocks.runtimeLog.mock.calls.at(-1)?.[0] as string;
expect(output).toContain('Created profile "chrome-live"');
expect(output).toContain("transport: chrome-mcp");
expect(output).not.toContain("port: 0");
});
});

View File

@ -1,5 +1,6 @@
import type { Command } from "commander"; import type { Command } from "commander";
import type { import type {
BrowserTransport,
BrowserCreateProfileResult, BrowserCreateProfileResult,
BrowserDeleteProfileResult, BrowserDeleteProfileResult,
BrowserResetProfileResult, BrowserResetProfileResult,
@ -101,6 +102,29 @@ function logBrowserTabs(tabs: BrowserTab[], json?: boolean) {
); );
} }
function usesChromeMcpTransport(params: {
transport?: BrowserTransport;
driver?: "openclaw" | "extension" | "existing-session";
}): boolean {
return params.transport === "chrome-mcp" || params.driver === "existing-session";
}
function formatBrowserConnectionSummary(params: {
transport?: BrowserTransport;
driver?: "openclaw" | "extension" | "existing-session";
isRemote?: boolean;
cdpPort?: number | null;
cdpUrl?: string | null;
}): string {
if (usesChromeMcpTransport(params)) {
return "transport: chrome-mcp";
}
if (params.isRemote) {
return `cdpUrl: ${params.cdpUrl ?? "(unset)"}`;
}
return `port: ${params.cdpPort ?? "(unset)"}`;
}
export function registerBrowserManageCommands( export function registerBrowserManageCommands(
browser: Command, browser: Command,
parentOpts: (cmd: Command) => BrowserParentOpts, parentOpts: (cmd: Command) => BrowserParentOpts,
@ -122,8 +146,15 @@ export function registerBrowserManageCommands(
`profile: ${status.profile ?? "openclaw"}`, `profile: ${status.profile ?? "openclaw"}`,
`enabled: ${status.enabled}`, `enabled: ${status.enabled}`,
`running: ${status.running}`, `running: ${status.running}`,
`cdpPort: ${status.cdpPort}`, `transport: ${
`cdpUrl: ${status.cdpUrl ?? `http://127.0.0.1:${status.cdpPort}`}`, usesChromeMcpTransport(status) ? "chrome-mcp" : (status.transport ?? "cdp")
}`,
...(!usesChromeMcpTransport(status)
? [
`cdpPort: ${status.cdpPort ?? "(unset)"}`,
`cdpUrl: ${status.cdpUrl ?? `http://127.0.0.1:${status.cdpPort}`}`,
]
: []),
`browser: ${status.chosenBrowser ?? "unknown"}`, `browser: ${status.chosenBrowser ?? "unknown"}`,
`detectedBrowser: ${status.detectedBrowser ?? "unknown"}`, `detectedBrowser: ${status.detectedBrowser ?? "unknown"}`,
`detectedPath: ${detectedDisplay}`, `detectedPath: ${detectedDisplay}`,
@ -407,7 +438,7 @@ export function registerBrowserManageCommands(
const status = p.running ? "running" : "stopped"; const status = p.running ? "running" : "stopped";
const tabs = p.running ? ` (${p.tabCount} tabs)` : ""; const tabs = p.running ? ` (${p.tabCount} tabs)` : "";
const def = p.isDefault ? " [default]" : ""; const def = p.isDefault ? " [default]" : "";
const loc = p.isRemote ? `cdpUrl: ${p.cdpUrl}` : `port: ${p.cdpPort}`; const loc = formatBrowserConnectionSummary(p);
const remote = p.isRemote ? " [remote]" : ""; const remote = p.isRemote ? " [remote]" : "";
const driver = p.driver !== "openclaw" ? ` [${p.driver}]` : ""; const driver = p.driver !== "openclaw" ? ` [${p.driver}]` : "";
return `${p.name}: ${status}${tabs}${def}${remote}${driver}\n ${loc}, color: ${p.color}`; return `${p.name}: ${status}${tabs}${def}${remote}${driver}\n ${loc}, color: ${p.color}`;
@ -453,7 +484,7 @@ export function registerBrowserManageCommands(
if (printJsonResult(parent, result)) { if (printJsonResult(parent, result)) {
return; return;
} }
const loc = result.isRemote ? ` cdpUrl: ${result.cdpUrl}` : ` port: ${result.cdpPort}`; const loc = ` ${formatBrowserConnectionSummary(result)}`;
defaultRuntime.log( defaultRuntime.log(
info( info(
`🦞 Created profile "${result.profile}"\n${loc}\n color: ${result.color}${ `🦞 Created profile "${result.profile}"\n${loc}\n color: ${result.color}${

View File

@ -79,6 +79,36 @@ describe("runBrowserProxyCommand", () => {
); );
}); });
it("includes chrome-mcp transport in timeout diagnostics when no CDP URL exists", async () => {
dispatcherMocks.dispatch
.mockImplementationOnce(async () => {
await new Promise(() => {});
})
.mockResolvedValueOnce({
status: 200,
body: {
running: true,
transport: "chrome-mcp",
cdpHttp: true,
cdpReady: false,
cdpUrl: null,
},
});
await expect(
runBrowserProxyCommand(
JSON.stringify({
method: "GET",
path: "/snapshot",
profile: "chrome-live",
timeoutMs: 5,
}),
),
).rejects.toThrow(
/browser proxy timed out for GET \/snapshot after 5ms; ws-backed browser action; profile=chrome-live; status\(running=true, cdpHttp=true, cdpReady=false, transport=chrome-mcp\)/,
);
});
it("keeps non-timeout browser errors intact", async () => { it("keeps non-timeout browser errors intact", async () => {
dispatcherMocks.dispatch.mockResolvedValue({ dispatcherMocks.dispatch.mockResolvedValue({
status: 500, status: 500,

View File

@ -164,6 +164,7 @@ async function readBrowserProxyStatus(params: {
const body = response.body as Record<string, unknown>; const body = response.body as Record<string, unknown>;
return { return {
running: body.running, running: body.running,
transport: body.transport,
cdpHttp: body.cdpHttp, cdpHttp: body.cdpHttp,
cdpReady: body.cdpReady, cdpReady: body.cdpReady,
cdpUrl: body.cdpUrl, cdpUrl: body.cdpUrl,
@ -194,6 +195,9 @@ function formatBrowserProxyTimeoutMessage(params: {
`cdpHttp=${String(params.status.cdpHttp)}`, `cdpHttp=${String(params.status.cdpHttp)}`,
`cdpReady=${String(params.status.cdpReady)}`, `cdpReady=${String(params.status.cdpReady)}`,
]; ];
if (typeof params.status.transport === "string" && params.status.transport.trim()) {
statusParts.push(`transport=${params.status.transport}`);
}
if (typeof params.status.cdpUrl === "string" && params.status.cdpUrl.trim()) { if (typeof params.status.cdpUrl === "string" && params.status.cdpUrl.trim()) {
statusParts.push(`cdpUrl=${params.status.cdpUrl}`); statusParts.push(`cdpUrl=${params.status.cdpUrl}`);
} }