fix(matrix): split partial and quiet preview streaming (#61450)

Merged via squash.

Prepared head SHA: 6a0d7d1348
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
This commit is contained in:
Gustavo Madeira Santana 2026-04-05 18:23:07 -04:00 committed by GitHub
parent 1582bbbfc5
commit 8a841b531f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
16 changed files with 682 additions and 114 deletions

View File

@ -110,6 +110,34 @@ Docs: https://docs.openclaw.ai
- Config/All Settings: keep the raw config view intact when sensitive fields are blank instead of corrupting or dropping the rendered snapshot. (#28214) Thanks @solodmd.
- Plugin SDK/facades: back-fill bundled plugin facade sentinels before plugin-id tracking re-enters config loading, so CLI/provider startup no longer crashes with `shouldNormalizeGoogleProviderConfig is not a function` or other empty-facade reads during bundled plugin re-entry. Thanks @adam91holt.
- Plugins/facades: back-fill facade sentinels before tracked-plugin resolution re-enters config loading, so facade exports stay defined during circular provider normalization. (#61180) Thanks @adam91holt.
- Discord/image generation: include the real generated `MEDIA:` paths in tool output and avoid duplicate plain-output media requeueing so Discord image replies stop pointing at missing local files.
- Slack: route live DM replies back to the concrete inbound DM channel while keeping persisted routing metadata user-scoped, so normal assistant replies stop disappearing when pairing and system messages still arrive. (#59030) Thanks @afurm.
- Discord/reply tags: strip leaked `[[reply_to_current]]` control tags from preview text and honor explicit reply-tag threading during final delivery, so Discord replies stay attached to the triggering message instead of printing reply metadata into chat.
- Telegram: fix current-model checks in the model picker, HTML-format non-default `/model` confirmations, explicit topic replies, persisted reaction ownership across restarts, caption-media placeholder and `file_id` preservation on download failure, and upgraded-install inbound image reads. (#60384, #60042, #59634, #59207, #59948, #59971) Thanks @sfuminya, @GitZhangChi, @dashhuang, @samzong, @v1p0r, and @neeravmakwana.
- Telegram: restore DM voice-note preflight transcription so direct-message audio stops arriving as raw `<media:audio>` placeholders. (#61008) Thanks @manueltarouca.
- Telegram/reasoning: only create a Telegram reasoning preview lane when the session is explicitly `reasoning:stream`, so hidden `<think>` traces from streamed replies stop surfacing as chat previews on normal sessions. Thanks @vincentkoc.
- Telegram/native command menu: trim long menu descriptions before dropping commands so sub-100 command sets can still fit Telegram's payload budget and keep more `/` entries visible. (#61129) Thanks @neeravmakwana.
- Feishu/reasoning: only expose streamed reasoning previews when the session is explicitly `reasoning:stream`, so hidden reasoning traces do not surface on normal streaming sessions. Thanks @vincentkoc.
- Discord: keep REST, webhook, and monitor traffic on the configured proxy, preserve component-only media sends, honor `@everyone` and `@here` mention gates, keep ACK reactions on the active account, and split voice connect/playback timeouts so auto-join is more reliable. (#57465, #60361, #60345) Thanks @geekhuashan.
- WhatsApp: restore `channels.whatsapp.blockStreaming` and reset watchdog timeouts after reconnect so quiet chats stop falling into reconnect loops. (#60007, #60069) Thanks @MonkeyLeeT and @mcaxtr.
- Memory: keep `memory-core` builtin embedding registration on the already-registered path so selecting `memory-core` no longer recurses through plugin discovery and crashes during startup. (#61402) Thanks @ngutman.
- MS Teams: download inline DM images via Graph API and preserve channel reply threading in proactive fallback. (#52212, #55198) Thanks @Ted-developer and @hyojin.
- MS Teams: replace the deprecated Teams SDK HttpPlugin stub with `httpServerAdapter` so recurring gateway deprecation warnings stop firing and the Express 5 compatibility workaround stays on the supported SDK path. (#60939) Thanks @coolramukaka-sys.
- Matrix/exec approvals: anchor seeded approval reactions to the primary Matrix prompt event, resolve them from event metadata instead of prompt text, and clean up chunked approval prompts correctly. (#60931) Thanks @gumadeiras.
- Matrix: recover more reliably when secret storage or recovery keys are missing by recreating secret storage during repair and backup reset, hold crypto snapshot locks during persistence, and surface explicit too-large attachment markers. (#59846, #59851, #60599, #60289) Thanks @al3mart, @emonty, and @efe-arv.
- Android/Talk Mode: cancel in-flight `talk.speak` playback when speech is explicitly stopped, so stale replies stop starting after barge-in or manual stop. (#61164) Thanks @obviyus.
- Android/Talk Mode: restore spoken assistant replies on node-scoped sessions by keeping reply routing synced to the resolved node session key and pausing mic capture during reply playback. (#60306) Thanks @MKV21.
- Android/Talk Mode: restore voice replies on gateway-backed talk mode sessions by updating embedded runner transport overrides to the current agent transport API. (#61214) Thanks @obviyus.
- Voice-call/OpenAI: pass full plugin config into realtime transcription provider resolution so streaming calls can discover the bundled OpenAI realtime transcription provider again. Fixes #60936. Thanks @sliekens and @vincentkoc.
- Control UI/chat: add a per-session thinking-level picker in the chat header and mobile chat settings, and keep the browser bundle on UI-local thinking/session-key helpers so Safari no longer crashes on Node-only imports before rendering chat controls.
- Control UI: keep Stop visible during tool-only execution, preserve pending-send busy state, and clear stale ClawHub search results as soon as the query changes. (#54528, #59800, #60267) Thanks @chziyue and @frankekn.
- Control UI/avatar: honor `ui.assistant.avatar` when serving `/avatar/:agentId` so Appearance UI avatar paths stop falling back to initials placeholders. (#60778) Thanks @hannasdev.
- Control UI/cron: highlight the Cron refresh button while refresh is in flight so the page's loading state stays visible even when prior data remains on screen. (#60394) Thanks @coder-zhuzm.
- Control UI/Overview: prevent gateway access token/password visibility toggle buttons from overlapping their inputs at narrow widths. (#56924) Thanks @bbddbb1.
- CLI/skills JSON: route `skills list --json`, `skills info --json`, and `skills check --json` output to stdout instead of stderr so machine-readable consumers receive JSON on the expected stream again. (#60914; fixes #57599; landed from contributor PR #57611 by @Aftabbs) Thanks @Aftabbs.
- CLI/Commander: preserve Commander-computed exit codes for argument and help-error paths, and cover the user-argv parse mode in the regression tests so invalid CLI invocations no longer report success when exits are intercepted. (#60923) Thanks @Linux2010.
- Cron: replay interrupted recurring jobs on the first gateway restart instead of waiting for a second restart. (#60583) Thanks @joelnishanth.
- Cron: send failure notifications through the job's primary delivery channel using the same session context as successful delivery when no explicit `failureDestination` is configured. (#60622) Thanks @artwalker.
- Live model switching: only treat explicit user-driven model changes as pending live switches, so fallback rotation, heartbeat overrides, and compaction no longer trip `LiveSessionModelSwitchError` before making an API call. (#60266) Thanks @kiranvk-2011.
- Exec approvals: reuse durable exact-command `allow-always` approvals in allowlist mode so identical reruns stop prompting, and tighten Windows interpreter/path approval handling so wrapper and malformed-path cases fail closed more consistently. (#59880, #59780, #58040, #59182) Thanks @luoyanglang, @SnowSky1, and @pgondhi987.
- Node exec approvals: keep node-host `system.run` approvals bound to the prepared execution plan across async forwarding, so mutable script operands still get approval-time binding and drift revalidation instead of dropping back to unbound execution.
@ -190,6 +218,7 @@ Docs: https://docs.openclaw.ai
- Tasks/maintenance: reconcile stale cron and chat-backed CLI task rows against live cron-job and agent-run ownership instead of treating any persisted session key as proof that the task is still running. (#60310) Thanks @lml2468.
- Plugins: suppress trust-warning noise during non-activating snapshot and CLI metadata loads. (#61427) Thanks @gumadeiras.
- Agents/video generation: accept `agents.defaults.videoGenerationModel` in strict config validation and `openclaw config set/get`, so gateways using `video_generate` no longer fail to boot after enabling a video model.
- Matrix/streaming: add a quiet preview mode for streamed Matrix replies, keep legacy `partial` preview-first behavior, and finalize quiet media captions correctly so previews stop notifying early without dropping final text semantics. (#61450) Thanks @gumadeiras.
## 2026.4.2

View File

@ -178,9 +178,9 @@ This is a practical baseline config with DM pairing, room allowlist, and E2EE en
Matrix reply streaming is opt-in.
Set `channels.matrix.streaming` to `"partial"` when you want OpenClaw to send a single draft reply,
edit that draft in place while the model is generating text, and then finalize it when the reply is
done:
Set `channels.matrix.streaming` to `"partial"` when you want OpenClaw to send a single live preview
reply, edit that preview in place while the model is generating text, and then finalize it when the
reply is done:
```json5
{
@ -193,15 +193,164 @@ done:
```
- `streaming: "off"` is the default. OpenClaw waits for the final reply and sends it once.
- `streaming: "partial"` creates one editable preview message for the current assistant block instead of sending multiple partial messages.
- `blockStreaming: true` enables separate Matrix progress messages. With `streaming: "partial"`, Matrix keeps the live draft for the current block and preserves completed blocks as separate messages.
- When `streaming: "partial"` and `blockStreaming` is off, Matrix only edits the live draft and sends the completed reply once that block or turn finishes.
- `streaming: "partial"` creates one editable preview message for the current assistant block using normal Matrix text messages. This preserves Matrix's legacy preview-first notification behavior, so stock clients may notify on the first streamed preview text instead of the finished block.
- `streaming: "quiet"` creates one editable quiet preview notice for the current assistant block. Use this only when you also configure recipient push rules for finalized preview edits.
- `blockStreaming: true` enables separate Matrix progress messages. With preview streaming enabled, Matrix keeps the live draft for the current block and preserves completed blocks as separate messages.
- When preview streaming is on and `blockStreaming` is off, Matrix edits the live draft in place and finalizes that same event when the block or turn finishes.
- If the preview no longer fits in one Matrix event, OpenClaw stops preview streaming and falls back to normal final delivery.
- Media replies still send attachments normally. If a stale preview can no longer be reused safely, OpenClaw redacts it before sending the final media reply.
- Preview edits cost extra Matrix API calls. Leave streaming off if you want the most conservative rate-limit behavior.
`blockStreaming` does not enable draft previews by itself.
Use `streaming: "partial"` for preview edits; then add `blockStreaming: true` only if you also want completed assistant blocks to remain visible as separate progress messages.
Use `streaming: "partial"` or `streaming: "quiet"` for preview edits; then add `blockStreaming: true` only if you also want completed assistant blocks to remain visible as separate progress messages.
If you need stock Matrix notifications without custom push rules, use `streaming: "partial"` for preview-first behavior or leave `streaming` off for final-only delivery. With `streaming: "off"`:
- `blockStreaming: true` sends each finished block as a normal notifying Matrix message.
- `blockStreaming: false` sends only the final completed reply as a normal notifying Matrix message.
### Self-hosted push rules for quiet finalized previews
If you run your own Matrix infrastructure and want quiet previews to notify only when a block or
final reply is done, set `streaming: "quiet"` and add a per-user push rule for finalized preview edits.
This is usually a recipient-user setup, not a homeserver-global config change:
Quick map before you start:
- recipient user = the person who should receive the notification
- bot user = the OpenClaw Matrix account that sends the reply
- use the recipient user's access token for the API calls below
- match `sender` in the push rule against the bot user's full MXID
1. Configure OpenClaw to use quiet previews:
```json5
{
channels: {
matrix: {
streaming: "quiet",
},
},
}
```
2. Make sure the recipient account already receives normal Matrix push notifications. Quiet preview
rules only work if that user already has working pushers/devices.
3. Get the recipient user's access token.
- Use the receiving user's token, not the bot's token.
- Reusing an existing client session token is usually easiest.
- If you need to mint a fresh token, you can log in through the standard Matrix Client-Server API:
```bash
curl -sS -X POST \
"https://matrix.example.org/_matrix/client/v3/login" \
-H "Content-Type: application/json" \
--data '{
"type": "m.login.password",
"identifier": {
"type": "m.id.user",
"user": "@alice:example.org"
},
"password": "REDACTED"
}'
```
4. Verify the recipient account already has pushers:
```bash
curl -sS \
-H "Authorization: Bearer $USER_ACCESS_TOKEN" \
"https://matrix.example.org/_matrix/client/v3/pushers"
```
If this returns no active pushers/devices, fix normal Matrix notifications first before adding the
OpenClaw rule below.
OpenClaw marks finalized text-only preview edits with:
```json
{
"com.openclaw.finalized_preview": true
}
```
5. Create an override push rule for each recipient account which should receive these notifications:
```bash
curl -sS -X PUT \
"https://matrix.example.org/_matrix/client/v3/pushrules/global/override/openclaw-finalized-preview" \
-H "Authorization: Bearer $USER_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
--data '{
"conditions": [
{ "kind": "event_match", "key": "type", "pattern": "m.room.message" },
{
"kind": "event_property_is",
"key": "content.m\\.relates_to.rel_type",
"value": "m.replace"
},
{
"kind": "event_property_is",
"key": "content.com\\.openclaw\\.finalized_preview",
"value": true
},
{ "kind": "event_match", "key": "sender", "pattern": "@bot:example.org" }
],
"actions": [
"notify",
{ "set_tweak": "sound", "value": "default" },
{ "set_tweak": "highlight", "value": false }
]
}'
```
Replace these values before you run the command:
- `https://matrix.example.org`: your homeserver base URL
- `$USER_ACCESS_TOKEN`: the receiving user's access token
- `@bot:example.org`: your OpenClaw Matrix bot MXID, not the receiving user's MXID
The rule is evaluated against the event sender:
- authenticate with the receiving user's token
- match `sender` against the OpenClaw bot MXID
6. Verify the rule exists:
```bash
curl -sS \
-H "Authorization: Bearer $USER_ACCESS_TOKEN" \
"https://matrix.example.org/_matrix/client/v3/pushrules/global/override/openclaw-finalized-preview"
```
7. Test a streamed reply. In quiet mode, the room should show a quiet draft preview and the final
in-place edit should notify once the block or turn finishes.
Notes:
- Create the rule with the receiving user's access token, not the bot's.
- New user-defined `override` rules are inserted ahead of default suppress rules, so no extra ordering parameter is needed.
- This only affects text-only preview edits that OpenClaw can safely finalize in place. Media fallbacks and stale-preview fallbacks still use normal Matrix delivery.
- If `GET /_matrix/client/v3/pushers` shows no pushers, the user does not yet have working Matrix push delivery for this account/device.
#### Synapse
For Synapse, the setup above is usually enough by itself:
- No special `homeserver.yaml` change is required for finalized OpenClaw preview notifications.
- If your Synapse deployment already sends normal Matrix push notifications, the user token + `pushrules` call above is the main setup step.
- If you run Synapse behind a reverse proxy or workers, make sure `/_matrix/client/.../pushrules/` reaches Synapse correctly.
- If you run Synapse workers, make sure pushers are healthy. Push delivery is handled by the main process or `synapse.app.pusher` / configured pusher workers.
#### Tuwunel
For Tuwunel, use the same setup flow and push-rule API call shown above:
- No Tuwunel-specific config is required for the finalized preview marker itself.
- If normal Matrix notifications already work for that user, the user token + `pushrules` call above is the main setup step.
- If notifications seem to disappear while the user is active on another device, check whether `suppress_push_when_active` is enabled. Tuwunel added this option in Tuwunel 1.4.2 on September 12, 2025, and it can intentionally suppress pushes to other devices while one device is active.
## Encryption and verification
@ -833,7 +982,7 @@ Live directory lookup uses the logged-in Matrix account:
- `historyLimit`: max room messages to include as group history context. Falls back to `messages.groupChat.historyLimit`. Set `0` to disable.
- `replyToMode`: `off`, `first`, or `all`.
- `markdown`: optional Markdown rendering configuration for outbound Matrix text.
- `streaming`: `off` (default), `partial`, `true`, or `false`. `partial` and `true` enable single-message draft previews with edit-in-place updates.
- `streaming`: `off` (default), `partial`, `quiet`, `true`, or `false`. `partial` and `true` enable preview-first draft updates with normal Matrix text messages. `quiet` uses non-notifying preview notices for self-hosted push-rule setups.
- `blockStreaming`: `true` enables separate progress messages for completed assistant blocks while draft preview streaming is active.
- `threadReplies`: `off`, `inbound`, or `always`.
- `threadBindings`: per-channel overrides for thread-bound session routing and lifecycle.

View File

@ -78,4 +78,13 @@ describe("MatrixConfigSchema SecretInput", () => {
}
expect(result.data.rooms?.["!room:example.org"]?.account).toBe("axis");
});
it("accepts quiet Matrix streaming mode", () => {
const result = MatrixConfigSchema.safeParse({
homeserver: "https://matrix.example.org",
accessToken: "token",
streaming: "quiet",
});
expect(result.success).toBe(true);
});
});

View File

@ -84,7 +84,7 @@ export const MatrixConfigSchema = z.object({
groupPolicy: GroupPolicySchema.optional(),
contextVisibility: ContextVisibilityModeSchema.optional(),
blockStreaming: z.boolean().optional(),
streaming: z.union([z.enum(["partial", "off"]), z.boolean()]).optional(),
streaming: z.union([z.enum(["partial", "quiet", "off"]), z.boolean()]).optional(),
replyToMode: z.enum(["off", "first", "all", "batched"]).optional(),
threadReplies: z.enum(["off", "inbound", "always"]).optional(),
textChunkLimit: z.number().optional(),

View File

@ -69,7 +69,7 @@ describe("createMatrixDraftStream", () => {
vi.useRealTimers();
});
it("sends a new message on first update", async () => {
it("sends a normal text preview on first partial update", async () => {
const stream = createMatrixDraftStream({
roomId: "!room:test",
client,
@ -80,14 +80,36 @@ describe("createMatrixDraftStream", () => {
await stream.flush();
expect(sendMessageMock).toHaveBeenCalledTimes(1);
expect(sendMessageMock.mock.calls[0]?.[1]).toMatchObject({
msgtype: "m.text",
});
expect(stream.eventId()).toBe("$evt1");
});
it("edits the message on subsequent updates", async () => {
it("sends quiet preview notices when quiet mode is enabled", async () => {
const stream = createMatrixDraftStream({
roomId: "!room:test",
client,
cfg: {} as import("../types.js").CoreConfig,
mode: "quiet",
});
stream.update("Hello");
await stream.flush();
expect(sendMessageMock).toHaveBeenCalledTimes(1);
expect(sendMessageMock.mock.calls[0]?.[1]).toMatchObject({
msgtype: "m.notice",
});
expect(sendMessageMock.mock.calls[0]?.[1]).not.toHaveProperty("m.mentions");
});
it("edits the message on subsequent quiet updates", async () => {
const stream = createMatrixDraftStream({
roomId: "!room:test",
client,
cfg: {} as import("../types.js").CoreConfig,
mode: "quiet",
});
stream.update("Hello");
@ -102,13 +124,18 @@ describe("createMatrixDraftStream", () => {
// First call = initial send, second call = edit (both go through sendMessage)
expect(sendMessageMock).toHaveBeenCalledTimes(2);
expect(sendMessageMock.mock.calls[1]?.[1]).toMatchObject({
msgtype: "m.notice",
"m.new_content": { msgtype: "m.notice" },
});
});
it("coalesces rapid updates within throttle window", async () => {
it("coalesces rapid quiet updates within throttle window", async () => {
const stream = createMatrixDraftStream({
roomId: "!room:test",
client,
cfg: {} as import("../types.js").CoreConfig,
mode: "quiet",
});
stream.update("A");
@ -122,6 +149,11 @@ describe("createMatrixDraftStream", () => {
expect(sendMessageMock.mock.calls[0][1]).toMatchObject({ body: "A" });
// Edit uses "* <text>" prefix per Matrix m.replace spec.
expect(sendMessageMock.mock.calls[1][1]).toMatchObject({ body: "* ABC" });
expect(sendMessageMock.mock.calls[0][1]).toMatchObject({ msgtype: "m.notice" });
expect(sendMessageMock.mock.calls[1][1]).toMatchObject({
msgtype: "m.notice",
"m.new_content": { msgtype: "m.notice" },
});
});
it("skips no-op updates", async () => {
@ -178,6 +210,7 @@ describe("createMatrixDraftStream", () => {
roomId: "!room:test",
client,
cfg: {} as import("../types.js").CoreConfig,
mode: "quiet",
});
stream.update("Block 1");
@ -296,7 +329,6 @@ describe("createMatrixDraftStream", () => {
expect(sendMessageMock).not.toHaveBeenCalled();
expect(stream.eventId()).toBeUndefined();
expect(stream.mustDeliverFinalNormally()).toBe(true);
expect(log).toHaveBeenCalledWith(
expect.stringContaining("preview exceeded single-event limit"),
);
@ -317,7 +349,6 @@ describe("createMatrixDraftStream", () => {
await stream.flush();
expect(sendMessageMock).not.toHaveBeenCalled();
expect(stream.mustDeliverFinalNormally()).toBe(true);
expect(log).toHaveBeenCalledWith(
expect.stringContaining("preview exceeded single-event limit"),
);

View File

@ -2,8 +2,25 @@ import { createDraftStreamLoop } from "openclaw/plugin-sdk/channel-lifecycle";
import type { CoreConfig } from "../types.js";
import type { MatrixClient } from "./sdk.js";
import { editMessageMatrix, prepareMatrixSingleText, sendSingleTextMessageMatrix } from "./send.js";
import { MsgType } from "./send/types.js";
const DEFAULT_THROTTLE_MS = 1000;
type MatrixDraftPreviewMode = "partial" | "quiet";
function resolveDraftPreviewOptions(mode: MatrixDraftPreviewMode): {
msgtype: typeof MsgType.Text | typeof MsgType.Notice;
includeMentions?: boolean;
} {
if (mode === "quiet") {
return {
msgtype: MsgType.Notice,
includeMentions: false,
};
}
return {
msgtype: MsgType.Text,
};
}
export type MatrixDraftStream = {
/** Update the draft with the latest accumulated text for the current block. */
@ -16,8 +33,8 @@ export type MatrixDraftStream = {
reset: () => void;
/** The event ID of the current draft message, if any. */
eventId: () => string | undefined;
/** The last text successfully sent or edited. */
lastSentText: () => string;
/** True when the provided text matches the last rendered draft payload. */
matchesPreparedText: (text: string) => boolean;
/** True when preview streaming must fall back to normal final delivery. */
mustDeliverFinalNormally: () => boolean;
};
@ -26,6 +43,7 @@ export function createMatrixDraftStream(params: {
roomId: string;
client: MatrixClient;
cfg: CoreConfig;
mode?: MatrixDraftPreviewMode;
threadId?: string;
replyToId?: string;
/** When true, reset() restores the original replyToId instead of clearing it. */
@ -34,6 +52,7 @@ export function createMatrixDraftStream(params: {
log?: (message: string) => void;
}): MatrixDraftStream {
const { roomId, client, cfg, threadId, accountId, log } = params;
const preview = resolveDraftPreviewOptions(params.mode ?? "partial");
let currentEventId: string | undefined;
let lastSentText = "";
@ -59,8 +78,6 @@ export function createMatrixDraftStream(params: {
);
return false;
}
// If the initial send failed, stop trying for this block. The deliver
// callback will fall back to deliverMatrixReplies.
if (sendFailed) {
return false;
}
@ -75,6 +92,8 @@ export function createMatrixDraftStream(params: {
replyToId,
threadId,
accountId,
msgtype: preview.msgtype,
includeMentions: preview.includeMentions,
});
currentEventId = result.messageId;
lastSentText = preparedText.trimmedText;
@ -85,6 +104,8 @@ export function createMatrixDraftStream(params: {
cfg,
threadId,
accountId,
msgtype: preview.msgtype,
includeMentions: preview.includeMentions,
});
lastSentText = preparedText.trimmedText;
}
@ -94,16 +115,11 @@ export function createMatrixDraftStream(params: {
const isPreviewLimitError =
err instanceof Error && err.message.startsWith("Matrix single-message text exceeds limit");
if (isPreviewLimitError) {
// Once the preview no longer fits in one editable event, preserve the
// current preview as-is and fall back to normal final delivery.
finalizeInPlaceBlocked = true;
}
if (!currentEventId) {
// First send failed — give up for this block so the deliver callback
// falls through to normal delivery.
sendFailed = true;
}
// Signal failure so the loop stops retrying.
stopped = true;
return false;
}
@ -148,7 +164,11 @@ export function createMatrixDraftStream(params: {
stop,
reset,
eventId: () => currentEventId,
lastSentText: () => lastSentText,
matchesPreparedText: (text: string) =>
prepareMatrixSingleText(text, {
cfg,
accountId,
}).trimmedText === lastSentText,
mustDeliverFinalNormally: () => sendFailed || finalizeInPlaceBlocked,
};
}

View File

@ -1,6 +1,6 @@
import { vi } from "vitest";
import type { RuntimeEnv, RuntimeLogger } from "../../runtime-api.js";
import type { MatrixRoomConfig, ReplyToMode } from "../../types.js";
import type { MatrixRoomConfig, MatrixStreamingMode, ReplyToMode } from "../../types.js";
import type { MatrixClient } from "../sdk.js";
import { createMatrixRoomMessageHandler, type MatrixMonitorHandlerParams } from "./handler.js";
import { EventType, type MatrixRawEvent, type RoomMessageEventContent } from "./types.js";
@ -32,7 +32,7 @@ type MatrixHandlerTestHarnessOptions = {
threadReplies?: "off" | "inbound" | "always";
dmThreadReplies?: "off" | "inbound" | "always";
dmSessionScope?: "per-user" | "per-room";
streaming?: "partial" | "off";
streaming?: MatrixStreamingMode;
blockStreamingEnabled?: boolean;
dmEnabled?: boolean;
dmPolicy?: "pairing" | "allowlist" | "open" | "disabled";

View File

@ -8,6 +8,7 @@ import {
} from "openclaw/plugin-sdk/conversation-runtime";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { installMatrixMonitorTestRuntime } from "../../test-runtime.js";
import { MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY } from "../send/types.js";
import { createMatrixRoomMessageHandler } from "./handler.js";
import {
createMatrixHandlerTestHarness,
@ -1988,6 +1989,7 @@ describe("matrix monitor handler draft streaming", () => {
function createStreamingHarness(opts?: {
replyToMode?: "off" | "first" | "all" | "batched";
blockStreamingEnabled?: boolean;
streaming?: "partial" | "quiet";
}) {
let capturedDeliver: DeliverFn | undefined;
let capturedReplyOpts: ReplyOpts | undefined;
@ -2007,7 +2009,7 @@ describe("matrix monitor handler draft streaming", () => {
const redactEventMock = vi.fn(async () => "$redacted");
const { handler } = createMatrixHandlerTestHarness({
streaming: "partial",
streaming: opts?.streaming ?? "quiet",
blockStreamingEnabled: opts?.blockStreamingEnabled ?? false,
replyToMode: opts?.replyToMode ?? "off",
client: { redactEvent: redactEventMock },
@ -2067,7 +2069,7 @@ describe("matrix monitor handler draft streaming", () => {
return { dispatch, redactEventMock };
}
it("finalizes a single partial-preview block in place when block streaming is enabled", async () => {
it("finalizes a single quiet-preview block in place when block streaming is enabled", async () => {
const { dispatch, redactEventMock } = createStreamingHarness({ blockStreamingEnabled: true });
const { deliver, opts, finish } = await dispatch();
@ -2080,12 +2082,76 @@ describe("matrix monitor handler draft streaming", () => {
await deliver({ text: "Single block" }, { kind: "final" });
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft1",
"Single block",
expect.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
await finish();
});
it("preserves completed blocks by rotating to a new draft when block streaming is enabled", async () => {
it("keeps partial preview-first finalization on the existing draft when text is unchanged", async () => {
const { dispatch, redactEventMock } = createStreamingHarness({
blockStreamingEnabled: true,
streaming: "partial",
});
const { deliver, opts, finish } = await dispatch();
opts.onPartialReply?.({ text: "Single block" });
await vi.waitFor(() => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"Single block",
expect.not.objectContaining({
msgtype: "m.notice",
includeMentions: false,
}),
);
await deliver({ text: "Single block" }, { kind: "final" });
expect(editMessageMatrixMock).not.toHaveBeenCalled();
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
await finish();
});
it("still edits partial preview-first drafts when the final text changes", async () => {
const { dispatch, redactEventMock } = createStreamingHarness({
blockStreamingEnabled: true,
streaming: "partial",
});
const { deliver, opts, finish } = await dispatch();
opts.onPartialReply?.({ text: "Single" });
await vi.waitFor(() => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
await deliver({ text: "Single block" }, { kind: "final" });
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft1",
"Single block",
expect.not.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
await finish();
});
it("preserves completed blocks by rotating to a new quiet preview", async () => {
const { dispatch, redactEventMock } = createStreamingHarness({ blockStreamingEnabled: true });
const { deliver, opts, finish } = await dispatch();
@ -2097,6 +2163,14 @@ describe("matrix monitor handler draft streaming", () => {
deliverMatrixRepliesMock.mockClear();
await deliver({ text: "Block one" }, { kind: "block" });
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft1",
"Block one",
expect.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
@ -2112,6 +2186,14 @@ describe("matrix monitor handler draft streaming", () => {
await deliver({ text: "Block two" }, { kind: "final" });
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft2",
"Block two",
expect.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
await finish();
@ -2198,17 +2280,13 @@ describe("matrix monitor handler draft streaming", () => {
const { dispatch } = createStreamingHarness();
const { deliver, opts, finish } = await dispatch();
// Simulate streaming: partial reply creates draft message.
opts.onPartialReply?.({ text: "Hello" });
// Wait for the draft stream's immediate send to complete.
await vi.waitFor(() => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
// Make the final edit fail.
editMessageMatrixMock.mockRejectedValueOnce(new Error("rate limited"));
// Deliver final — should catch edit failure and fall back.
await deliver({ text: "Hello world" }, { kind: "block" });
expect(deliverMatrixRepliesMock).toHaveBeenCalledTimes(1);
@ -2309,6 +2387,8 @@ describe("matrix monitor handler draft streaming", () => {
"Alpha",
expect.anything(),
);
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
await vi.waitFor(() => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
@ -2358,6 +2438,8 @@ describe("matrix monitor handler draft streaming", () => {
"Alpha",
expect.anything(),
);
expect(deliverMatrixRepliesMock).not.toHaveBeenCalled();
expect(redactEventMock).not.toHaveBeenCalled();
await vi.waitFor(() => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
@ -2400,7 +2482,14 @@ describe("matrix monitor handler draft streaming", () => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
expect(sendSingleTextMessageMatrixMock.mock.calls[0]?.[1]).toBe("Beta");
expect(editMessageMatrixMock).not.toHaveBeenCalled();
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft1",
"Alpha",
expect.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
sendSingleTextMessageMatrixMock.mockClear();
editMessageMatrixMock.mockClear();
@ -2414,7 +2503,14 @@ describe("matrix monitor handler draft streaming", () => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
expect(sendSingleTextMessageMatrixMock.mock.calls[0]?.[1]).toBe("Gamma");
expect(editMessageMatrixMock).not.toHaveBeenCalled();
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft2",
"Beta",
expect.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
await finish();
});
@ -2431,7 +2527,7 @@ describe("matrix monitor handler draft streaming", () => {
let capturedReplyOpts: ReplyOpts | undefined;
const { handler } = createMatrixHandlerTestHarness({
streaming: "partial",
streaming: "quiet",
createReplyDispatcherWithTyping: () => ({
dispatcher: { markComplete: () => {}, waitForIdle: async () => {} },
replyOptions: {},
@ -2555,6 +2651,46 @@ describe("matrix monitor handler draft streaming", () => {
await finish();
});
it("finalizes quiet drafts before reusing unchanged media captions", async () => {
const { dispatch, redactEventMock } = createStreamingHarness({ streaming: "quiet" });
const { deliver, opts, finish } = await dispatch();
opts.onPartialReply?.({ text: "@room screenshot ready" });
await vi.waitFor(() => {
expect(sendSingleTextMessageMatrixMock).toHaveBeenCalledTimes(1);
});
deliverMatrixRepliesMock.mockClear();
await deliver(
{
text: "@room screenshot ready",
mediaUrl: "https://example.com/image.png",
},
{ kind: "final" },
);
expect(editMessageMatrixMock).toHaveBeenCalledWith(
"!room:example.org",
"$draft1",
"@room screenshot ready",
expect.objectContaining({
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
}),
);
expect(redactEventMock).not.toHaveBeenCalled();
expect(deliverMatrixRepliesMock).toHaveBeenCalledWith(
expect.objectContaining({
replies: [
expect.objectContaining({
mediaUrl: "https://example.com/image.png",
text: undefined,
}),
],
}),
);
await finish();
});
it("redacts stale draft and sends the final once when a later preview exceeds the event limit", async () => {
const { dispatch, redactEventMock } = createStreamingHarness();
const { deliver, opts, finish } = await dispatch();
@ -2628,6 +2764,27 @@ describe("matrix monitor handler block streaming config", () => {
expect(capturedDisableBlockStreaming).toBe(true);
});
it("keeps block streaming disabled when quiet previews are on and block streaming is off", async () => {
let capturedDisableBlockStreaming: boolean | undefined;
const { handler } = createMatrixHandlerTestHarness({
streaming: "quiet",
dispatchReplyFromConfig: vi.fn(
async (args: { replyOptions?: { disableBlockStreaming?: boolean } }) => {
capturedDisableBlockStreaming = args.replyOptions?.disableBlockStreaming;
return { queuedFinal: false, counts: { final: 0, block: 0, tool: 0 } };
},
) as never,
});
await handler(
"!room:example.org",
createMatrixTextMessageEvent({ eventId: "$msg1", body: "hello" }),
);
expect(capturedDisableBlockStreaming).toBe(true);
});
it("allows shared block streaming when partial previews and block streaming are both enabled", async () => {
let capturedDisableBlockStreaming: boolean | undefined;

View File

@ -6,7 +6,12 @@ import {
} from "openclaw/plugin-sdk/config-runtime";
import { getSessionBindingService } from "openclaw/plugin-sdk/conversation-runtime";
import { evaluateSupplementalContextVisibility } from "openclaw/plugin-sdk/security-runtime";
import type { CoreConfig, MatrixRoomConfig, ReplyToMode } from "../../types.js";
import type {
CoreConfig,
MatrixRoomConfig,
MatrixStreamingMode,
ReplyToMode,
} from "../../types.js";
import { createMatrixDraftStream } from "../draft-stream.js";
import { isMatrixMediaSizeLimitError } from "../media-errors.js";
import {
@ -32,6 +37,7 @@ import {
sendTypingMatrix,
} from "../send.js";
import { resolveMatrixStoredSessionMeta } from "../session-store-metadata.js";
import { MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY } from "../send/types.js";
import { resolveMatrixMonitorAccessState } from "./access-state.js";
import { resolveMatrixAckReactionConfig } from "./ack-config.js";
import { resolveMatrixAllowListMatch } from "./allowlist.js";
@ -76,6 +82,18 @@ const MAX_TRACKED_PAIRING_REPLY_SENDERS = 512;
const MAX_TRACKED_SHARED_DM_CONTEXT_NOTICES = 512;
type MatrixAllowBotsMode = "off" | "mentions" | "all";
async function redactMatrixDraftEvent(
client: MatrixClient,
roomId: string,
draftEventId: string,
): Promise<void> {
await client.redactEvent(roomId, draftEventId).catch(() => {});
}
function buildMatrixFinalizedPreviewContent(): Record<string, unknown> {
return { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true };
}
export type MatrixMonitorHandlerParams = {
client: MatrixClient;
core: PluginRuntime;
@ -96,7 +114,7 @@ export type MatrixMonitorHandlerParams = {
dmThreadReplies?: "off" | "inbound" | "always";
/** DM session grouping behavior. */
dmSessionScope?: "per-user" | "per-room";
streaming: "partial" | "off";
streaming: MatrixStreamingMode;
blockStreamingEnabled: boolean;
dmEnabled: boolean;
dmPolicy: "open" | "pairing" | "allowlist" | "disabled";
@ -1280,14 +1298,15 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
});
},
});
const draftStreamingEnabled = streaming === "partial";
const draftStreamingEnabled = streaming !== "off";
const quietDraftStreaming = streaming === "quiet";
const draftReplyToId = replyToMode !== "off" && !threadTarget ? _messageId : undefined;
let currentDraftReplyToId = draftReplyToId;
const draftStream = draftStreamingEnabled
? createMatrixDraftStream({
roomId,
client,
cfg,
mode: quietDraftStreaming ? "quiet" : "partial",
threadId: threadTarget,
replyToId: draftReplyToId,
preserveReplyId: replyToMode === "all",
@ -1308,6 +1327,7 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
let latestDraftFullText = "";
const pendingDraftBoundaries: PendingDraftBoundary[] = [];
const latestQueuedDraftBoundaryOffsets = new Map<number, number>();
let currentDraftReplyToId = draftReplyToId;
// Set after the first final payload consumes the draft event so
// subsequent finals go through normal delivery.
let draftConsumed = false;
@ -1380,9 +1400,8 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
const hasMedia = Boolean(payload.mediaUrl) || (payload.mediaUrls?.length ?? 0) > 0;
await draftStream.stop();
const draftEventId = draftStream.eventId();
// After the first final payload consumes the draft, subsequent
// finals must go through normal delivery to avoid overwriting.
if (draftConsumed) {
await deliverMatrixReplies({
cfg,
@ -1400,16 +1419,6 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
return;
}
// Read event id after stop() — flush may have created the
// initial message while draining pending text.
const draftEventId = draftStream.eventId();
// If the payload carries a reply target that differs from the
// draft's, fall through to normal delivery — Matrix edits
// cannot change the reply relation on an existing event.
// Skip when replyToMode is "off" (replies stripped anyway)
// or when threadTarget is set (thread relations take
// precedence over replyToId in deliverMatrixReplies).
const payloadReplyToId = payload.replyToId?.trim() || undefined;
const payloadReplyMismatch =
replyToMode !== "off" &&
@ -1424,58 +1433,61 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
!payloadReplyMismatch &&
!mustDeliverFinalNormally
) {
// Text-only: final edit of the draft message. Skip if
// stop() already flushed identical text to avoid a
// redundant API call that wastes rate-limit budget.
if (payload.text !== draftStream.lastSentText()) {
try {
try {
const requiresFinalEdit =
quietDraftStreaming || !draftStream.matchesPreparedText(payload.text);
if (requiresFinalEdit) {
await editMessageMatrix(roomId, draftEventId, payload.text, {
client,
cfg,
threadId: threadTarget,
accountId: _route.accountId,
});
} catch {
// Edit failed (rate limit, server error) — redact the
// stale draft and fall back to normal delivery so the
// user still gets the final answer.
await client.redactEvent(roomId, draftEventId).catch(() => {});
await deliverMatrixReplies({
cfg,
replies: [payload],
roomId,
client,
runtime,
textLimit,
replyToMode,
threadId: threadTarget,
accountId: _route.accountId,
mediaLocalRoots,
tableMode,
extraContent: quietDraftStreaming
? buildMatrixFinalizedPreviewContent()
: undefined,
});
}
} catch {
await redactMatrixDraftEvent(client, roomId, draftEventId);
await deliverMatrixReplies({
cfg,
replies: [payload],
roomId,
client,
runtime,
textLimit,
replyToMode,
threadId: threadTarget,
accountId: _route.accountId,
mediaLocalRoots,
tableMode,
});
}
draftConsumed = true;
} else if (draftEventId && hasMedia && !payloadReplyMismatch) {
// Media payload: finalize draft text, send media separately.
let textEditOk = !mustDeliverFinalNormally;
if (textEditOk && payload.text && payload.text !== draftStream.lastSentText()) {
textEditOk = await editMessageMatrix(roomId, draftEventId, payload.text, {
const payloadText = payload.text;
const requiresFinalTextEdit =
quietDraftStreaming ||
(typeof payloadText === "string" &&
!draftStream.matchesPreparedText(payloadText));
if (textEditOk && payloadText && requiresFinalTextEdit) {
textEditOk = await editMessageMatrix(roomId, draftEventId, payloadText, {
client,
cfg,
threadId: threadTarget,
accountId: _route.accountId,
extraContent: quietDraftStreaming
? buildMatrixFinalizedPreviewContent()
: undefined,
}).then(
() => true,
() => false,
);
}
const reusesDraftAsFinalText = Boolean(payload.text?.trim()) && textEditOk;
// If the text edit failed, or there is no final text to reuse
// the preview, redact the stale draft and include text in media
// delivery so the final caption is not lost.
if (!reusesDraftAsFinalText) {
await client.redactEvent(roomId, draftEventId).catch(() => {});
await redactMatrixDraftEvent(client, roomId, draftEventId);
}
await deliverMatrixReplies({
cfg,
@ -1494,11 +1506,8 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
});
draftConsumed = true;
} else {
// Redact stale draft when the final delivery will create a
// new message (reply-target mismatch, preview overflow, or no
// usable draft).
if (draftEventId && (payloadReplyMismatch || mustDeliverFinalNormally)) {
await client.redactEvent(roomId, draftEventId).catch(() => {});
await redactMatrixDraftEvent(client, roomId, draftEventId);
}
await deliverMatrixReplies({
cfg,
@ -1515,9 +1524,6 @@ export function createMatrixRoomMessageHandler(params: MatrixMonitorHandlerParam
});
}
// Only reset for intermediate blocks — after the final delivery
// the stream must stay stopped so late async callbacks cannot
// create ghost messages.
if (info.kind === "block") {
draftConsumed = false;
advanceDraftBlockBoundary({ fallbackToLatestEnd: true });

View File

@ -215,8 +215,12 @@ export async function monitorMatrixProvider(opts: MonitorMatrixOpts = {}): Promi
const historyLimit = Math.max(0, accountConfig.historyLimit ?? globalGroupChatHistoryLimit ?? 0);
const mediaMaxMb = opts.mediaMaxMb ?? accountConfig.mediaMaxMb ?? DEFAULT_MEDIA_MAX_MB;
const mediaMaxBytes = Math.max(1, mediaMaxMb) * 1024 * 1024;
const streaming: "partial" | "off" =
accountConfig.streaming === true || accountConfig.streaming === "partial" ? "partial" : "off";
const streaming: "partial" | "quiet" | "off" =
accountConfig.streaming === true || accountConfig.streaming === "partial"
? "partial"
: accountConfig.streaming === "quiet"
? "quiet"
: "off";
const blockStreamingEnabled = accountConfig.blockStreaming === true;
const startupMs = Date.now();
const startupGraceMs = 0;

View File

@ -9,6 +9,7 @@ import {
sendSingleTextMessageMatrix,
sendTypingMatrix,
} from "./send.js";
import { MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY } from "./send/types.js";
const loadOutboundMediaFromUrlMock = vi.hoisted(() => vi.fn());
const loadWebMediaMock = vi.fn().mockResolvedValue({
@ -602,6 +603,39 @@ describe("sendSingleTextMessageMatrix", () => {
expect(sendMessage).not.toHaveBeenCalled();
});
it("supports quiet draft preview sends without mention metadata", async () => {
const { client, sendMessage } = makeClient();
await sendSingleTextMessageMatrix("room:!room:example", "@room hi @alice:example.org", {
client,
msgtype: "m.notice",
includeMentions: false,
});
expect(sendMessage.mock.calls[0]?.[1]).toMatchObject({
msgtype: "m.notice",
body: "@room hi @alice:example.org",
});
expect(sendMessage.mock.calls[0]?.[1]).not.toHaveProperty("m.mentions");
expect(
(sendMessage.mock.calls[0]?.[1] as { formatted_body?: string }).formatted_body,
).not.toContain("matrix.to");
});
it("merges extra content fields into single-event sends", async () => {
const { client, sendMessage } = makeClient();
await sendSingleTextMessageMatrix("room:!room:example", "done", {
client,
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
});
expect(sendMessage.mock.calls[0]?.[1]).toMatchObject({
body: "done",
[MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true,
});
});
});
describe("editMessageMatrix mentions", () => {
@ -677,6 +711,57 @@ describe("editMessageMatrix mentions", () => {
},
});
});
it("supports quiet draft preview edits without mention metadata", async () => {
const { client, sendMessage, getEvent } = makeClient();
getEvent.mockResolvedValue({
content: {
body: "@room hi @alice:example.org",
"m.mentions": { room: true, user_ids: ["@alice:example.org"] },
},
});
await editMessageMatrix("room:!room:example", "$original", "@room hi @alice:example.org", {
client,
msgtype: "m.notice",
includeMentions: false,
});
expect(sendMessage.mock.calls[0]?.[1]).toMatchObject({
msgtype: "m.notice",
"m.new_content": {
msgtype: "m.notice",
},
});
expect(sendMessage.mock.calls[0]?.[1]).not.toHaveProperty("m.mentions");
expect(sendMessage.mock.calls[0]?.[1]?.["m.new_content"]).not.toHaveProperty("m.mentions");
expect(
(sendMessage.mock.calls[0]?.[1] as { formatted_body?: string }).formatted_body,
).not.toContain("matrix.to");
expect(
(
sendMessage.mock.calls[0]?.[1] as {
"m.new_content"?: { formatted_body?: string };
}
)["m.new_content"]?.formatted_body,
).not.toContain("matrix.to");
});
it("merges extra content fields into edit payloads and m.new_content", async () => {
const { client, sendMessage } = makeClient();
await editMessageMatrix("room:!room:example", "$original", "done", {
client,
extraContent: { [MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true },
});
expect(sendMessage.mock.calls[0]?.[1]).toMatchObject({
[MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true,
"m.new_content": {
[MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY]: true,
},
});
});
});
describe("sendPollMatrix mentions", () => {

View File

@ -33,9 +33,11 @@ import {
EventType,
MsgType,
RelationType,
type MatrixExtraContentFields,
type MatrixOutboundContent,
type MatrixSendOpts,
type MatrixSendResult,
type MatrixTextMsgType,
} from "./send/types.js";
const MATRIX_TEXT_LIMIT = 4000;
@ -102,6 +104,16 @@ function hasMatrixMentionsMetadata(content: Record<string, unknown> | undefined)
return Boolean(content && Object.hasOwn(content, "m.mentions"));
}
function withMatrixExtraContentFields<T extends Record<string, unknown>>(
content: T,
extraContent?: MatrixExtraContentFields,
): T {
if (!extraContent) {
return content;
}
return { ...content, ...extraContent };
}
async function resolvePreviousEditMentions(params: {
client: MatrixClient;
content: Record<string, unknown> | undefined;
@ -398,6 +410,9 @@ export async function sendSingleTextMessageMatrix(
replyToId?: string;
threadId?: string;
accountId?: string;
msgtype?: MatrixTextMsgType;
includeMentions?: boolean;
extraContent?: MatrixExtraContentFields;
} = {},
): Promise<MatrixSendResult> {
const { trimmedText, convertedText, singleEventLimit, fitsInSingleEvent } =
@ -425,11 +440,17 @@ export async function sendSingleTextMessageMatrix(
const relation = normalizedThreadId
? buildThreadRelation(normalizedThreadId, opts.replyToId)
: buildReplyRelation(opts.replyToId);
const content = buildTextContent(convertedText, relation);
const content = withMatrixExtraContentFields(
buildTextContent(convertedText, relation, {
msgtype: opts.msgtype,
}),
opts.extraContent,
);
await enrichMatrixFormattedContent({
client,
content,
markdown: convertedText,
includeMentions: opts.includeMentions,
});
const eventId = await client.sendMessage(resolvedRoom, content);
return {
@ -468,6 +489,9 @@ export async function editMessageMatrix(
threadId?: string;
accountId?: string;
timeoutMs?: number;
msgtype?: MatrixTextMsgType;
includeMentions?: boolean;
extraContent?: MatrixExtraContentFields;
} = {},
): Promise<string> {
return await withResolvedMatrixSendClient(
@ -486,22 +510,30 @@ export async function editMessageMatrix(
accountId: opts.accountId,
});
const convertedText = getCore().channel.text.convertMarkdownTables(newText, tableMode);
const newContent = buildTextContent(convertedText);
const newContent = withMatrixExtraContentFields(
buildTextContent(convertedText, undefined, {
msgtype: opts.msgtype,
}),
opts.extraContent,
);
await enrichMatrixFormattedContent({
client,
content: newContent,
markdown: convertedText,
includeMentions: opts.includeMentions,
});
const previousEvent = await getPreviousMatrixEvent(client, resolvedRoom, originalEventId);
const previousContent = resolvePreviousEditContent(previousEvent);
const previousMentions = await resolvePreviousEditMentions({
client,
content: previousContent,
});
const replaceMentions = diffMatrixMentions(
extractMatrixMentions(newContent),
previousMentions,
);
const replaceMentions =
opts.includeMentions === false
? undefined
: diffMatrixMentions(
extractMatrixMentions(newContent),
await resolvePreviousEditMentions({
client,
content: resolvePreviousEditContent(
await getPreviousMatrixEvent(client, resolvedRoom, originalEventId),
),
}),
);
const replaceRelation: Record<string, unknown> = {
rel_type: RelationType.Replace,
@ -522,10 +554,12 @@ export async function editMessageMatrix(
...(typeof newContent.formatted_body === "string"
? { formatted_body: `* ${newContent.formatted_body}` }
: {}),
"m.mentions": replaceMentions,
"m.new_content": newContent,
"m.relates_to": replaceRelation,
};
if (replaceMentions !== undefined) {
content["m.mentions"] = replaceMentions;
}
const eventId = await client.sendMessage(resolvedRoom, content);
return eventId ?? "";

View File

@ -1,5 +1,6 @@
import { getMatrixRuntime } from "../../runtime.js";
import {
markdownToMatrixHtml,
resolveMatrixMentionsInMarkdown,
renderMarkdownToMatrixHtmlWithMentions,
type MatrixMentions,
@ -13,20 +14,45 @@ import {
type MatrixRelation,
type MatrixReplyRelation,
type MatrixTextContent,
type MatrixTextMsgType,
type MatrixThreadRelation,
} from "./types.js";
const getCore = () => getMatrixRuntime();
export function buildTextContent(body: string, relation?: MatrixRelation): MatrixTextContent {
async function renderMatrixFormattedContent(params: {
client: MatrixClient;
markdown?: string | null;
includeMentions?: boolean;
}): Promise<{ html?: string; mentions?: MatrixMentions }> {
const markdown = params.markdown ?? "";
if (params.includeMentions === false) {
const html = markdownToMatrixHtml(markdown).trimEnd();
return { html: html || undefined };
}
const { html, mentions } = await renderMarkdownToMatrixHtmlWithMentions({
markdown,
client: params.client,
});
return { html, mentions };
}
export function buildTextContent(
body: string,
relation?: MatrixRelation,
opts: {
msgtype?: MatrixTextMsgType;
} = {},
): MatrixTextContent {
const msgtype = opts.msgtype ?? MsgType.Text;
return relation
? {
msgtype: MsgType.Text,
msgtype,
body,
"m.relates_to": relation,
}
: {
msgtype: MsgType.Text,
msgtype,
body,
};
}
@ -35,12 +61,18 @@ export async function enrichMatrixFormattedContent(params: {
client: MatrixClient;
content: MatrixFormattedContent;
markdown?: string | null;
includeMentions?: boolean;
}): Promise<void> {
const { html, mentions } = await renderMarkdownToMatrixHtmlWithMentions({
markdown: params.markdown ?? "",
const { html, mentions } = await renderMatrixFormattedContent({
client: params.client,
markdown: params.markdown,
includeMentions: params.includeMentions,
});
params.content["m.mentions"] = mentions;
if (mentions) {
params.content["m.mentions"] = mentions;
} else {
delete params.content["m.mentions"];
}
if (!html) {
delete params.content.format;
delete params.content.formatted_body;

View File

@ -38,6 +38,8 @@ export const EventType = {
RoomMessage: "m.room.message",
} as const;
export const MATRIX_OPENCLAW_FINALIZED_PREVIEW_KEY = "com.openclaw.finalized_preview" as const;
export type MatrixDirectAccountData = Record<string, string[]>;
export type MatrixReplyRelation = {
@ -110,9 +112,13 @@ export type MatrixMediaMsgType =
| typeof MsgType.Video
| typeof MsgType.File;
export type MatrixTextMsgType = typeof MsgType.Text | typeof MsgType.Notice;
export type MediaKind = "image" | "audio" | "video" | "document" | "unknown";
export type MatrixFormattedContent = MessageEventContent & {
format?: string;
formatted_body?: string;
};
export type MatrixExtraContentFields = Record<string, unknown>;

View File

@ -83,6 +83,8 @@ export type MatrixExecApprovalConfig = {
target?: MatrixExecApprovalTarget;
};
export type MatrixStreamingMode = "partial" | "quiet" | "off";
export type MatrixNetworkConfig = {
/** Dangerous opt-in for trusted private/internal Matrix homeservers. */
dangerouslyAllowPrivateNetwork?: boolean;
@ -189,16 +191,20 @@ export type MatrixConfig = {
/**
* Streaming mode for Matrix replies.
* - `"partial"`: edit a single draft message in place for the current
* assistant block as the model generates text using normal Matrix text
* messages. This preserves legacy preview-first notification behavior.
* - `"quiet"`: edit a single quiet draft notice in place for the current
* assistant block as the model generates text.
* - `"off"`: deliver the full reply once the model finishes.
* - Use `blockStreaming: true` when you want completed assistant blocks to
* stay visible as separate progress messages. When combined with
* `"partial"`, Matrix keeps a live draft for the current block and
* preview streaming, Matrix keeps a live draft for the current block and
* preserves completed blocks as separate messages.
* - `true` maps to `"partial"`, `false` maps to `"off"`.
* - `true` maps to `"partial"`, `false` maps to `"off"` for backward
* compatibility.
* Default: `"off"`.
*/
streaming?: "partial" | "off" | boolean;
streaming?: MatrixStreamingMode | boolean;
};
export type CoreConfig = {

View File

@ -6692,7 +6692,7 @@ export const GENERATED_BUNDLED_CHANNEL_CONFIG_METADATA = [
anyOf: [
{
type: "string",
enum: ["partial", "off"],
enum: ["partial", "quiet", "off"],
},
{
type: "boolean",