docs: improve music generation docs

This commit is contained in:
Peter Steinberger 2026-04-06 01:59:07 +01:00
parent f4ffac6fe9
commit c5a310bf84
No known key found for this signature in database
3 changed files with 43 additions and 2 deletions

View File

@ -422,6 +422,19 @@ If you want to rely on env keys (e.g. exported in your `~/.profile`), run local
- Optional auth behavior:
- `OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1` to force profile-store auth and ignore env-only overrides
## Music generation live
- Test: `extensions/music-generation-providers.live.test.ts`
- Enable: `OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts`
- Scope:
- Exercises the shared bundled music-generation provider path
- Currently covers Google and MiniMax
- Loads provider env vars from your login shell (`~/.profile`) before probing
- Skips providers with no usable auth/profile/model
- Optional narrowing:
- `OPENCLAW_LIVE_MUSIC_GENERATION_PROVIDERS="google,minimax"`
- `OPENCLAW_LIVE_MUSIC_GENERATION_MODELS="google/lyria-3-clip-preview,minimax/music-2.5+"`
## Docker runners (optional "works in Linux" checks)
These Docker runners split into two buckets:

View File

@ -106,7 +106,7 @@ Plugins can register additional tools. Some examples:
- [Lobster](/tools/lobster) — typed workflow runtime with resumable approvals
- [LLM Task](/tools/llm-task) — JSON-only LLM step for structured output
- [Music Generation](/tools/music-generation) — plugin-provided `music_generate` tool surfaces
- [Music Generation](/tools/music-generation) — shared `music_generate` tool plus plugin-provided workflow variants
- [Diffs](/tools/diffs) — diff viewer and renderer
- [OpenProse](/prose) — markdown-first workflow orchestration

View File

@ -56,6 +56,16 @@ For direct synchronous contexts without a session-backed agent run, the built-in
tool still falls back to inline generation and returns the final media path in
the tool result.
Example prompts:
```text
Generate a cinematic piano track with soft strings and no vocals.
```
```text
Generate an energetic chiptune loop about launching a rocket at sunrise.
```
### Workflow-driven plugin generation
The bundled `comfy` plugin can also provide `music_generate` using a
@ -92,13 +102,25 @@ runtime:
/tool music_generate action=list
```
Use `action: "status"` to inspect the active session-backed music task:
```text
/tool music_generate action=status
```
Direct generation example:
```text
/tool music_generate prompt="Dreamy lo-fi hip hop with vinyl texture and gentle rain" instrumental=true
```
## Built-in tool parameters
| Parameter | Type | Description |
| ----------------- | -------- | ------------------------------------------------------------------------------------------------- |
| `prompt` | string | Music generation prompt (required for `action: "generate"`) |
| `action` | string | `"generate"` (default), `"status"` for the current session task, or `"list"` to inspect providers |
| `model` | string | Provider/model override, e.g. `google/lyria-3-pro-preview` or `comfy/workflow` |
| `model` | string | Provider/model override, e.g. `google/lyria-3-pro-preview` or `comfy/workflow` |
| `lyrics` | string | Optional lyrics when the provider supports explicit lyric input |
| `instrumental` | boolean | Request instrumental-only output when the provider supports it |
| `image` | string | Single reference image path or URL |
@ -161,6 +183,12 @@ error includes details from each attempt.
- ComfyUI support is workflow-driven and depends on the configured graph plus
node mapping for prompt/output fields.
## Choosing the right path
- Use the shared provider-backed path when you want model selection, provider failover, and the built-in async task/status flow.
- Use a plugin path such as ComfyUI when you need a custom workflow graph or a provider that is not part of the shared bundled music capability.
- If you are debugging ComfyUI-specific behavior, see [ComfyUI](/providers/comfy). If you are debugging shared provider behavior, start with [Google (Gemini)](/providers/google) or [MiniMax](/providers/minimax).
## Live tests
Opt-in live coverage for the shared bundled providers: