Merge remote-tracking branch 'origin/master' into timeout-design
# Conflicts: # docs/event-producer-consumer.md # packages/README.md
This commit is contained in:
Dudu-0223
@@ -1,6 +1,6 @@
|
||||
# AGENTS.md
|
||||
|
||||
This is the monorepo of the DeepSeek Harness group; it hosts **DeepSeek Harness SDK**, a plugin-based SDK for building agent harnesses. The codebase is built on the vendored Cordis framework, microkernel-style: **everything is a plugin**. Read [docs/architecture.md](docs/architecture.md) before changing `packages/`; the documentation standard is [docs/AGENTS.md](docs/AGENTS.md).
|
||||
This is the DeepSeek Harness group's monorepo; it hosts **DeepSeek Harness SDK**, a plugin-based SDK for building agent harnesses. The codebase is built on the vendored Cordis framework, microkernel-style: **everything is a plugin**. Read [docs/architecture.md](docs/architecture.md) before changing `packages/`; the documentation standard is [docs/AGENTS.md](docs/AGENTS.md).
|
||||
|
||||
## Pre-release stance: foundation over blast radius
|
||||
|
||||
@@ -19,11 +19,12 @@ packages/ Harness packages at packages/<group>/<pkg>/, all named @deepseek-ai
|
||||
compact/ compaction seam + basic backend
|
||||
subagent/ subagent seam + spawn/fork/ACP backends + delegation tool
|
||||
todo/ the todo_write tool
|
||||
guard/ loop-hygiene plugins
|
||||
hooks/ Claude Code / Codex hook bridges + shared wire-protocol library
|
||||
session-persistence/ persistence seam + JSONL/SQLite backends
|
||||
ui/ ACP bridge + app-boot glue + the stdio/ACP app bins
|
||||
support/ dev/test infrastructure: invariants, llm-replay, subagent-mock
|
||||
util/ zero-dependency utilities (Branded<B>)
|
||||
support/ dev/test infrastructure packages
|
||||
util/ zero-dependency utilities
|
||||
examples/ Runnable demos: thin cordis.yml leaves over the app packages (see examples/AGENTS.md)
|
||||
docs/ architecture, generated catalogs, RFCs, postmortems, cookbook (see docs/AGENTS.md)
|
||||
scripts/ repo gates and generators
|
||||
@@ -69,7 +70,7 @@ printf '%s\n' "$out" | grep -q '\[tool call\] echo({"text":"ci smoke"})'
|
||||
printf '%s\n' "$out" | grep -q '\[tool result\] ECHO: CI SMOKE'
|
||||
ls .sessions/_no-cwd/main-session-*.jsonl >/dev/null
|
||||
rm -rf .sessions
|
||||
pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts
|
||||
pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts
|
||||
```
|
||||
|
||||
`test:coverage`, not `test`, is the gating run ([why](docs/testing.md)); a sign-off counts only for commands actually run.
|
||||
|
||||
@@ -44,6 +44,7 @@ flowchart LR
|
||||
pkg_hooks_codex["hooks-codex"]
|
||||
pkg_code_runtime["code-runtime"]
|
||||
svc_codeRuntime["ctx.codeRuntime<br/>Code-execution seam"]
|
||||
pkg_code_runtime_worker["code-runtime-worker"]
|
||||
pkg_fs["fs"]
|
||||
svc_fs["ctx.fs<br/>Filesystem provider seam"]
|
||||
pkg_fs_local["fs-local"]
|
||||
@@ -67,6 +68,7 @@ flowchart LR
|
||||
pkg_bash --> svc_bash
|
||||
pkg_bash_local --> svc_bash
|
||||
pkg_code_runtime --> svc_codeRuntime
|
||||
pkg_code_runtime_worker --> svc_codeRuntime
|
||||
pkg_compact --> svc_compact
|
||||
pkg_compact_basic --> svc_compact
|
||||
pkg_fs --> svc_fs
|
||||
@@ -137,7 +139,7 @@ flowchart LR
|
||||
| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/ui/acp), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`stdio-agent`](../packages/ui/stdio-agent), [`invariants`](../packages/support/invariants) | - | Owns live Agent handles and the create/resume factory seam. |
|
||||
| `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-core`](../packages/core/agent-core) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. |
|
||||
| `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local) | [`tool-bash`](../packages/bash/tool-bash), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors can replace bash-local. |
|
||||
| `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | - | - | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the Code Mode RFC specifies the worker-thread backend and the tool-registry consumer). |
|
||||
| `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | - | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the Code Mode RFC specifies the worker-thread backend and the tool-registry consumer). |
|
||||
| `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-policy contributes observed-state checks through the fs/* event gate. |
|
||||
| `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend currently consumes the pre-step event directly; a model-facing compact tool remains deferred. |
|
||||
| `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-mock`](../packages/support/subagent-mock) | [`tool-subagent`](../packages/subagent/tool-subagent) | - | Providers implement transports; tool-subagent exposes one configured provider as a model-facing tool name. |
|
||||
|
||||
@@ -141,6 +141,42 @@ export interface Config {
|
||||
|
||||
Source: [`packages/bash/bash-local/src/index.ts:29`](../packages/bash/bash-local/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-code-runtime-worker`
|
||||
|
||||
```ts config-catalog
|
||||
/** Plugin config: every execution cap, changeable from `cordis.yml` (no hardcoded tunables). */
|
||||
export interface Config {
|
||||
/**
|
||||
* Busy-time budget in milliseconds: the run fails with kind `'timeout'`
|
||||
* once the worker's MEASURED event-loop active time
|
||||
* (`worker.performance.eventLoopUtilization()`) exceeds this. Metering
|
||||
* measured busy time — not wall time, not host-side pending-call
|
||||
* bookkeeping — is what makes the budget both fair (a program awaiting a
|
||||
* slow tool accrues nothing) and ungameable (a hot loop accrues whether
|
||||
* or not a decoy dispatch is in flight).
|
||||
*/
|
||||
computeMs?: number
|
||||
/**
|
||||
* Wall-clock ceiling in milliseconds; never pauses for anything. The
|
||||
* backstop for what busy-time cannot see (a program awaiting a promise
|
||||
* nobody will resolve).
|
||||
*/
|
||||
maxWallMs?: number
|
||||
/** Shared byte budget for captured log text (console + raw stream writes), truncation marked in-band. */
|
||||
maxLogBytes?: number
|
||||
/**
|
||||
* Byte cap for the completion value, measured by its real cross-boundary
|
||||
* size (string bytes, or structured-clone wire size); an oversized or
|
||||
* non-cloneable value crosses as a capped string rendering.
|
||||
*/
|
||||
maxValueBytes?: number
|
||||
/** The worker's max old-generation heap in MiB (`resourceLimits`); overflow kills the worker, surfacing as kind `'worker-exit'`. */
|
||||
maxOldGenerationSizeMb?: number
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:29`](../packages/code-runtime/code-runtime-worker/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-compact-basic`
|
||||
|
||||
Requires: `llm`
|
||||
@@ -353,6 +389,38 @@ export interface Config {
|
||||
|
||||
Source: [`packages/support/llm-replay/src/index.ts:429`](../packages/support/llm-replay/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-repeat-tool-guard`
|
||||
|
||||
```ts config-catalog
|
||||
/**
|
||||
* Plugin config, validated by the same-named schemastery schema plus the
|
||||
* load-time checks in `apply` (misconfiguration fails loud: an empty
|
||||
* `thresholds` list, a non-integer, a value below 2, or a duplicate throws at
|
||||
* plugin load, never a silent fall-back). `include`/`exclude` entries are
|
||||
* `*`-wildcard predicates over tool names at call time, not references to
|
||||
* registry entries — a pattern matching no currently registered tool is valid
|
||||
* (`exclude: [mcp_*]` must stay legal in a deployment that loads no MCP tools).
|
||||
*/
|
||||
export interface Config {
|
||||
/** Consecutive-repeat counts that trigger a reminder (default `[3, 5, 8]`). */
|
||||
thresholds?: number[]
|
||||
/** Tool-name patterns to track; empty means every tool is tracked. */
|
||||
include?: string[]
|
||||
/** Tool-name patterns transparent to the chain (neither count nor reset). */
|
||||
exclude?: string[]
|
||||
/**
|
||||
* Maximum characters of canonical arguments quoted in the DETAILED reminder
|
||||
* (default 500). Large payloads (a `write` body, a long command) would
|
||||
* otherwise ride into the next request unbounded — precisely in a loop
|
||||
* scenario; the cap bounds the reminder, never the detection (the chain key
|
||||
* always compares the FULL canonical string).
|
||||
*/
|
||||
argumentsPreviewChars?: number
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/guard/repeat-tool-guard/src/index.ts:55`](../packages/guard/repeat-tool-guard/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-session-persistence-jsonl`
|
||||
|
||||
Requires: `sessions`
|
||||
|
||||
@@ -11,11 +11,11 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:271`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:420`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - |
|
||||
| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:349`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) |
|
||||
| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:362`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:362`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) |
|
||||
| `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:289`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - |
|
||||
| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:385`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - |
|
||||
| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:304`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:280`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:280`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:395`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - |
|
||||
| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:408`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:123`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
@@ -33,7 +33,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:44`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - |
|
||||
| `tools/change` | `emit` | [`packages/core/tools/src/index.ts:118`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - |
|
||||
| `tools/execute` | `waterfall` | [`packages/core/tools/src/index.ts:97`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`timeout-policy`](../packages/timeout/timeout-policy) |
|
||||
| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:113`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:113`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) |
|
||||
| `tools/pre-execute` | `waterfall` | [`packages/core/tools/src/index.ts:77`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
|
||||
Maintenance mode: hybrid generated: Cordis event declarations and most producer/listener edges are AST-scanned; dynamic dispatch sites are classified in `scripts/gen-doc-graphs.ts`.
|
||||
@@ -85,9 +85,14 @@ flowchart TD
|
||||
end
|
||||
subgraph group_code_runtime["packages/code-runtime"]
|
||||
pkg_code_runtime["code-runtime"]
|
||||
pkg_code_runtime_worker["code-runtime-worker"]
|
||||
end
|
||||
subgraph group_guard["packages/guard"]
|
||||
pkg_repeat_tool_guard["repeat-tool-guard"]
|
||||
end
|
||||
pkg_llm --> pkg_brand
|
||||
pkg_bash --> pkg_brand
|
||||
pkg_code_runtime_worker --> pkg_code_runtime
|
||||
pkg_llm_deepseek --> pkg_llm
|
||||
pkg_llm_pi_ai --> pkg_llm
|
||||
pkg_session --> pkg_brand
|
||||
@@ -169,6 +174,8 @@ flowchart TD
|
||||
pkg_acp --> pkg_session
|
||||
pkg_acp --> pkg_session_persistence
|
||||
pkg_acp --> pkg_tools
|
||||
pkg_repeat_tool_guard --> pkg_agent
|
||||
pkg_repeat_tool_guard --> pkg_tools
|
||||
pkg_agent_core --> pkg_agent
|
||||
pkg_agent_core --> pkg_agent_loop
|
||||
pkg_agent_core --> pkg_invariants
|
||||
@@ -226,6 +233,7 @@ flowchart TD
|
||||
| [`code-runtime`](../packages/code-runtime/code-runtime) | `code-runtime` | — |
|
||||
| [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand) |
|
||||
| [`bash`](../packages/bash/bash) | `bash` | [`brand`](../packages/util/brand) |
|
||||
| [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime) |
|
||||
| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) |
|
||||
| [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) |
|
||||
| [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm) |
|
||||
@@ -258,6 +266,7 @@ flowchart TD
|
||||
| [`tool-todo`](../packages/todo/tool-todo) | `todo` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`tools`](../packages/core/tools) |
|
||||
| [`hooks-codex`](../packages/hooks/hooks-codex) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`tools`](../packages/core/tools) |
|
||||
| [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`tools`](../packages/core/tools) |
|
||||
| [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) |
|
||||
| [`agent-core`](../packages/core/agent-core) | `core` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tool-bash`](../packages/bash/tool-bash), [`tools`](../packages/core/tools) |
|
||||
| [`subagent-acp`](../packages/subagent/subagent-acp) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent) |
|
||||
| [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
|
||||
|
||||
+1
-1
@@ -13,7 +13,6 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
|
||||
| [Code Mode — the model writes TypeScript against the tool registry](proposed/feature/2026-06-15-code-mode.md) | 2026-06-15 |
|
||||
| [Pre-tool input rewrite — a consistent design](proposed/feature/2026-06-30-pre-tool-input-rewrite.md) | 2026-06-30 |
|
||||
| [Claude Code and Codex subagent backends (out-of-process delegation to external coding agents)](proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md) | 2026-07-07 |
|
||||
| [Repeat-tool-call guard plugin](proposed/feature/2026-07-08-repeat-tool-guard.md) | 2026-07-08 |
|
||||
|
||||
### Simplification
|
||||
|
||||
@@ -63,6 +62,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
|
||||
| [SessionStore fork API](implemented/feature/2026-06-30-session-store-fork-api.md) | 2026-06-30 |
|
||||
| [Subagent lifecycle enrichment — lastAssistantMessage (observe-only)](implemented/feature/2026-06-30-subagent-observe-enrich.md) | 2026-06-30 |
|
||||
| [Explicit model-facing tool order](implemented/feature/2026-07-06-explicit-tool-order.md) | 2026-07-06 |
|
||||
| [Repeat-tool-call guard plugin](implemented/feature/2026-07-08-repeat-tool-guard.md) | 2026-07-08 |
|
||||
|
||||
### Simplification
|
||||
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# RFC: Repeat-tool-call guard plugin
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
A model stuck in a loop re-issues the same tool call with byte-identical arguments — re-running a failing grep, re-reading an unchanged file, polling a command that already gave its answer — and each round trip burns tokens, wall-clock, and (for paid APIs) money without adding information. The harness has nothing that notices: the loop has no step budget, no plugin tracks call repetition, and the model only escapes when it happens to vary its own behavior. The failure mode is real and cheap to detect — [pi-repeat-tool-guard](https://github.com/Kingwl/pi-repeat-tool-guard) ships exactly this as a pi coding-agent extension: count consecutive identical calls and, past a threshold, append a `<system-reminder>` telling the model to stop repeating itself and change course.
|
||||
|
||||
The harness already has every seam the pi extension uses, and better ones: [the interception-seams RFC](2026-06-30-interception-seams.md) gives `tools/post-execute` a sanctioned way to attach model-facing context to a finished call, the loop buffers and injects that context with call/result adjacency preserved, and injected context is a logged `context/message` — so a native guard satisfies the model-visible ⟺ logged rule with no new session event. What was missing was only the plugin itself.
|
||||
|
||||
## Decision
|
||||
|
||||
The guard is a loop-hygiene plugin, not a model-facing tool: it never appears in the tool list, never vetoes or rewrites a call, and adds exactly one behavior — it watches each agent's stream of tool calls, counts runs of consecutive calls to the same tool with identical canonicalized arguments, and at configured run lengths injects an escalating advisory reminder telling the model to stop repeating itself, re-read the last result, and either change approach or conclude. The purpose is to break unproductive loops within a few wasted steps instead of letting them run to the turn's natural end — while leaving the decision (retry differently, gather more evidence, or finish) entirely with the model, so a legitimately repeated call is delayed by nothing and blocked by nothing.
|
||||
|
||||
The plugin is `@deepseek-ai/dsh-repeat-tool-guard` at `packages/guard/repeat-tool-guard/`, opening the `guard/` group for loop-hygiene plugins (single-package groups have precedent: [the todo-write RFC](2026-06-29-todo-write-tool.md) shipped `todo/tool-todo`). It registers three listeners and holds all state in plugin-local maps keyed by `AgentId` — the tool registry is a context-level singleton whose waterfalls interleave every agent's calls (subagents run on the same context), so per-agent keying is correctness, not polish.
|
||||
|
||||
- **`tools/post-execute` (waterfall)** — the one detection point. The listener receives `(exec, result)` together, so counting and reminder delivery need no cross-event pending map (the pi extension needs one only because its `tool_call`/`tool_result` hooks are separate events). It always delegates via `next()` and, when a threshold is hit, folds a reminder onto the downstream decision's `additionalContext` — the observe-and-enrich posture [the hooks bridges](2026-06-30-hook-bridges.md) already use, honoring the waterfall contract. Counting happens here rather than in `tools/pre-execute` because post-execute also runs for denied calls (`ToolRegistry.execute` routes a deny through the same pipeline), and a model hammering a denied call is exactly the loop worth breaking.
|
||||
- **`agent/prompt-submit` (waterfall)** — pure reset hook: delegate via `next()`, clear the submitting agent's chain. A user interjection changes the context; repetition across it is not a loop.
|
||||
- **`agent/status` (emit)** — on `disposed`, drop the agent's state, bounding the maps over harness lifetime.
|
||||
|
||||
### Detection semantics
|
||||
|
||||
The chain key is `(tool name, canonical arguments)`; a call identical to the previous tracked call increments the agent's consecutive counter, a different tracked call resets it to 1. Canonicalization is a deep key-sort plus `JSON.stringify`: `ToolExecution.arguments` is by construction the loop's `JSON.parse` output (or the raw string fallback for malformed argument JSON, which is itself a comparable value), so the pi original's bigint/circular/`undefined` handling has no inputs here and is deliberately dropped.
|
||||
|
||||
Two deliberate rules, both documented in [the package README](../../../../packages/guard/repeat-tool-guard/README.md) because they are behavior a reader would otherwise guess at:
|
||||
|
||||
- **Untracked calls are transparent to the chain.** A call excluded by `include`/`exclude` neither increments nor resets the counter, so `grep X → todo_write → grep X` still counts as two consecutive `grep X` when `todo_write` is excluded. This is what makes exclusion useful — bookkeeping tools interleaved into a loop must not launder it — and it is the pi extension's (undocumented) semantics, kept on purpose and written down.
|
||||
- **Calls without an agent are ignored.** A direct `ctx.tools.execute()` caller (tests, non-loop consumers) has no model to remind and no `AgentId` to key on.
|
||||
|
||||
### Reminder delivery
|
||||
|
||||
Reminders ride `additionalContext` (source `{kind: 'plugin', plugin: 'repeat-tool-guard'}` — the label is load-bearing per `HookContext`), never a `content` replacement: the `tool/result` event stays the tool's own output for audit, and the loop appends buffered context as `context/message`(s) after the step's results, which the session renders as the tagged synthetic-user envelope and derived history replays. Thresholds escalate: the first configured threshold gets a short "you are repeating yourself, analyze the previous result" nudge; each later threshold gets the detailed form naming the tool, the repeat count, and the canonical arguments (head-truncated at `argumentsPreviewChars`, default 500 — a looping `write`-sized payload must not ride into the next request unbounded; the chain key always compares the full canonical string), and stating that the calls made no progress. The pi original hardcodes the gentle text to the literal count 3; the guard keys it to `thresholds[0]`, fixing that bug in the port. When the downstream decision already carries `additionalContext` (a hook bridge on the same call), the guard concatenates content under its own `source` — a `HookContext` holds one `MessageSource`, and `source.kind` is what framing depends on.
|
||||
|
||||
### Config
|
||||
|
||||
```yaml
|
||||
- id: repeat-tool-guard
|
||||
name: '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
config:
|
||||
thresholds: [3, 5, 8] # default; consecutive counts that trigger a reminder
|
||||
include: [] # tool-name patterns to track; empty ⇒ all tools
|
||||
exclude: [todo_write] # tool-name patterns transparent to the chain
|
||||
argumentsPreviewChars: 500 # default; cap on arguments quoted in the detailed reminder
|
||||
```
|
||||
|
||||
`thresholds` is validated at load and throws on an empty list, a non-integer, a value below 2, or a duplicate — misconfiguration fails loud, replacing the pi original's silent fall-back to defaults. `include`/`exclude` entries support `*` wildcards. Patterns are predicates over whatever tools exist at call time, not references to a registry entry, so an entry matching no currently registered tool is NOT an error — unlike `toolOrder`'s referent check, `exclude: [mcp_*]` must stay valid in a deployment that loads no MCP tools.
|
||||
|
||||
## Testing
|
||||
|
||||
**Unit** — the suite drives a real agent loop against a scripted mock adapter (no network) and covers, at per-file 100%: counting/reset semantics (identical, different-tracked, untracked-transparent, prompt-submit reset, disposal cleanup, per-agent isolation), canonicalization (deep key-order insensitivity), threshold escalation including the `thresholds[0]` gentle-text rule, denied-call counting, no-agent transparency, wildcard escaping, config fail-loud cases, and both fold-onto-downstream paths (block and accept-with-replacement). **Snapshot** — the `repeat-tool-guard` scenario in the acp-agent example suite scripts five identical `todo_write` calls and pins both reminder tiers (gentle at the third, detailed at the fifth) as `context/message`s in the ACP transcript and the session log; the guard is loaded in the example's live tree (`cordis.yml`), inert for every other scenario (none repeats a call three times). The scenario is authored keyless (like `error-finish`/`cancel`): deterministically forcing a live model to repeat one call three times is not a stable recording. **e2e** — none: the plugin is provider-independent and deterministic, and the seam contracts it relies on are e2e-covered by their owners.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Append the reminder into the tool result** (`accept` with replaced `content` — the pi extension's mechanism, which patches result content because that is the only channel its API offers) — rejected: it makes the logged `tool/result` lie about what the tool returned, and `additionalContext` exists precisely as the separate sanctioned channel for post-execute commentary, with loop-level buffering that preserves call/result adjacency.
|
||||
- **Count in `tools/pre-execute` with a pending-reminder map** (the pi two-phase shape) — rejected: post-execute alone sees `(exec, result)` together and also fires for denied calls, so one listener with no cross-event state covers strictly more attempts with less machinery.
|
||||
- **Escalate to `block` at the highest threshold** — rejected for the initial scope: a blocked call punishes legitimate identical repeats (polling a long-running terminal, re-checking a file the agent expects to change), and an advisory reminder keeps the model in control. Revisit with evidence; the decision shape (`PostToolDecision`) already supports it.
|
||||
- **A per-deployment external hook via the CC/Codex bridges** (a `PostToolUse` script) — rejected as the answer: it works for one deployment, but a shipped, unit-tested, `cordis.yml`-configurable plugin is the harness-native form, without per-call subprocess cost.
|
||||
- **A loop-level step or repetition budget in `agent-loop`** — rejected: "plugins, not loop changes"; a hard step budget is a blunter, orthogonal control that would need its own proposal.
|
||||
- **Fuzzy/near-identical detection** (normalized paths, similar-but-not-equal arguments) — rejected: exact match after canonicalization is cheap, deterministic, and explainable to the model; similarity thresholds invite false positives and need evidence before they earn complexity.
|
||||
- **Placing the package in `core/`** — rejected: core is the product spine; a behavioral guard is an optional leaf plugin, and the `todo/` precedent is a small dedicated group per plugin family.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The reminder is advisory by design: idempotent polling patterns that repeat identical calls on purpose still receive nudges past the thresholds, and the pressure valves are config (`thresholds`, `exclude`) plus reminder text that explicitly allows finishing when enough evidence has been gathered. Each trigger costs reminder tokens on the next request; thresholds bound the frequency.
|
||||
- Chain state is in-memory only: a session resumed from persistence starts with a fresh chain, so a loop spanning a resume draws its reminders later than a live one — accepted, the guard is a heuristic nudge, not a logged invariant, and persisting counter state would buy little for real complexity.
|
||||
- When multiple post-execute producers attach context on one call, the fold concatenates under the guard's `source`; ordering between plugins follows listener registration order. The seam cannot represent mixed provenance — a limit inherited from `HookContext`, not owned by this plugin.
|
||||
- Implementing the snapshot tier surfaced a hidden assumption in the suite kit: the fixture guard equated "authored model scenario" with "override-driven". The `Scenario` table now carries an explicit `overridden` flag, and the sidecar's presence is checked BOTH ways against it (an unregistered stray sidecar would silently replace the derived script) — the suite kit is stricter than it was before this plugin existed.
|
||||
|
||||
## Deferred
|
||||
|
||||
- Compaction does not reset chains: a compacted history changes what the model sees, but the repetition risk usually survives compaction.
|
||||
- Escalating to `block` at a high threshold is not implemented; `PostToolDecision` already supports it if evidence arrives.
|
||||
- Subagent chains stay isolated per agent; no sharing mechanism exists until a concrete case appears.
|
||||
@@ -68,7 +68,7 @@ The replay plugin lives in its own package, `@deepseek-ai/dsh-llm-replay` (`pack
|
||||
|
||||
### Two subcommands, replay in the default gate
|
||||
|
||||
`pnpm run test:snapshot` runs replay (keyless) and is composed into the default `pnpm run test` gate so every PR gets the regression check (the main `vitest.config.ts` include stays narrow; the gate is `test && test:snapshot`). `pnpm run test:snapshot:record` requires `DEEPSEEK_API_KEY` (loaded from repo `.env` first), hits the real API, harvests the produced `session.jsonl` (the replay source AND the expected-log artifact), and `--update`s the stdout golden in one pass. Both forward a scenario filter. A missing fixture in replay **fails loud** with a "record first" message rather than self-skipping (the e2e self-skip rule is a CI-secret accommodation, not appropriate here — a committed-fixture test that silently vanishes is a coverage hole). A no-model scenario's `session.jsonl` simply has no `assistant/chunk` events (empty derived script); fail-loud still applies if a model call happens with no entry. An orphan-fixture guard test fails on a golden/fixture not referenced by any scenario (Vitest does not prune orphaned raw goldens), and a per-kind required-fixture guard asserts each scenario ships exactly the files its kind needs (`input.json` + `stdout.golden.jsonl` + `session.jsonl` for ALL scenarios — the harness passes `<dir>/session.jsonl` to `llm-replay` unconditionally, so even a no-model scenario needs its header-only fixture or `loadReplayScript()` fails; `replay.override.json` additionally for authored model scenarios).
|
||||
`pnpm run test:snapshot` runs replay (keyless) and is composed into the default `pnpm run test` gate so every PR gets the regression check (the main `vitest.config.ts` include stays narrow; the gate is `test && test:snapshot`). `pnpm run test:snapshot:record` requires `DEEPSEEK_API_KEY` (loaded from repo `.env` first), hits the real API, harvests the produced `session.jsonl` (the replay source AND the expected-log artifact), and `--update`s the stdout golden in one pass. Both forward a scenario filter. A missing fixture in replay **fails loud** with a "record first" message rather than self-skipping (the e2e self-skip rule is a CI-secret accommodation, not appropriate here — a committed-fixture test that silently vanishes is a coverage hole). A no-model scenario's `session.jsonl` simply has no `assistant/chunk` events (empty derived script); fail-loud still applies if a model call happens with no entry. An orphan-fixture guard test fails on a golden/fixture not referenced by any scenario (Vitest does not prune orphaned raw goldens), and a per-kind required-fixture guard asserts each scenario ships exactly the files its kind needs (`input.json` + `stdout.golden.jsonl` + `session.jsonl` for ALL scenarios — the harness passes `<dir>/session.jsonl` to `llm-replay` unconditionally, so even a no-model scenario needs its header-only fixture or `loadReplayScript()` fails; `replay.override.json` exactly for the scenarios whose table entry sets `overridden` — required with the flag, forbidden without it, because the harness forwards the sidecar purely on file existence and an unregistered stray would silently replace the derived script).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -1,79 +0,0 @@
|
||||
# RFC: Repeat-tool-call guard plugin
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Problem
|
||||
|
||||
A model stuck in a loop re-issues the same tool call with byte-identical arguments — re-running a failing grep, re-reading an unchanged file, polling a command that already gave its answer — and each round trip burns tokens, wall-clock, and (for paid APIs) money without adding information. The harness has nothing that notices: the loop has no step budget, no plugin tracks call repetition, and the model only escapes when it happens to vary its own behavior. The failure mode is real and cheap to detect — [pi-repeat-tool-guard](https://github.com/Kingwl/pi-repeat-tool-guard) ships exactly this as a pi coding-agent extension: count consecutive identical calls and, past a threshold, append a `<system-reminder>` telling the model to stop repeating itself and change course.
|
||||
|
||||
The harness already has every seam the pi extension uses, and better ones: [the interception-seams RFC](../../implemented/feature/2026-06-30-interception-seams.md) gives `tools/post-execute` a sanctioned way to attach model-facing context to a finished call, the loop buffers and injects that context with call/result adjacency preserved, and injected context is a logged `context/message` — so a native guard satisfies the model-visible ⟺ logged rule with no new session event. What is missing is only the plugin itself.
|
||||
|
||||
## Proposal
|
||||
|
||||
The guard is a loop-hygiene plugin, not a model-facing tool: it never appears in the tool list, never vetoes or rewrites a call, and adds exactly one behavior — it watches each agent's stream of tool calls, counts runs of consecutive calls to the same tool with identical canonicalized arguments, and at configured run lengths injects an escalating advisory reminder telling the model to stop repeating itself, re-read the last result, and either change approach or conclude. The purpose is to break unproductive loops within a few wasted steps instead of letting them run to the turn's natural end — while leaving the decision (retry differently, gather more evidence, or finish) entirely with the model, so a legitimately repeated call is delayed by nothing and blocked by nothing.
|
||||
|
||||
The shape: one new leaf plugin package, `@deepseek-ai/dsh-repeat-tool-guard` at `packages/guard/repeat-tool-guard/`, opening a `guard/` group for loop-hygiene plugins (single-package groups have precedent: [the todo-write RFC](../../implemented/feature/2026-06-29-todo-write-tool.md) shipped `todo/tool-todo`). The plugin registers three listeners via `ctx.effect()` and holds all state in plugin-local maps keyed by `AgentId` — the tool registry is a context-level singleton whose waterfalls interleave every agent's calls (subagents run on the same context), so per-agent keying is correctness, not polish.
|
||||
|
||||
- **`tools/post-execute` (waterfall)** — the one detection point. The listener receives `(exec, result)` together, so counting and reminder delivery need no cross-event pending map (the pi extension needs one only because its `tool_call`/`tool_result` hooks are separate events). It always delegates via `next()` and, when a threshold is hit, folds a reminder onto the downstream decision's `additionalContext` — the observe-and-enrich posture [the hooks bridges](../../implemented/feature/2026-06-30-hook-bridges.md) already use, honoring the waterfall contract. Counting happens here rather than in `tools/pre-execute` because post-execute also runs for denied calls (`ToolRegistry.execute` routes a deny through the same pipeline), and a model hammering a denied call is exactly the loop worth breaking.
|
||||
- **`agent/prompt-submit` (waterfall)** — pure reset hook: delegate via `next()`, clear the submitting agent's chain. A user interjection changes the context; repetition across it is not a loop.
|
||||
- **`agent/status` (emit)** — on `disposed`, drop the agent's state, bounding the maps over harness lifetime.
|
||||
|
||||
### Detection semantics
|
||||
|
||||
The chain key is `(tool name, canonical arguments)`; a call identical to the previous tracked call increments the agent's consecutive counter, a different tracked call resets it to 1. Canonicalization is a deep key-sort plus `JSON.stringify`: `ToolExecution.arguments` is by construction the loop's `JSON.parse` output (or the raw string fallback for malformed argument JSON, which is itself a comparable value), so the pi original's bigint/circular/`undefined` handling has no inputs here and is deliberately dropped.
|
||||
|
||||
Two deliberate rules, both documented in the package README because they are behavior a reader would otherwise guess at:
|
||||
|
||||
- **Untracked calls are transparent to the chain.** A call excluded by `include`/`exclude` neither increments nor resets the counter, so `grep X → todo_write → grep X` still counts as two consecutive `grep X` when `todo_write` is excluded. This is what makes exclusion useful — bookkeeping tools interleaved into a loop must not launder it — and it is the pi extension's (undocumented) semantics, kept on purpose and written down.
|
||||
- **Calls without an agent are ignored.** A direct `ctx.tools.execute()` caller (tests, future non-loop consumers) has no model to remind and no `AgentId` to key on.
|
||||
|
||||
### Reminder delivery
|
||||
|
||||
Reminders ride `additionalContext` (source `{kind: 'plugin', plugin: 'repeat-tool-guard'}` — the label is load-bearing per `HookContext`), never a `content` replacement: the `tool/result` event stays the tool's own output for audit, and the loop already appends buffered context as `context/message`(s) after the step's results, which the session renders as the tagged synthetic-user envelope and derived history replays. Thresholds escalate: the first configured threshold gets a short "you are repeating yourself, analyze the previous result" nudge; each later threshold gets the detailed form naming the tool, the repeat count, and the canonical arguments, and stating that the calls made no progress. The pi original hardcodes the gentle text to the literal count 3; the guard keys it to `thresholds[0]`, fixing that bug in the port. When the downstream decision already carries `additionalContext` (a hook bridge on the same call), the guard folds content following the shared-merge precedent in `dsh-hook-protocol`.
|
||||
|
||||
### Config
|
||||
|
||||
```yaml
|
||||
- id: repeat-tool-guard
|
||||
name: '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
config:
|
||||
thresholds: [3, 5, 8] # default; consecutive counts that trigger a reminder
|
||||
include: [] # tool-name patterns to track; empty ⇒ all tools
|
||||
exclude: [todo_write] # tool-name patterns transparent to the chain
|
||||
```
|
||||
|
||||
`thresholds` is validated at load and throws on an empty list, a non-integer, a value below 2, or a duplicate — misconfiguration fails loud, replacing the pi original's silent fall-back to defaults. `include`/`exclude` entries support `*` wildcards. Patterns are predicates over whatever tools exist at call time, not references to a registry entry, so an entry matching no currently registered tool is NOT an error — unlike `toolOrder`'s referent check, `exclude: [mcp_*]` must stay valid in a deployment that loads no MCP tools.
|
||||
|
||||
### Testing
|
||||
|
||||
Coverage named at plan time, per tier: **unit** — counting/reset semantics (identical, different-tracked, untracked-transparent, prompt-submit reset, disposal cleanup, per-agent isolation), canonicalization, threshold escalation including the `thresholds[0]` gentle-text rule, config fail-loud cases, and the fold-onto-downstream-decision path, to per-file 100% like every `packages/*/*/src` file. **Snapshot** — one scripted-replay scenario where the model repeats a call to threshold and the reminder `context/message` appears in the transcript, pinning the model-visible text and its envelope (this is a transcript-surface change; the ACP snapshot suite is the tier that owns it). **e2e** — none: the plugin is provider-independent and deterministic, and forcing a live model to repeat a call three times is not a stable test; the seam contracts it relies on are already e2e-covered by their owners.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Append the reminder into the tool result** (`accept` with replaced `content` — the pi extension's mechanism, which patches result content because that is the only channel its API offers) — rejected: it makes the logged `tool/result` lie about what the tool returned, and `additionalContext` exists precisely as the separate sanctioned channel for post-execute commentary, with loop-level buffering that preserves call/result adjacency.
|
||||
- **Count in `tools/pre-execute` with a pending-reminder map** (the pi two-phase shape) — rejected: post-execute alone sees `(exec, result)` together and also fires for denied calls, so one listener with no cross-event state covers strictly more attempts with less machinery.
|
||||
- **Escalate to `block` at the highest threshold** — rejected for the initial scope: a blocked call punishes legitimate identical repeats (polling a long-running terminal, re-checking a file the agent expects to change), and an advisory reminder keeps the model in control. Revisit with evidence; the decision shape (`PostToolDecision`) already supports it.
|
||||
- **A per-deployment external hook via the CC/Codex bridges** (a `PostToolUse` script) — rejected as the answer: it works today for one deployment, but a shipped, unit-tested, `cordis.yml`-configurable plugin is the harness-native form, without per-call subprocess cost.
|
||||
- **A loop-level step or repetition budget in `agent-loop`** — rejected: "plugins, not loop changes"; a hard step budget is a blunter, orthogonal control that would need its own proposal.
|
||||
- **Fuzzy/near-identical detection** (normalized paths, similar-but-not-equal arguments) — rejected: exact match after canonicalization is cheap, deterministic, and explainable to the model; similarity thresholds invite false positives and need evidence before they earn complexity.
|
||||
- **Placing the package in `core/`** — rejected: core is the product spine; a behavioral guard is an optional leaf plugin, and the `todo/` precedent is a small dedicated group per plugin family.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `packages/guard/repeat-tool-guard/` exists, registers all listeners through `ctx.effect()`, and is loadable from a `cordis.yml` with the config above; the config catalog regenerates with its entry.
|
||||
- Invalid `thresholds` (empty, non-integer, `< 2`, duplicate) throw at plugin load.
|
||||
- Unit suite covers the semantics list above at per-file 100%; a snapshot scenario replays a threshold-crossing repetition and pins the reminder `context/message` in the transcript on macOS and Linux.
|
||||
- The reminder is reconstructable from the session log alone (it is an ordinary `context/message` with a plugin source — no new session event).
|
||||
- The package README opens with the plugin's purpose — an advisory loop-breaker that is not a model-facing tool, never blocks or rewrites a call, and only injects reminders — then documents the transparency rule, the per-agent keying, and the in-memory-only state; `doc-sync` is green.
|
||||
|
||||
## Risks
|
||||
|
||||
- **False positives on legitimately repeated calls.** Idempotent polling patterns repeat identical calls on purpose; the reminder is advisory and thresholds/`exclude` are the pressure valves, but a badly tuned deployment adds noise to the transcript. Mitigation: conservative defaults and the reminder text explicitly allowing "finish the task if enough evidence has been gathered".
|
||||
- **Reminder tokens are model-visible cost.** Each trigger appends a paragraph to the next request; thresholds bound the frequency, but a pathological agent can hit 3/5/8 repeatedly across different keys.
|
||||
- **State is in-memory only.** A session resumed from persistence starts with a fresh chain, so a loop spanning a resume gets its reminders later than a live one — accepted: the guard is a heuristic nudge, not a logged invariant, and persisting counter state would buy little for real complexity.
|
||||
- **Multiple context producers on one call.** When a hook bridge and the guard both attach `additionalContext`, ordering follows listener registration order; the fold keeps both, but the combined envelope's readability depends on merge behavior that this RFC inherits rather than owns.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should compaction reset chains? A compacted history changes what the model sees, but the repetition risk usually survives compaction; the initial answer is no.
|
||||
- Should subagents inherit the parent's thresholds via config only, or ever share chain state? Per-agent isolation is the proposed default; sharing looks like a smell until a concrete case appears.
|
||||
+1
-1
@@ -25,7 +25,7 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
|
||||
|
||||
- A plugin shipped via `cordis.yml` needs at least one test through the REAL Loader path: hand-built `ctx.plugin({...})` mounts bypass `unwrapExports` and cannot catch a broken export shape ([postmortem 0001](postmortem/0001-acp-default-export-drops-inject.md); export-shape rules in [packages/AGENTS.md](../packages/AGENTS.md)).
|
||||
- A guard only guards if the regression actually fails it. For a plugin without `inject` (bundle/composition plugins), a Loader smoke stays green under a broken export shape — add an explicit `expect('default' in mod).toBe(false)` plus an `unwrapExports` round-trip assertion, and prove it: introduce the regression, watch red, revert.
|
||||
- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). Keep the built-bin smokes green (`packages/ui/*/tests/built-bin.e2e.ts`), and assert a genuinely-missing config exits non-zero.
|
||||
- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). The same applies to any non-index runtime entry the built package resolves at run time (the worker-thread runtime's sibling `lib/worker.js`). Keep the built-artifact smokes green (`packages/ui/*/tests/built-bin.e2e.ts`, `packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts`), and assert a genuinely-missing config exits non-zero.
|
||||
- An e2e that spawns an example from a temp cwd sets `TSX_TSCONFIG_PATH` to the repo-root tsconfig, or it silently falls back to stale built `lib/` ([examples/AGENTS.md](../examples/AGENTS.md)).
|
||||
|
||||
## When a snapshot test is required
|
||||
|
||||
@@ -6,7 +6,7 @@ The DeepSeek Harness SDK agent demo exposed as an **Agent Client Protocol (ACP)*
|
||||
pnpm run demo:acp # needs DEEPSEEK_API_KEY (repo-root .env or env)
|
||||
```
|
||||
|
||||
This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand), the swappable DeepSeek, bash, and filesystem backends, and the model-facing `read`/`write`/`edit`/`subagent`/`subagent_fork`/`todo_write` tool entries. The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC.
|
||||
This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand), the swappable DeepSeek, bash, and filesystem backends, the model-facing `read`/`write`/`edit`/`subagent`/`subagent_fork`/`todo_write` tool entries, and the advisory `repeat-tool-guard` loop-hygiene plugin. The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC.
|
||||
|
||||
## stdout is the protocol
|
||||
|
||||
|
||||
@@ -33,6 +33,8 @@ flowchart LR
|
||||
cfg --> plugin_acp_tool_subagent_fork
|
||||
plugin_acp_tool_todo["tool-todo<br/>@deepseek-ai/dsh-tool-todo"]
|
||||
cfg --> plugin_acp_tool_todo
|
||||
plugin_acp_repeat_tool_guard["repeat-tool-guard<br/>@deepseek-ai/dsh-repeat-tool-guard"]
|
||||
cfg --> plugin_acp_repeat_tool_guard
|
||||
plugin_acp_fs_local["fs-local<br/>@deepseek-ai/dsh-fs-local"]
|
||||
cfg --> plugin_acp_fs_local
|
||||
plugin_acp_fs_policy["fs-policy<br/>@deepseek-ai/dsh-fs-policy"]
|
||||
@@ -56,6 +58,7 @@ flowchart LR
|
||||
| `tool-subagent` | `@deepseek-ai/dsh-tool-subagent` |
|
||||
| `tool-subagent-fork` | `@deepseek-ai/dsh-tool-subagent` |
|
||||
| `tool-todo` | `@deepseek-ai/dsh-tool-todo` |
|
||||
| `repeat-tool-guard` | `@deepseek-ai/dsh-repeat-tool-guard` |
|
||||
| `fs-local` | `@deepseek-ai/dsh-fs-local` |
|
||||
| `fs-policy` | `@deepseek-ai/dsh-fs-policy` |
|
||||
| `tool-fs` | `@deepseek-ai/dsh-tool-fs` |
|
||||
|
||||
@@ -86,6 +86,14 @@
|
||||
- id: tool-todo
|
||||
name: '@deepseek-ai/dsh-tool-todo'
|
||||
|
||||
# The repeat-tool-call guard: advisory reminders (injected context, never a
|
||||
# block) when the model re-issues the same tool call with identical arguments;
|
||||
# defaults [3, 5, 8]. Loaded here so the snapshot tier exercises the reminder
|
||||
# transcript (the repeat-tool-guard scenario) — no other scenario repeats a
|
||||
# call three times, so it is inert everywhere else.
|
||||
- id: repeat-tool-guard
|
||||
name: '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
|
||||
# Filesystem capability stack: local provider, read-before-write/edit policy
|
||||
# gate, then the model-facing read/write/edit tools. Relative filesystem paths
|
||||
# resolve from the server launch cwd; the documented Zed setup launches this
|
||||
|
||||
@@ -39,8 +39,13 @@ const SCENARIOS: Scenario[] = [
|
||||
{ name: 'fs-read-window', hasModelTurn: true, recorded: true },
|
||||
{ name: 'fs-policy-reject', hasModelTurn: true, recorded: true },
|
||||
{ name: 'multi-turn', hasModelTurn: true, recorded: true },
|
||||
{ name: 'error-finish', hasModelTurn: true, recorded: false },
|
||||
{ name: 'cancel', hasModelTurn: true, recorded: false },
|
||||
{ name: 'error-finish', hasModelTurn: true, recorded: false, overridden: true },
|
||||
// Keyless, authored (like error-finish/cancel): deterministically forcing a
|
||||
// LIVE model to repeat one call three times is not a stable recording, so
|
||||
// the fixture scripts five identical todo_write calls and pins BOTH reminder
|
||||
// tiers (gentle at 3, detailed at 5) as context/message in transcript and log.
|
||||
{ name: 'repeat-tool-guard', hasModelTurn: true, recorded: false },
|
||||
{ name: 'cancel', hasModelTurn: true, recorded: false, overridden: true },
|
||||
{ name: 'subagent-spawn', hasModelTurn: true, recorded: true, childSessions: 1 },
|
||||
{ name: 'subagent-multi', hasModelTurn: true, recorded: true, childSessions: 2 },
|
||||
{ name: 'subagent-fork', hasModelTurn: true, recorded: true, childSessions: 1 },
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"steps": [
|
||||
{ "op": "initialize" },
|
||||
{ "op": "newSession" },
|
||||
{ "op": "prompt", "text": "Write the todo list 'watch the kettle boil' five times in a row without changing it, then reply DONE." }
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"}
|
||||
{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}}
|
||||
{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"Write the todo list 'watch the kettle boil' five times in a row without changing it, then reply DONE."}],"source":{"kind":"user"}},"surfaceOp":"append"}
|
||||
{"type":"step/start","seq":2,"time":0,"data":{"turn":1,"step":1}}
|
||||
{"type":"request/header","seq":3,"time":0,"data":{"header":{"config":{"model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
|
||||
{"type":"assistant/chunk","seq":4,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}}
|
||||
{"type":"assistant/chunk","seq":5,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"call_1","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}
|
||||
{"type":"assistant/chunk","seq":6,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_1","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}}
|
||||
{"type":"assistant/chunk","seq":7,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}}
|
||||
{"type":"assistant/chunk","seq":8,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
|
||||
{"type":"assistant/message","seq":9,"time":0,"data":{"turn":1,"step":1,"content":[{"type":"tool-call","id":"call_1","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[4,5,6,7,8],"surfaceOp":"append"}
|
||||
{"type":"tool/call","seq":10,"time":0,"data":{"turn":1,"step":1,"callId":"call_1","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}
|
||||
{"type":"todo/write","seq":11,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}}
|
||||
{"type":"tool/result","seq":12,"time":0,"data":{"turn":1,"step":1,"callId":"call_1","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[10],"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":13,"time":0,"data":{"turn":1,"step":1}}
|
||||
{"type":"step/start","seq":14,"time":0,"data":{"turn":1,"step":2}}
|
||||
{"type":"assistant/chunk","seq":15,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}}
|
||||
{"type":"assistant/chunk","seq":16,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":0,"id":"call_2","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}
|
||||
{"type":"assistant/chunk","seq":17,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_2","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}}
|
||||
{"type":"assistant/chunk","seq":18,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}}
|
||||
{"type":"assistant/chunk","seq":19,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
|
||||
{"type":"assistant/message","seq":20,"time":0,"data":{"turn":1,"step":2,"content":[{"type":"tool-call","id":"call_2","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[15,16,17,18,19],"surfaceOp":"append"}
|
||||
{"type":"tool/call","seq":21,"time":0,"data":{"turn":1,"step":2,"callId":"call_2","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}
|
||||
{"type":"todo/write","seq":22,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}}
|
||||
{"type":"tool/result","seq":23,"time":0,"data":{"turn":1,"step":2,"callId":"call_2","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[21],"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":24,"time":0,"data":{"turn":1,"step":2}}
|
||||
{"type":"step/start","seq":25,"time":0,"data":{"turn":1,"step":3}}
|
||||
{"type":"assistant/chunk","seq":26,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}}
|
||||
{"type":"assistant/chunk","seq":27,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"tool-call-delta","index":0,"id":"call_3","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}
|
||||
{"type":"assistant/chunk","seq":28,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_3","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}}
|
||||
{"type":"assistant/chunk","seq":29,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}}
|
||||
{"type":"assistant/chunk","seq":30,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
|
||||
{"type":"assistant/message","seq":31,"time":0,"data":{"turn":1,"step":3,"content":[{"type":"tool-call","id":"call_3","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[26,27,28,29,30],"surfaceOp":"append"}
|
||||
{"type":"tool/call","seq":32,"time":0,"data":{"turn":1,"step":3,"callId":"call_3","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}
|
||||
{"type":"todo/write","seq":33,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}}
|
||||
{"type":"tool/result","seq":34,"time":0,"data":{"turn":1,"step":3,"callId":"call_3","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[32],"surfaceOp":"append"}
|
||||
{"type":"context/message","seq":35,"time":0,"data":{"content":[{"type":"text","text":"You are repeating the exact same tool call with identical arguments. Carefully analyze the previous result before calling again: if the task is not complete, try a different approach or different arguments instead of repeating the call."}],"source":{"kind":"plugin","plugin":"repeat-tool-guard"}},"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":36,"time":0,"data":{"turn":1,"step":3}}
|
||||
{"type":"step/start","seq":37,"time":0,"data":{"turn":1,"step":4}}
|
||||
{"type":"assistant/chunk","seq":38,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}}
|
||||
{"type":"assistant/chunk","seq":39,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"tool-call-delta","index":0,"id":"call_4","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}
|
||||
{"type":"assistant/chunk","seq":40,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_4","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}}
|
||||
{"type":"assistant/chunk","seq":41,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}}
|
||||
{"type":"assistant/chunk","seq":42,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
|
||||
{"type":"assistant/message","seq":43,"time":0,"data":{"turn":1,"step":4,"content":[{"type":"tool-call","id":"call_4","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[38,39,40,41,42],"surfaceOp":"append"}
|
||||
{"type":"tool/call","seq":44,"time":0,"data":{"turn":1,"step":4,"callId":"call_4","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}
|
||||
{"type":"todo/write","seq":45,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}}
|
||||
{"type":"tool/result","seq":46,"time":0,"data":{"turn":1,"step":4,"callId":"call_4","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[44],"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":47,"time":0,"data":{"turn":1,"step":4}}
|
||||
{"type":"step/start","seq":48,"time":0,"data":{"turn":1,"step":5}}
|
||||
{"type":"assistant/chunk","seq":49,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}}
|
||||
{"type":"assistant/chunk","seq":50,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"tool-call-delta","index":0,"id":"call_5","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}
|
||||
{"type":"assistant/chunk","seq":51,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_5","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}}
|
||||
{"type":"assistant/chunk","seq":52,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}}
|
||||
{"type":"assistant/chunk","seq":53,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
|
||||
{"type":"assistant/message","seq":54,"time":0,"data":{"turn":1,"step":5,"content":[{"type":"tool-call","id":"call_5","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[49,50,51,52,53],"surfaceOp":"append"}
|
||||
{"type":"tool/call","seq":55,"time":0,"data":{"turn":1,"step":5,"callId":"call_5","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}
|
||||
{"type":"todo/write","seq":56,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}}
|
||||
{"type":"tool/result","seq":57,"time":0,"data":{"turn":1,"step":5,"callId":"call_5","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[55],"surfaceOp":"append"}
|
||||
{"type":"context/message","seq":58,"time":0,"data":{"content":[{"type":"text","text":"Repeated tool call detected:\n- tool: todo_write\n- consecutive_calls: 5\n- arguments: {\"todos\":[{\"content\":\"watch the kettle boil\",\"status\":\"in_progress\"}]}\nThe repeated calls are not making progress. Do not call this tool with these exact arguments again. Inspect the latest result and choose a different action, different arguments, or finish the task if enough evidence has been gathered."}],"source":{"kind":"plugin","plugin":"repeat-tool-guard"}},"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":59,"time":0,"data":{"turn":1,"step":5}}
|
||||
{"type":"step/start","seq":60,"time":0,"data":{"turn":1,"step":6}}
|
||||
{"type":"assistant/chunk","seq":61,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}
|
||||
{"type":"assistant/chunk","seq":62,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"text-delta","index":0,"text":"DONE."}}}
|
||||
{"type":"assistant/chunk","seq":63,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE."}}}}
|
||||
{"type":"assistant/chunk","seq":64,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":3}}}}
|
||||
{"type":"assistant/chunk","seq":65,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
|
||||
{"type":"assistant/message","seq":66,"time":0,"data":{"turn":1,"step":6,"content":[{"type":"text","text":"DONE."}],"usage":{"inputTokens":10,"outputTokens":3}},"sourceEventSeqs":[61,62,63,64,65],"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":67,"time":0,"data":{"turn":1,"step":6}}
|
||||
{"type":"turn/end","seq":68,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}}
|
||||
@@ -0,0 +1,19 @@
|
||||
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}}
|
||||
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_1","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_1","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_2","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_2","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_3","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_3","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_4","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_4","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_5","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_5","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"DONE."}}}}
|
||||
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}
|
||||
@@ -35,6 +35,10 @@
|
||||
"entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
},
|
||||
"packages/code-runtime/code-runtime-worker": {
|
||||
"entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
},
|
||||
"packages/llm/llm-deepseek": {
|
||||
"entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
|
||||
+4
-3
@@ -11,13 +11,14 @@ Packages are grouped by modular role at `packages/<group>/<pkg>/`. The group dir
|
||||
| [`core/`](core/README.md) | Product API spine: session, system-prompt, tools, agent, and the concrete loop | Product — stable surface |
|
||||
| [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface |
|
||||
| [`bash/`](bash/README.md) | Bash capability family: the executor seam, a local impl, and the model-facing tool | Product — stable surface |
|
||||
| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the abstract runtime seam for model-written programs | Product — stable surface |
|
||||
| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the abstract runtime seam for model-written programs + a worker-thread backend | Product — stable surface |
|
||||
| [`fs/`](fs/README.md) | Filesystem capability family: the abstract seam, a local impl, and the model-facing file tools | Product — stable surface |
|
||||
| [`compact/`](compact/README.md) | Compaction capability family: the abstract seam + a basic backend (tool deferred) | Product — stable surface |
|
||||
| [`subagent/`](subagent/README.md) | Subagent capability family: the provider-registry seam and the model-facing delegation tool | Product — stable surface |
|
||||
| [`web/`](web/README.md) | Web capability family: the abstract seam, search/fetch provider impls, and the model-facing web tools | Product — stable surface |
|
||||
| [`timeout/`](timeout/README.md) | Tool-call timeout policy: the `tools/execute` deadline enforcer | Product — stable surface |
|
||||
| [`todo/`](todo/README.md) | Todo/planning family: the model-facing `todo_write` tool (whole-list task tracking on the session log) | Product — stable surface |
|
||||
| [`todo/`](todo/README.md) | Todo/planning family: the model-facing `todo_write` tool | Product — stable surface |
|
||||
| [`guard/`](guard/README.md) | Loop-hygiene guards: advisory repeat-call reminders | Product — stable surface |
|
||||
| [`hooks/`](hooks/README.md) | Hook bridges + the shared Claude Code / Codex wire-protocol library | Product — stable surface |
|
||||
| [`session-persistence/`](session-persistence/README.md) | Persistence capability family: the seam + JSONL/SQLite backends | Product — stable surface |
|
||||
| [`ui/`](ui/README.md) | Editor/client integration surfaces (the ACP bridge) + the app packages | Product — stable surface |
|
||||
@@ -30,6 +31,6 @@ The split is the point: a package's group says whether it is part of the product
|
||||
|
||||
The inter-package dependency graph is generated: [docs/module-graph.md](../docs/module-graph.md) (`pnpm run gen-module-graph`, freshness-gated in CI).
|
||||
|
||||
The rule it must obey: **extension plugins depend on interfaces, never on the concrete loop.** `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. The sanctioned exception is a **composition/bundle** package like `dsh-agent-core`, whose whole job is to assemble the concrete spine: it depends on `dsh-agent-loop` (and the other concrete spine plugins) on purpose. The rule constrains plugins that EXTEND the system, not the bundle that COMPOSES it — swapping the loop means shipping a different bundle, not rewiring every extension. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)).
|
||||
The rule it must obey: **extension plugins depend on interfaces, never on the concrete loop.** `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. The sanctioned exception is a **composition/bundle** package like `dsh-agent-core`, whose whole job is to assemble the concrete spine: it depends on `dsh-agent-loop` (and the other concrete spine plugins) on purpose. The rule constrains plugins that EXTEND the system, not the bundle that COMPOSES it. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)).
|
||||
|
||||
Each package has its own `README.md` with purpose, service API, events, extension points, and deliberate non-goals (TODOs).
|
||||
@@ -90,8 +90,8 @@ describe('LocalBashExecutor.run', () => {
|
||||
|
||||
it('kill escalation uses the configured graceMs (a TERM-trapping task dies by SIGKILL)', async () => {
|
||||
const { bash } = await setup() // setup pins graceMs: 200 via config
|
||||
const task = bash.start(bash.resolve({ command: 'trap \'\' TERM; sleep 60' }))
|
||||
await new Promise(resolve => setTimeout(resolve, 100))
|
||||
const task = bash.start(bash.resolve({ command: 'trap \'\' TERM; echo ready; while :; do sleep 60 & wait $!; done' }))
|
||||
await readUntil(bash, task.id, 'ready\n')
|
||||
bash.kill(task.id)
|
||||
await task.done
|
||||
expect(task.signal).toBe('SIGKILL')
|
||||
|
||||
@@ -55,6 +55,20 @@ async function waitForStdout(running: RunningBash, expected: string, timeoutMs =
|
||||
throw new Error(`stdout did not include ${JSON.stringify(expected)} after ${timeoutMs}ms`)
|
||||
}
|
||||
|
||||
async function waitForPidFile(path: string, timeoutMs = 5_000): Promise<number> {
|
||||
const deadline = Date.now() + timeoutMs
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const pid = Number(readFileSync(path, 'utf8').trim())
|
||||
if (Number.isSafeInteger(pid) && pid > 0) return pid
|
||||
} catch {
|
||||
// The child shell has not written the pid file yet.
|
||||
}
|
||||
await new Promise(resolve => setTimeout(resolve, 20))
|
||||
}
|
||||
throw new Error(`pid file ${path} was not written after ${timeoutMs}ms`)
|
||||
}
|
||||
|
||||
describe('runBash', () => {
|
||||
it('captures stdout on success', async () => {
|
||||
const result = await runBash(spec('echo hello')).done
|
||||
@@ -109,7 +123,7 @@ describe('runBash', () => {
|
||||
})
|
||||
|
||||
it('escalates to SIGKILL when SIGTERM is trapped', async () => {
|
||||
const running = runBash(spec('trap \'\' TERM; echo ready; sleep 60', { graceMs: 200 }))
|
||||
const running = runBash(spec('trap \'\' TERM; echo ready; while :; do sleep 60 & wait $!; done', { graceMs: 200 }))
|
||||
await waitForStdout(running, 'ready\n')
|
||||
running.kill()
|
||||
const result = await running.done
|
||||
@@ -121,8 +135,7 @@ describe('runBash', () => {
|
||||
// group must take the sleep down with bash.
|
||||
const pidFile = join(spillDir, `grandchild-${Date.now()}.pid`)
|
||||
const running = runBash(spec(`sleep 60 & echo $! > ${pidFile}; wait`))
|
||||
await new Promise(resolve => setTimeout(resolve, 300))
|
||||
const grandchild = Number(readFileSync(pidFile, 'utf8').trim())
|
||||
const grandchild = await waitForPidFile(pidFile)
|
||||
expect(grandchild).toBeGreaterThan(0)
|
||||
|
||||
running.kill()
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
# code-runtime/ — code-execution capability family
|
||||
|
||||
The code-execution capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's Code Mode, and the first implementation (a Node worker-thread backend) is specified alongside it in the [Code Mode RFC](../../docs/rfc/proposed/feature/2026-06-15-code-mode.md). **Product** packages.
|
||||
The code-execution capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's Code Mode, specified alongside the seam in the [Code Mode RFC](../../docs/rfc/proposed/feature/2026-06-15-code-mode.md). **Product** packages.
|
||||
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
| `code-runtime/` | Abstract code-execution seam (interface + vocabulary) | `ctx.codeRuntime` |
|
||||
| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend: fresh worker per run, TypeScript via host-side type-strip (annotations advisory, never type-checked), port-bridged bindings, budget/heap containment | registers `ctx.codeRuntime` |
|
||||
|
||||
The interface lives at `code-runtime/code-runtime/`. Backends differ by execution substrate (worker thread, process, container) and by source language — both readonly descriptors on the service — and register `ctx.codeRuntime` without touching the interface or its consumer; that split is what makes a hardened backend a drop-in later.
|
||||
The interface lives at `code-runtime/code-runtime/`; the shipped backend at `code-runtime/code-runtime-worker/`. Backends differ by execution substrate (worker thread, process, container) and by source language — both readonly descriptors on the service — and register `ctx.codeRuntime` without touching the interface or its consumer; that split is what makes a hardened backend a drop-in later.
|
||||
@@ -0,0 +1,32 @@
|
||||
# @deepseek-ai/dsh-code-runtime-worker
|
||||
|
||||
Worker-thread implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam: `WorkerCodeRuntime` runs each program in ONE fresh Node `worker_threads.Worker` — TypeScript in, type-stripped host-side, bindings bridged over the message port, `{ value, logs, error? }` out. **Containment, not a security boundary**: trust posture is bash-equivalent by design (the [Code Mode RFC](../../../docs/rfc/proposed/feature/2026-06-15-code-mode.md) § Trust posture), with containment bash does not have — separate isolate, empty environment, heap cap, hard termination.
|
||||
|
||||
## Config
|
||||
|
||||
```yaml
|
||||
- id: code-runtime
|
||||
name: '@deepseek-ai/dsh-code-runtime-worker'
|
||||
config:
|
||||
computeMs: 60000 # busy-time budget (measured event-loop active time)
|
||||
maxWallMs: 600000 # wall-clock ceiling; never pauses for anything
|
||||
maxLogBytes: 65536 # shared byte budget for captured log text
|
||||
maxValueBytes: 32768 # rendered-completion-value cap
|
||||
maxOldGenerationSizeMb: 512 # worker heap cap (resourceLimits)
|
||||
```
|
||||
|
||||
Every field is validated (positive numbers) and defaulted; there are no other tunables.
|
||||
|
||||
## Design
|
||||
|
||||
- **One fresh worker per run, no pooling** — a program's world dies with its worker: no cross-run state to log, state bleed unrepresentable, runs reconstructable from the session log alone.
|
||||
- **Type-strip host-side, in execution context** — the program is wrapped in an async-function shell, stripped with `node:module`'s `stripTypeScriptTypes` (erasable syntax only — `enum`/namespaces are rejected as a program `exception` and no worker spawns), and sliced back out byte-positioned; it then executes as the body of an `AsyncFunction`, so top-level `await`/`return` work.
|
||||
- **The port assumes a hostile peer** — model code can reach `parentPort` and forge traffic, so every inbound message is shape-validated and REBUILT before anything reads it (`null`, primitives, junk types, and malformed payloads drop without a throw; forged extra fields never ride along), the host answers each call id at most once, resolves binding names as OWN properties only (a forged `constructor` cannot walk a prototype chain), drops post-settlement replies, and converts a non-cloneable binding resolution into an error reply. Forged `log`/`done` messages cannot bypass the caps: one host-side ledger bounds everything that lands in `logs`, and the completion value is re-capped host-side. Worker-side namespaces are null-prototype with `defineProperty`, so `__proto__`-shaped binding names are ordinary keys.
|
||||
- **Two independent budgets, because the peer is hostile** — `computeMs` meters the worker's MEASURED busy time (`worker.performance.eventLoopUtilization()` polling): a hot loop cannot hide behind a pending decoy dispatch, and a program awaiting a slow tool accrues nothing. `maxWallMs` backstops what busy time cannot see (awaiting a promise nobody resolves). Both funnel into `worker.terminate()`, which ends hot synchronous loops too; heap overflow surfaces as the worker's OOM exit (`kind: 'worker-exit'`).
|
||||
- **Logs stream eagerly** — console/stdout/stderr entries cross the port as they happen, so a timed-out or killed program still shows what it printed. ONE shared `maxLogBytes` ledger bounds everything: streamed entries, forged port traffic, and pipe bytes that bypass the patched streams (appended after), with the overflow marked in-band once.
|
||||
- **Empty environment** — the worker gets `env: {}` and `execArgv: []`: no ambient credentials (stronger than the scrubbed-env rule for spawned commands) and no inherited loader flags.
|
||||
- **Dispose to quiescence** — teardown fails in-flight runs as `abort` and AWAITS each worker's exit before resolving.
|
||||
|
||||
## The worker entry, unbuilt and built
|
||||
|
||||
`worker.ts` is deliberately erasable-only TypeScript with type-only cross-package imports: unbuilt (vitest/tsx), the host spawns `src/worker.ts` directly and Node's native type stripping loads it; built, the entry ships as the sibling bundle `lib/worker.js` (its own tsdown entry). The built path is pinned by `tests/built-lib.e2e.ts`, the real-load-path guard from [docs/testing.md](../../../docs/testing.md).
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-code-runtime-worker",
|
||||
"description": "Worker-thread implementation of the DeepSeek Harness code-execution seam",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/worker.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-code-runtime": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-code-runtime": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,301 @@
|
||||
/**
|
||||
* Worker-side execution logic, written as plain functions over an injected
|
||||
* port so the unit suite can run every line IN-PROCESS against a fake port
|
||||
* (a real worker thread is a separate V8 isolate the coverage provider
|
||||
* cannot observe). The real worker entry (`worker.ts`) is a thin
|
||||
* self-executing glue file over {@link runWorkerMain}, excluded from
|
||||
* coverage the same way `bin.ts` entrypoints are, and exercised end-to-end
|
||||
* by the integration tests that spawn real workers.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-code-runtime-worker/src/bootstrap
|
||||
*/
|
||||
|
||||
import { inspect } from 'node:util'
|
||||
import { serialize } from 'node:v8'
|
||||
import type { CodeLogEntry } from '@deepseek-ai/dsh-code-runtime'
|
||||
import { logTruncationMarker } from './protocol.ts'
|
||||
import type { DoneMessage, ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts'
|
||||
|
||||
/** The port surface the bootstrap needs — satisfied by `parentPort` and by the tests' fake. */
|
||||
export interface BootstrapPort {
|
||||
postMessage(message: WorkerToHost): void
|
||||
on(event: 'message', listener: (message: ReplyMessage) => void): void
|
||||
}
|
||||
|
||||
/**
|
||||
* A writable stream's `write` slot, as the bootstrap patches it (see
|
||||
* {@link captureStreamWrites}). Method-typed so the real
|
||||
* `process.stdout`/`process.stderr` (narrower chunk parameters) remain
|
||||
* assignable.
|
||||
*/
|
||||
export interface PatchableStream {
|
||||
write(chunk: unknown, ...rest: unknown[]): boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Ordered log capture under one shared byte budget, delivered to a sink as
|
||||
* each entry lands (the real sink streams entries over the port eagerly, so
|
||||
* captured output survives a mid-run termination). Once the budget is
|
||||
* exhausted it emits exactly one in-band marker entry (on the `stderr`
|
||||
* diagnostics channel) and silently drops everything after — the cap is a
|
||||
* blast-radius bound, so "how much was lost" intentionally stays unmeasured.
|
||||
*/
|
||||
export class LogBuffer {
|
||||
private remaining: number
|
||||
private truncated = false
|
||||
// Explicit fields, not constructor parameter properties: this module loads
|
||||
// under Node's native strip-only mode, which rejects non-erasable syntax —
|
||||
// and parameter properties are non-erasable.
|
||||
private readonly maxBytes: number
|
||||
private readonly sink: (entry: CodeLogEntry) => void
|
||||
|
||||
constructor(maxBytes: number, sink: (entry: CodeLogEntry) => void) {
|
||||
this.maxBytes = maxBytes
|
||||
this.sink = sink
|
||||
this.remaining = maxBytes
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit one entry to the sink, charging its text against the budget (drops + marks once exhausted).
|
||||
* @param entry - the log entry to deliver.
|
||||
*/
|
||||
push(entry: CodeLogEntry): void {
|
||||
if (this.truncated) return
|
||||
const cost = Buffer.byteLength(entry.text, 'utf8')
|
||||
if (cost > this.remaining) {
|
||||
this.truncated = true
|
||||
this.sink({ source: 'stderr', text: logTruncationMarker(this.maxBytes) })
|
||||
return
|
||||
}
|
||||
this.remaining -= cost
|
||||
this.sink(entry)
|
||||
}
|
||||
}
|
||||
|
||||
/** The five console methods the shim captures, in the seam's level vocabulary. */
|
||||
const CONSOLE_LEVELS = ['log', 'info', 'warn', 'error', 'debug'] as const
|
||||
|
||||
/**
|
||||
* A `console` replacement whose five leveled methods render their arguments
|
||||
* `util.inspect`-style (matching real console formatting closely enough for
|
||||
* a model to recognize its own output) into the buffer. Only these five
|
||||
* exist — the program gets a deliberately small console, not Node's full
|
||||
* surface.
|
||||
* @param logs - the buffer every rendered line is pushed into.
|
||||
* @returns the five-method console object handed to the program.
|
||||
*/
|
||||
export function makeConsoleShim(logs: LogBuffer): Record<(typeof CONSOLE_LEVELS)[number], (...args: unknown[]) => void> {
|
||||
const render = (args: unknown[]): string =>
|
||||
args.map(arg => typeof arg === 'string' ? arg : inspect(arg, INSPECT_OPTIONS)).join(' ')
|
||||
const shim = Object.create(null) as Record<(typeof CONSOLE_LEVELS)[number], (...args: unknown[]) => void>
|
||||
for (const level of CONSOLE_LEVELS) {
|
||||
shim[level] = (...args: unknown[]) => { logs.push({ source: 'console', level, text: render(args) }) }
|
||||
}
|
||||
return shim
|
||||
}
|
||||
|
||||
/**
|
||||
* Redirect a stream's `write` into the log buffer (the program-visible
|
||||
* `process.stdout`/`process.stderr` in the real worker), so raw writes land
|
||||
* in emission order alongside console output instead of racing down a pipe.
|
||||
* The shim keeps Node's `write(chunk[, encoding][, callback])` contract: the
|
||||
* callback fires asynchronously once the chunk is admitted (a program
|
||||
* awaiting flush completion must complete, not sit until the wall timeout),
|
||||
* even for writes the exhausted budget drops.
|
||||
* @param logs - the buffer captured writes are pushed into.
|
||||
* @param stream - the stream whose `write` slot is patched.
|
||||
* @param source - the log source the captured writes are attributed to.
|
||||
* @returns the restore function (the in-process tests un-patch; the real
|
||||
* worker never needs to).
|
||||
*/
|
||||
export function captureStreamWrites(logs: LogBuffer, stream: PatchableStream, source: 'stdout' | 'stderr'): () => void {
|
||||
// The slot's VALUE is stored for restore and reassigned — never invoked
|
||||
// detached, so the unbound-method concern does not apply.
|
||||
// eslint-disable-next-line @typescript-eslint/unbound-method
|
||||
const original = stream.write
|
||||
stream.write = (chunk: unknown, ...rest: unknown[]): boolean => {
|
||||
logs.push({ source, text: typeof chunk === 'string' ? chunk : String(chunk) })
|
||||
// Node's optional-encoding shape: the callback is whichever of the next
|
||||
// two positions holds a function (a non-function there is the encoding).
|
||||
const callback = [rest[0], rest[1]].find(
|
||||
(arg): arg is (error?: Error | null) => void => typeof arg === 'function',
|
||||
)
|
||||
if (callback) queueMicrotask(() => { callback(null) })
|
||||
return true
|
||||
}
|
||||
return () => { stream.write = original }
|
||||
}
|
||||
|
||||
/** Bounded inspect options: deep enough to be useful, bounded so a pathological value cannot explode the rendering. */
|
||||
const INSPECT_OPTIONS = { depth: 4, maxArrayLength: 100, maxStringLength: 10_000 } as const
|
||||
|
||||
/**
|
||||
* The longest prefix of `text` whose UTF-8 encoding fits `maxBytes`, cut at
|
||||
* a code-point boundary (never mid-surrogate-pair). The byte caps are BYTE
|
||||
* caps — `String.prototype.slice` counts UTF-16 code units, up to 3× smaller
|
||||
* than what a multibyte string actually costs across the boundary.
|
||||
* @param text - the string to bound.
|
||||
* @param maxBytes - the UTF-8 byte budget the prefix must fit.
|
||||
* @returns the prefix (all of `text` when it already fits).
|
||||
*/
|
||||
export function truncateUtf8Bytes(text: string, maxBytes: number): string {
|
||||
if (Buffer.byteLength(text, 'utf8') <= maxBytes) return text
|
||||
let bytes = 0
|
||||
let end = 0
|
||||
for (const char of text) {
|
||||
const cost = Buffer.byteLength(char, 'utf8')
|
||||
if (bytes + cost > maxBytes) break
|
||||
bytes += cost
|
||||
end += char.length
|
||||
}
|
||||
return text.slice(0, end)
|
||||
}
|
||||
|
||||
/**
|
||||
* Prepare the program's completion value for the done message: a value whose
|
||||
* MEASURED cross-boundary size fits `maxValueBytes` crosses raw — exact
|
||||
* bytes for a string, the structured-clone wire size (`v8.serialize`) for
|
||||
* everything else, so a huge container whose BOUNDED inspect rendering
|
||||
* happens to be small cannot smuggle itself past the cap. Anything else
|
||||
* (non-cloneable, or oversized) is REPLACED by its bounded `util.inspect`
|
||||
* rendering, byte-truncated ({@link truncateUtf8Bytes}) with an in-band
|
||||
* marker — the seam contract's "a non-transferable value is replaced by a
|
||||
* string rendering", extended to oversized ones so a huge return cannot
|
||||
* flood the host.
|
||||
* @param value - the program's completion value.
|
||||
* @param maxValueBytes - the byte cap for the value.
|
||||
* @returns the done-message fragment: `{}` for `undefined`, else `{ value }`.
|
||||
*/
|
||||
export function prepareValue(value: unknown, maxValueBytes: number): { value?: unknown } {
|
||||
if (value === undefined) return {}
|
||||
if (typeof value === 'string') {
|
||||
if (Buffer.byteLength(value, 'utf8') <= maxValueBytes) return { value }
|
||||
} else {
|
||||
let size: number | undefined
|
||||
try {
|
||||
size = serialize(value).byteLength
|
||||
} catch {
|
||||
// Only the verdict matters: the value has parts the structured-clone
|
||||
// algorithm rejects (functions, classes, …) and must cross as its
|
||||
// rendering instead.
|
||||
size = undefined
|
||||
}
|
||||
if (size !== undefined && size <= maxValueBytes) return { value }
|
||||
}
|
||||
const rendered = typeof value === 'string' ? value : inspect(value, INSPECT_OPTIONS)
|
||||
const capped = Buffer.byteLength(rendered, 'utf8') > maxValueBytes
|
||||
? `${truncateUtf8Bytes(rendered, maxValueBytes)}… [truncated]`
|
||||
: rendered
|
||||
return { value: capped }
|
||||
}
|
||||
|
||||
/** One awaited binding call's settlement handles, keyed by call id in the pending map. */
|
||||
export interface PendingCall {
|
||||
resolve(value: unknown): void
|
||||
reject(error: Error): void
|
||||
}
|
||||
|
||||
/**
|
||||
* Route host replies into the pending-call map: each reply settles its call
|
||||
* at most once, and a reply for an unknown id (stray, or a duplicate answer
|
||||
* to an id already settled) is ignored. Shared wiring between
|
||||
* {@link runWorkerMain} and the tests that exercise {@link makeNamespaces}
|
||||
* standalone.
|
||||
* @param port - the port whose `message` events carry the replies.
|
||||
* @param pending - the id-keyed map of unsettled binding calls.
|
||||
*/
|
||||
export function wireReplies(port: BootstrapPort, pending: Map<number, PendingCall>): void {
|
||||
port.on('message', (message: ReplyMessage) => {
|
||||
const entry = pending.get(message.id)
|
||||
if (!entry) return
|
||||
pending.delete(message.id)
|
||||
if (message.ok) entry.resolve(message.value)
|
||||
else entry.reject(new Error(message.message))
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the binding namespace objects the program sees: one null-prototype
|
||||
* global per namespace, each declared name an own enumerable async function
|
||||
* that bridges over the port (`__proto__`/`constructor`/`toString` are
|
||||
* ordinary keys, never prototype collisions). A non-cloneable argument
|
||||
* rejects that one call with a descriptive error; the host's reply (`ok`
|
||||
* false) rejects it likewise, so a failed tool call surfaces in the program
|
||||
* as an ordinary promise rejection.
|
||||
* @param data - the boot payload's namespace declarations (globals + names).
|
||||
* @param port - the port binding calls are posted to.
|
||||
* @param pending - the id-keyed map each posted call parks its handles in.
|
||||
* @param nextId - the shared mutable id counter (worker-issued correlation ids).
|
||||
* @returns one namespace object per declaration, in declaration order.
|
||||
*/
|
||||
export function makeNamespaces(
|
||||
data: Pick<WorkerBootData, 'namespaces'>,
|
||||
port: BootstrapPort,
|
||||
pending: Map<number, PendingCall>,
|
||||
nextId: { value: number },
|
||||
): Record<string, unknown>[] {
|
||||
return data.namespaces.map(({ global, names }) => {
|
||||
const namespace = Object.create(null) as Record<string, unknown>
|
||||
for (const name of names) {
|
||||
Object.defineProperty(namespace, name, {
|
||||
enumerable: true,
|
||||
value: (args: unknown): Promise<unknown> => new Promise((resolve, reject) => {
|
||||
const id = nextId.value++
|
||||
pending.set(id, { resolve, reject })
|
||||
try {
|
||||
port.postMessage({ type: 'call', id, global, name, args })
|
||||
} catch (error: unknown) {
|
||||
pending.delete(id)
|
||||
reject(new Error(`binding arguments must be structured-cloneable: ${error instanceof Error ? error.message : String(error)}`))
|
||||
}
|
||||
}),
|
||||
})
|
||||
}
|
||||
return namespace
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one program to settlement and post the {@link DoneMessage}: wires the
|
||||
* reply handler, materializes the namespaces and console shim, compiles the
|
||||
* type-stripped body as an async function (top-level `await`/`return`
|
||||
* work), and reports a thrown program error as the done message's `error`
|
||||
* field. Exactly one done message is ever posted.
|
||||
* @param port - the message port to the host (the real `parentPort`, or the tests' fake).
|
||||
* @param data - the boot payload the host sent.
|
||||
* @param streams - the stream objects whose `write` is captured (the real
|
||||
* `process.stdout`/`process.stderr` in the worker; fakes in tests).
|
||||
* @returns resolves after the done message is posted (the tests await it;
|
||||
* the real entry lets the worker exit naturally).
|
||||
*/
|
||||
export async function runWorkerMain(
|
||||
port: BootstrapPort,
|
||||
data: WorkerBootData,
|
||||
streams: { stdout: PatchableStream; stderr: PatchableStream },
|
||||
): Promise<void> {
|
||||
const logs = new LogBuffer(data.maxLogBytes, (entry) => { port.postMessage({ type: 'log', entry }) })
|
||||
captureStreamWrites(logs, streams.stdout, 'stdout')
|
||||
captureStreamWrites(logs, streams.stderr, 'stderr')
|
||||
|
||||
const pending = new Map<number, PendingCall>()
|
||||
wireReplies(port, pending)
|
||||
|
||||
const nextId = { value: 1 }
|
||||
const namespaces = makeNamespaces(data, port, pending, nextId)
|
||||
const consoleShim = makeConsoleShim(logs)
|
||||
|
||||
let done: DoneMessage
|
||||
try {
|
||||
// The async function constructor, reached through an instance because
|
||||
// `AsyncFunction` is not a global. The program body is strict-mode.
|
||||
/* v8 ignore next -- the arrow exists only to reach the AsyncFunction constructor; it is never invoked. */
|
||||
const AsyncFunction = (async () => {}).constructor as new (...args: string[]) => (...fnArgs: unknown[]) => Promise<unknown>
|
||||
const fn = new AsyncFunction(...data.namespaces.map(namespace => namespace.global), 'console', `'use strict';\n${data.code}`)
|
||||
const value = await fn(...namespaces, consoleShim)
|
||||
done = { type: 'done', ...prepareValue(value, data.maxValueBytes) }
|
||||
} catch (error: unknown) {
|
||||
const message = error instanceof Error ? error.stack ?? error.message : String(error)
|
||||
done = { type: 'done', error: { message } }
|
||||
}
|
||||
port.postMessage(done)
|
||||
}
|
||||
@@ -0,0 +1,441 @@
|
||||
/**
|
||||
* Worker-thread implementation of the code-execution seam: one fresh Node
|
||||
* worker per run, executing the model's TypeScript after a host-side
|
||||
* type-strip, with bindings bridged over the message port. Containment, not
|
||||
* a security boundary (bash-equivalent trust — see the Code Mode RFC's
|
||||
* trust-posture section): the worker gets an EMPTY environment, a heap cap,
|
||||
* and two independent budgets — `computeMs` metered on the worker's
|
||||
* measured event-loop busy time (a hot loop cannot hide behind a pending
|
||||
* binding call) and a never-pausing `maxWallMs` ceiling — all funneling
|
||||
* into `worker.terminate()`, which ends hot synchronous loops too.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-code-runtime-worker
|
||||
*/
|
||||
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import { stripTypeScriptTypes } from 'node:module'
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
|
||||
import type { CodeBindingFunction, CodeLogEntry, CodeRunFailure, CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
|
||||
import { prepareValue, truncateUtf8Bytes } from './bootstrap.ts'
|
||||
import { logTruncationMarker } from './protocol.ts'
|
||||
import type { ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts'
|
||||
|
||||
export type { BootstrapPort, PatchableStream } from './bootstrap.ts'
|
||||
export type { CallMessage, DoneMessage, ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts'
|
||||
|
||||
/** Plugin config: every execution cap, changeable from `cordis.yml` (no hardcoded tunables). */
|
||||
export interface Config {
|
||||
/**
|
||||
* Busy-time budget in milliseconds: the run fails with kind `'timeout'`
|
||||
* once the worker's MEASURED event-loop active time
|
||||
* (`worker.performance.eventLoopUtilization()`) exceeds this. Metering
|
||||
* measured busy time — not wall time, not host-side pending-call
|
||||
* bookkeeping — is what makes the budget both fair (a program awaiting a
|
||||
* slow tool accrues nothing) and ungameable (a hot loop accrues whether
|
||||
* or not a decoy dispatch is in flight).
|
||||
*/
|
||||
computeMs?: number
|
||||
/**
|
||||
* Wall-clock ceiling in milliseconds; never pauses for anything. The
|
||||
* backstop for what busy-time cannot see (a program awaiting a promise
|
||||
* nobody will resolve).
|
||||
*/
|
||||
maxWallMs?: number
|
||||
/** Shared byte budget for captured log text (console + raw stream writes), truncation marked in-band. */
|
||||
maxLogBytes?: number
|
||||
/**
|
||||
* Byte cap for the completion value, measured by its real cross-boundary
|
||||
* size (string bytes, or structured-clone wire size); an oversized or
|
||||
* non-cloneable value crosses as a capped string rendering.
|
||||
*/
|
||||
maxValueBytes?: number
|
||||
/** The worker's max old-generation heap in MiB (`resourceLimits`); overflow kills the worker, surfacing as kind `'worker-exit'`. */
|
||||
maxOldGenerationSizeMb?: number
|
||||
}
|
||||
|
||||
/** {@link Config} after schemastery fills the defaults (every field present). */
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/**
|
||||
* How often the host samples the worker's event-loop utilization for the
|
||||
* `computeMs` budget. An internal cadence, not config: the only effect of
|
||||
* the interval is budget-expiry granularity (a run can overshoot by up to
|
||||
* one interval), and nothing a deployment could tune here improves that
|
||||
* without burning host CPU.
|
||||
*/
|
||||
const ELU_POLL_INTERVAL_MS = 25
|
||||
|
||||
/** ECMAScript reserved words that cannot be async-function parameter names — rejected as binding globals. */
|
||||
const RESERVED_WORDS = new Set([
|
||||
'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do',
|
||||
'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in',
|
||||
'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof',
|
||||
'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package',
|
||||
'private', 'protected', 'public', 'arguments', 'eval',
|
||||
])
|
||||
|
||||
/** Valid async-function parameter name (the binding global becomes one). */
|
||||
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/
|
||||
|
||||
/**
|
||||
* The shell a program is wrapped in for the type-strip, matching the
|
||||
* grammatical context it will execute in (an async function body, where
|
||||
* top-level `return` and `await` are legal — a bare module parse would
|
||||
* reject the `return`). Strip mode is position-preserving (removed syntax
|
||||
* becomes whitespace, nothing shifts), so the wrapper survives the strip
|
||||
* byte-identical and the body slices back out with the model's own
|
||||
* line/column positions intact.
|
||||
*/
|
||||
const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const
|
||||
|
||||
/** One in-flight run's host-side state, tracked for disposal. */
|
||||
interface LiveRun {
|
||||
worker: Worker
|
||||
settle(failure: CodeRunFailure): void
|
||||
finished: Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker entry module. Source runs unbuilt (`src/worker.ts`, loadable
|
||||
* directly on this repo's Node range via native type stripping — the file
|
||||
* is erasable-only with type-only relative imports); the built package
|
||||
* ships it as a sibling bundle (`lib/worker.js`, its own tsdown entry).
|
||||
* The URL *pathname*'s extension says which world this module is in —
|
||||
* pathname, because dev-time module runners (vitest) may suffix
|
||||
* `import.meta.url` with a query string; relative resolution drops it.
|
||||
*/
|
||||
/* v8 ignore next -- the './worker.js' arm is the built-lib world, unreachable unbuilt by construction; the built-lib e2e pins it. */
|
||||
const WORKER_URL = new URL(new URL(import.meta.url).pathname.endsWith('.ts') ? './worker.ts' : './worker.js', import.meta.url)
|
||||
|
||||
/** Render an unknown thrown value as a message, `Error` or not. */
|
||||
function messageOf(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
|
||||
/** The log sources / console levels the seam vocabulary admits, as runtime sets for inbound-message validation. */
|
||||
const LOG_SOURCES = new Set<string>(['console', 'stdout', 'stderr'])
|
||||
const LOG_LEVELS = new Set<string>(['log', 'info', 'warn', 'error', 'debug'])
|
||||
|
||||
/**
|
||||
* Runtime shape gate for inbound port traffic. The peer runs MODEL CODE and
|
||||
* can post anything — `null`, primitives, objects with poisoned fields — so
|
||||
* the compile-time `WorkerToHost` type means nothing here: everything is
|
||||
* re-validated and REBUILT field by field (a forged extra field never rides
|
||||
* along; a non-number call id can never be echoed into a reply). Junk returns
|
||||
* `undefined` and is dropped — a throw in the host's `message` listener would
|
||||
* crash the host process.
|
||||
*/
|
||||
function parseWorkerMessage(raw: unknown): WorkerToHost | undefined {
|
||||
if (typeof raw !== 'object' || raw === null) return undefined
|
||||
const m = raw as Record<string, unknown>
|
||||
switch (m.type) {
|
||||
case 'call': {
|
||||
if (typeof m.id !== 'number' || typeof m.global !== 'string' || typeof m.name !== 'string') return undefined
|
||||
return { type: 'call', id: m.id, global: m.global, name: m.name, args: m.args }
|
||||
}
|
||||
case 'log': {
|
||||
const entry = m.entry
|
||||
if (typeof entry !== 'object' || entry === null) return undefined
|
||||
const e = entry as Record<string, unknown>
|
||||
if (typeof e.text !== 'string') return undefined
|
||||
if (typeof e.source !== 'string' || !LOG_SOURCES.has(e.source)) return undefined
|
||||
if (e.level !== undefined && (typeof e.level !== 'string' || !LOG_LEVELS.has(e.level))) return undefined
|
||||
return {
|
||||
type: 'log',
|
||||
entry: {
|
||||
source: e.source as CodeLogEntry['source'],
|
||||
...e.level !== undefined ? { level: e.level as Exclude<CodeLogEntry['level'], undefined> } : {},
|
||||
text: e.text,
|
||||
},
|
||||
}
|
||||
}
|
||||
case 'done': {
|
||||
if (m.error === undefined) return { type: 'done', ...m.value !== undefined ? { value: m.value } : {} }
|
||||
const error = m.error
|
||||
if (typeof error !== 'object' || error === null) return undefined
|
||||
const message = (error as Record<string, unknown>).message
|
||||
if (typeof message !== 'string') return undefined
|
||||
return { type: 'done', ...m.value !== undefined ? { value: m.value } : {}, error: { message } }
|
||||
}
|
||||
default: return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Headroom the host's value re-cap grants over `maxValueBytes`: exactly the
|
||||
* truncation suffix {@link prepareValue} appends, so a value the WORKER
|
||||
* already capped (byte-exact prefix + this marker) passes through unchanged
|
||||
* instead of being marked twice.
|
||||
*/
|
||||
const VALUE_RENDER_SLACK = Buffer.byteLength('… [truncated]', 'utf8')
|
||||
|
||||
/**
|
||||
* The shipped {@link CodeRuntime} backend (`ctx.codeRuntime`). Registers as
|
||||
* the `codeRuntime` service; every cap comes from validated config. See the
|
||||
* module doc for the containment model and the class JSDoc on the seam for
|
||||
* the contract this implements (error-as-field, hostile-peer port,
|
||||
* no cross-run state, dispose to quiescence).
|
||||
*/
|
||||
export class WorkerCodeRuntime extends CodeRuntime {
|
||||
static Config: z<Config> = z.object({
|
||||
computeMs: z.number().default(60_000),
|
||||
maxWallMs: z.number().default(600_000),
|
||||
maxLogBytes: z.number().default(65_536),
|
||||
maxValueBytes: z.number().default(32_768),
|
||||
maxOldGenerationSizeMb: z.number().default(512),
|
||||
})
|
||||
|
||||
readonly language = 'typescript'
|
||||
readonly isolation = 'worker-thread'
|
||||
|
||||
private readonly config: ResolvedConfig
|
||||
private readonly live = new Set<LiveRun>()
|
||||
private disposed = false
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
// Schemastery filled the defaults; the cast records that. Positivity is a
|
||||
// semantic check the schema's plain number type does not carry.
|
||||
this.config = config as ResolvedConfig
|
||||
for (const [key, value] of Object.entries(this.config)) {
|
||||
if (!(Number.isFinite(value) && value > 0)) throw new Error(`dsh-code-runtime-worker: config.${key} must be a positive number, got ${String(value)}`)
|
||||
}
|
||||
ctx.effect(() => () => this.teardown(), 'worker code-runtime teardown')
|
||||
}
|
||||
|
||||
/**
|
||||
* Dispose to quiescence: mark the service unusable, fail every in-flight
|
||||
* run as aborted, and AWAIT each worker's exit so no worker outlives the
|
||||
* fiber.
|
||||
*/
|
||||
private async teardown(): Promise<void> {
|
||||
this.disposed = true
|
||||
const runs = [...this.live]
|
||||
for (const run of runs) run.settle({ kind: 'abort', message: 'runtime disposed' })
|
||||
await Promise.all(runs.map(run => run.finished))
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute one program in a fresh worker. Program outcomes — including a
|
||||
* type-strip syntax error, which never spawns a worker — resolve with
|
||||
* `result.error`; the method rejects only for seam misuse (a disposed
|
||||
* runtime, an invalid binding namespace).
|
||||
* @param request - the program, its bindings, and the abort signal.
|
||||
* @returns the run's outcome per the seam contract.
|
||||
*/
|
||||
async run(request: CodeRunRequest): Promise<CodeRunResult> {
|
||||
if (this.disposed) throw new Error('dsh-code-runtime-worker: run() after disposal')
|
||||
const bindings = this.validateBindings(request)
|
||||
if (request.signal?.aborted) {
|
||||
return { logs: [], error: { kind: 'abort', message: String(request.signal.reason) } }
|
||||
}
|
||||
|
||||
let code: string
|
||||
try {
|
||||
const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + request.program + STRIP_WRAP.suffix)
|
||||
code = stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length)
|
||||
} catch (error: unknown) {
|
||||
// A program that does not survive the type-strip (syntax error,
|
||||
// non-erasable syntax like `enum`) is a program failure, reported the
|
||||
// same way a thrown exception would be — and no worker ever spawns.
|
||||
return { logs: [], error: { kind: 'exception', message: messageOf(error) } }
|
||||
}
|
||||
|
||||
return await this.execute(request, code, bindings)
|
||||
}
|
||||
|
||||
/** Reject (seam misuse) malformed binding namespaces: non-identifier or reserved globals, duplicates, and the `console` collision. */
|
||||
private validateBindings(request: CodeRunRequest): Map<string, Record<string, CodeBindingFunction>> {
|
||||
const bindings = new Map<string, Record<string, CodeBindingFunction>>()
|
||||
for (const namespace of request.bindings) {
|
||||
if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) {
|
||||
throw new Error(`dsh-code-runtime-worker: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`)
|
||||
}
|
||||
if (namespace.global === 'console' || bindings.has(namespace.global)) {
|
||||
throw new Error(`dsh-code-runtime-worker: duplicate binding global ${JSON.stringify(namespace.global)}`)
|
||||
}
|
||||
bindings.set(namespace.global, namespace.functions)
|
||||
}
|
||||
return bindings
|
||||
}
|
||||
|
||||
/** Spawn the worker for one validated, type-stripped run and drive it to settlement. */
|
||||
private execute(
|
||||
request: CodeRunRequest,
|
||||
code: string,
|
||||
bindings: Map<string, Record<string, CodeBindingFunction>>,
|
||||
): Promise<CodeRunResult> {
|
||||
const bootData: WorkerBootData = {
|
||||
code,
|
||||
namespaces: [...bindings].map(([global, functions]) => ({ global, names: Object.keys(functions) })),
|
||||
maxLogBytes: this.config.maxLogBytes,
|
||||
maxValueBytes: this.config.maxValueBytes,
|
||||
}
|
||||
const worker = new Worker(WORKER_URL, {
|
||||
workerData: bootData,
|
||||
// Model code gets NO ambient environment — stronger than the scrubbed
|
||||
// env the defensive-patterns rule requires for spawned commands.
|
||||
env: {},
|
||||
// Hermetic flags too: without this the worker inherits the host
|
||||
// process's execArgv (a test runner's or tsx's loader hooks), which a
|
||||
// bare isolate with an empty environment cannot satisfy. The entry
|
||||
// needs nothing beyond native type stripping, on this repo's whole
|
||||
// Node range.
|
||||
execArgv: [],
|
||||
resourceLimits: { maxOldGenerationSizeMb: this.config.maxOldGenerationSizeMb },
|
||||
// Backstop capture: the bootstrap patches JS-level writes into its own
|
||||
// ordered buffer, so these pipes normally stay silent; anything that
|
||||
// still arrives (native-level writes) is appended after the done logs.
|
||||
stdout: true,
|
||||
stderr: true,
|
||||
})
|
||||
|
||||
return new Promise<CodeRunResult>((resolve) => {
|
||||
let settled = false
|
||||
const answered = new Set<number>()
|
||||
const logs: CodeLogEntry[] = []
|
||||
const strayLogs: CodeLogEntry[] = []
|
||||
|
||||
// ONE host-side ledger for everything that lands in `logs`/`strayLogs`,
|
||||
// whatever the path: honest port entries, FORGED port entries (model
|
||||
// code posting `log` messages directly, bypassing the worker-side
|
||||
// LogBuffer), and stray pipe bytes. On the first overflow it emits the
|
||||
// same in-band marker the worker's LogBuffer would and drops the rest,
|
||||
// so the documented cap is one shared `maxLogBytes` however it is hit.
|
||||
let logBudget = this.config.maxLogBytes
|
||||
let logsTruncated = false
|
||||
const admit = (entry: CodeLogEntry, sink: CodeLogEntry[]): void => {
|
||||
if (logsTruncated) return
|
||||
const cost = Buffer.byteLength(entry.text, 'utf8')
|
||||
if (cost > logBudget) {
|
||||
logsTruncated = true
|
||||
sink.push({ source: 'stderr', text: logTruncationMarker(this.config.maxLogBytes) })
|
||||
return
|
||||
}
|
||||
logBudget -= cost
|
||||
sink.push(entry)
|
||||
}
|
||||
|
||||
// No settled guard: `finish` snapshots the arrays when it resolves, so
|
||||
// a chunk flushing after settlement mutates only the discarded buffers,
|
||||
// and the ledger bounds that growth until the pipes close.
|
||||
const captureStray = (source: 'stdout' | 'stderr') => (chunk: Buffer) => {
|
||||
admit({ source, text: chunk.toString('utf8') }, strayLogs)
|
||||
}
|
||||
worker.stdout.on('data', captureStray('stdout'))
|
||||
worker.stderr.on('data', captureStray('stderr'))
|
||||
|
||||
// Settlement: exactly one outcome wins; every path funnels through
|
||||
// here, cleans up the timers/listeners, terminates the worker, and
|
||||
// resolves only after the worker actually exited (quiescence). Logs
|
||||
// streamed eagerly before the settlement are kept — a timed-out or
|
||||
// killed program still shows the model what it printed.
|
||||
let finishResolve!: () => void
|
||||
const finished = new Promise<void>((done) => { finishResolve = done })
|
||||
const finish = (result: Omit<CodeRunResult, 'logs'>): void => {
|
||||
if (settled) return
|
||||
settled = true
|
||||
clearInterval(eluTimer)
|
||||
clearTimeout(wallTimer)
|
||||
request.signal?.removeEventListener('abort', onAbort)
|
||||
this.live.delete(live)
|
||||
void worker.terminate().then(() => {
|
||||
finishResolve()
|
||||
resolve({ ...result, logs: [...logs, ...strayLogs] })
|
||||
})
|
||||
}
|
||||
|
||||
const onDone = (message: WorkerToHost): void => {
|
||||
if (message.type !== 'done') return
|
||||
// Re-cap the completion value HOST-side: the honest path already
|
||||
// capped it in the worker (prepareValue there), but a forged done
|
||||
// message bypasses the bootstrap entirely — without this, model code
|
||||
// could flood the host past maxValueBytes. Honest values pass
|
||||
// unchanged (see VALUE_RENDER_SLACK); the error text is bounded too.
|
||||
finish({
|
||||
...prepareValue(message.value, this.config.maxValueBytes + VALUE_RENDER_SLACK),
|
||||
...message.error ? { error: { kind: 'exception' as const, message: truncateUtf8Bytes(message.error.message, this.config.maxValueBytes) } } : {},
|
||||
})
|
||||
}
|
||||
|
||||
const onCall = (message: WorkerToHost): void => {
|
||||
if (message.type !== 'call' || settled) return
|
||||
// Hostile-peer rules: a duplicate id is ignored, an unknown name is
|
||||
// answered with a failure, and a binding throw/reject becomes the
|
||||
// program-side rejection — contained here, never a host crash.
|
||||
if (answered.has(message.id)) return
|
||||
answered.add(message.id)
|
||||
const reply = (payload: ReplyMessage): void => {
|
||||
if (settled) return
|
||||
try {
|
||||
worker.postMessage(payload)
|
||||
} catch {
|
||||
// The reply value failed structured clone; renegotiate as an error
|
||||
// reply, which is always clone-plain. Nothing else throws here.
|
||||
worker.postMessage({ type: 'reply', id: message.id, ok: false, message: 'binding resolution is not structured-cloneable' })
|
||||
}
|
||||
}
|
||||
const record = bindings.get(message.global)
|
||||
// Own-property lookup only: a forged name like 'constructor' or
|
||||
// 'hasOwnProperty' must not walk the record's prototype chain and
|
||||
// reach a callable the consumer never declared.
|
||||
const fn = record && Object.hasOwn(record, message.name) ? record[message.name] : undefined
|
||||
if (typeof fn !== 'function') {
|
||||
reply({ type: 'reply', id: message.id, ok: false, message: `unknown binding ${JSON.stringify(`${message.global}.${message.name}`)}` })
|
||||
return
|
||||
}
|
||||
void (async () => {
|
||||
try {
|
||||
reply({ type: 'reply', id: message.id, ok: true, value: await fn(message.args) })
|
||||
} catch (error: unknown) {
|
||||
reply({ type: 'reply', id: message.id, ok: false, message: messageOf(error) })
|
||||
}
|
||||
})()
|
||||
}
|
||||
|
||||
worker.on('message', (raw: unknown) => {
|
||||
// Parse before touching: the peer can post ANY shape, and a throw in
|
||||
// this listener would crash the host process. Junk drops silently.
|
||||
const message = parseWorkerMessage(raw)
|
||||
if (!message) return
|
||||
if (message.type === 'log' && !settled) admit(message.entry, logs)
|
||||
onCall(message)
|
||||
onDone(message)
|
||||
})
|
||||
worker.on('error', (error: Error) => {
|
||||
finish({ error: { kind: 'worker-exit', message: `worker error: ${error.message}` } })
|
||||
})
|
||||
worker.on('exit', (exitCode: number) => {
|
||||
finish({ error: { kind: 'worker-exit', message: `worker exited with code ${exitCode} before completing` } })
|
||||
})
|
||||
|
||||
// The compute budget reads the worker's own measured busy time, so a
|
||||
// hot loop expires it no matter what dispatches are in flight, while a
|
||||
// program idling on a slow binding accrues nothing.
|
||||
const eluTimer = setInterval(() => {
|
||||
const elu = worker.performance.eventLoopUtilization()
|
||||
if (elu.active > this.config.computeMs) {
|
||||
finish({ error: { kind: 'timeout', message: `compute budget exhausted (${this.config.computeMs}ms busy)` } })
|
||||
}
|
||||
}, ELU_POLL_INTERVAL_MS)
|
||||
const wallTimer = setTimeout(() => {
|
||||
finish({ error: { kind: 'timeout', message: `wall-clock ceiling reached (${this.config.maxWallMs}ms)` } })
|
||||
}, this.config.maxWallMs)
|
||||
const onAbort = (): void => {
|
||||
finish({ error: { kind: 'abort', message: String(request.signal?.reason) } })
|
||||
}
|
||||
request.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
|
||||
const live: LiveRun = {
|
||||
worker,
|
||||
finished,
|
||||
settle: (failure: CodeRunFailure) => { finish({ error: failure }) },
|
||||
}
|
||||
this.live.add(live)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
export default WorkerCodeRuntime
|
||||
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* Wire protocol between the host runtime and the worker bootstrap. Everything
|
||||
* crossing the message port is structured-clone-plain and versionless — both
|
||||
* ends ship in this package, always at the same version. The host treats
|
||||
* inbound traffic as HOSTILE (the worker runs model code, which can reach
|
||||
* `parentPort` via `import('node:worker_threads')` and forge any of these
|
||||
* shapes); the worker treats inbound traffic as trusted.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-code-runtime-worker/src/protocol
|
||||
*/
|
||||
|
||||
import type { CodeLogEntry } from '@deepseek-ai/dsh-code-runtime'
|
||||
|
||||
/** What the host hands the worker at spawn, via `workerData`. */
|
||||
export interface WorkerBootData {
|
||||
/** The type-stripped (plain JS) program body. */
|
||||
code: string
|
||||
/** Binding namespaces to materialize: the global name plus the function names (functions themselves stay host-side). */
|
||||
namespaces: { global: string; names: string[] }[]
|
||||
/** Shared byte budget for captured log text; exceeding it drops further entries after one in-band marker. */
|
||||
maxLogBytes: number
|
||||
/** Byte cap for the rendered completion value (see the value-preparation contract in bootstrap.ts). */
|
||||
maxValueBytes: number
|
||||
}
|
||||
|
||||
/** Worker → host: one bridged binding call. */
|
||||
export interface CallMessage {
|
||||
type: 'call'
|
||||
/** Worker-issued correlation id; the host answers each id at most once and ignores duplicates. */
|
||||
id: number
|
||||
/** The namespace global the call targets. */
|
||||
global: string
|
||||
/** The function name within the namespace. */
|
||||
name: string
|
||||
/** The single argument, structured-clone-plain. */
|
||||
args: unknown
|
||||
}
|
||||
|
||||
/** Worker → host: one captured log entry, streamed eagerly so output survives a mid-run termination (timeout, abort, OOM). */
|
||||
export interface LogMessage {
|
||||
type: 'log'
|
||||
entry: CodeLogEntry
|
||||
}
|
||||
|
||||
/**
|
||||
* Worker → host: the program settled. `error` carries a program exception
|
||||
* (the only failure the bootstrap itself can report — budgets, aborts, and
|
||||
* substrate death are observed host-side). `value` is present only on a
|
||||
* clean completion that produced one (already size-capped and
|
||||
* clone-safe per the bootstrap's value preparation). Logs are NOT carried
|
||||
* here — they streamed eagerly as {@link LogMessage}s.
|
||||
*/
|
||||
export interface DoneMessage {
|
||||
type: 'done'
|
||||
value?: unknown
|
||||
error?: { message: string }
|
||||
}
|
||||
|
||||
/** Every message the worker sends. */
|
||||
export type WorkerToHost = CallMessage | LogMessage | DoneMessage
|
||||
|
||||
/** Host → worker: the answer to one {@link CallMessage}. */
|
||||
export type ReplyMessage =
|
||||
| { type: 'reply'; id: number; ok: true; value: unknown }
|
||||
| { type: 'reply'; id: number; ok: false; message: string }
|
||||
|
||||
/**
|
||||
* The in-band marker entry text announcing that log capture stopped at the
|
||||
* byte budget. Shared wire vocabulary: the worker's LogBuffer emits it when
|
||||
* ITS budget exhausts, and the host emits the identical text when its own
|
||||
* ledger drops an entry first (forged port traffic, stray pipe bytes) — so
|
||||
* a truncated run reads the same however the cap was hit.
|
||||
* @param maxBytes - the configured `maxLogBytes` the marker names.
|
||||
* @returns the marker line.
|
||||
*/
|
||||
export function logTruncationMarker(maxBytes: number): string {
|
||||
return `[dsh-code-runtime-worker] log capture truncated at ${maxBytes} bytes`
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
/**
|
||||
* The worker-thread entrypoint: self-executing glue over
|
||||
* `bootstrap.ts`'s {@link runWorkerMain}, kept to the spawn wiring alone.
|
||||
* Like `bin.ts` CLI entrypoints, this file executes only inside a spawned
|
||||
* worker isolate — a place the coverage provider cannot observe — so it is
|
||||
* excluded from the coverage gate while every line of actual logic lives in
|
||||
* `bootstrap.ts`, unit-tested in-process; the real spawn path is pinned by
|
||||
* the integration tests that run genuine workers.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-code-runtime-worker/src/worker
|
||||
*/
|
||||
|
||||
import { parentPort, workerData } from 'node:worker_threads'
|
||||
import { runWorkerMain } from './bootstrap.ts'
|
||||
import type { WorkerBootData } from './protocol.ts'
|
||||
|
||||
// A worker always has a parent port; guard loudly rather than run detached.
|
||||
if (!parentPort) throw new Error('dsh-code-runtime-worker: worker entry loaded outside a worker thread')
|
||||
|
||||
await runWorkerMain(parentPort, workerData as WorkerBootData, { stdout: process.stdout, stderr: process.stderr })
|
||||
@@ -0,0 +1,273 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { EventEmitter } from 'node:events'
|
||||
import { LogBuffer, makeConsoleShim, makeNamespaces, captureStreamWrites, prepareValue, runWorkerMain, truncateUtf8Bytes, wireReplies } from '@deepseek-ai/dsh-code-runtime-worker/src/bootstrap.ts'
|
||||
import type { BootstrapPort, PatchableStream, PendingCall } from '@deepseek-ai/dsh-code-runtime-worker/src/bootstrap.ts'
|
||||
import type { ReplyMessage, WorkerToHost } from '@deepseek-ai/dsh-code-runtime-worker/src/protocol.ts'
|
||||
import type { CodeLogEntry } from '@deepseek-ai/dsh-code-runtime'
|
||||
|
||||
/**
|
||||
* An in-process stand-in for the worker's parentPort: the test plays the
|
||||
* HOST side — inspect what the bootstrap posted, feed replies back — so
|
||||
* every line of worker-side logic runs under coverage without spawning an
|
||||
* isolate (real-worker behavior is pinned by runtime.spec.ts).
|
||||
*/
|
||||
class FakePort implements BootstrapPort {
|
||||
sent: WorkerToHost[] = []
|
||||
private readonly emitter = new EventEmitter()
|
||||
/** Host-scripted responder; return undefined to leave the call pending. */
|
||||
respond: (message: WorkerToHost) => ReplyMessage | undefined = () => undefined
|
||||
|
||||
postMessage(message: WorkerToHost): void {
|
||||
this.sent.push(message)
|
||||
const reply = this.respond(message)
|
||||
if (reply) queueMicrotask(() => this.emitter.emit('message', reply))
|
||||
}
|
||||
|
||||
on(event: 'message', listener: (message: ReplyMessage) => void): void {
|
||||
this.emitter.on(event, listener)
|
||||
}
|
||||
|
||||
deliver(message: ReplyMessage): void {
|
||||
this.emitter.emit('message', message)
|
||||
}
|
||||
|
||||
logs(): CodeLogEntry[] {
|
||||
return this.sent.filter(message => message.type === 'log').map(message => message.entry)
|
||||
}
|
||||
|
||||
done(): WorkerToHost | undefined {
|
||||
return this.sent.find(message => message.type === 'done')
|
||||
}
|
||||
}
|
||||
|
||||
function fakeStreams(): { stdout: PatchableStream; stderr: PatchableStream } {
|
||||
return { stdout: { write: () => true }, stderr: { write: () => true } }
|
||||
}
|
||||
|
||||
const BOOT = { maxLogBytes: 65_536, maxValueBytes: 32_768 }
|
||||
|
||||
describe('LogBuffer', () => {
|
||||
it('streams entries to the sink until the byte budget, then emits one marker and drops the rest', () => {
|
||||
const seen: CodeLogEntry[] = []
|
||||
const buffer = new LogBuffer(10, entry => seen.push(entry))
|
||||
buffer.push({ source: 'console', level: 'log', text: '12345' })
|
||||
buffer.push({ source: 'console', level: 'log', text: '123456' })
|
||||
buffer.push({ source: 'console', level: 'log', text: 'dropped' })
|
||||
expect(seen.map(entry => entry.text)).toEqual([
|
||||
'12345',
|
||||
'[dsh-code-runtime-worker] log capture truncated at 10 bytes',
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('makeConsoleShim', () => {
|
||||
it('captures the five levels and renders non-strings inspect-style', () => {
|
||||
const seen: CodeLogEntry[] = []
|
||||
const shim = makeConsoleShim(new LogBuffer(1_000, entry => seen.push(entry)))
|
||||
shim.log('plain', { a: 1 })
|
||||
shim.info('i')
|
||||
shim.warn('w')
|
||||
shim.error('e')
|
||||
shim.debug('d')
|
||||
expect(seen.map(entry => entry.level)).toEqual(['log', 'info', 'warn', 'error', 'debug'])
|
||||
expect(seen[0]?.text).toBe('plain { a: 1 }')
|
||||
expect(seen.every(entry => entry.source === 'console')).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
describe('captureStreamWrites', () => {
|
||||
it('redirects writes into the buffer and restores on request', () => {
|
||||
const seen: CodeLogEntry[] = []
|
||||
const buffer = new LogBuffer(1_000, entry => seen.push(entry))
|
||||
let underlying = ''
|
||||
const stream: PatchableStream = { write: (chunk: unknown) => { underlying += String(chunk); return true } }
|
||||
const restore = captureStreamWrites(buffer, stream, 'stdout')
|
||||
stream.write('captured', 'utf8')
|
||||
stream.write(Buffer.from('bytes'))
|
||||
restore()
|
||||
stream.write('after')
|
||||
expect(seen.map(entry => entry.text)).toEqual(['captured', 'bytes'])
|
||||
expect(seen[0]).toMatchObject({ source: 'stdout' })
|
||||
expect(underlying).toBe('after')
|
||||
})
|
||||
|
||||
it('invokes the write callback asynchronously, in both optional-encoding shapes', async () => {
|
||||
const buffer = new LogBuffer(1_000, () => {})
|
||||
const stream: PatchableStream = { write: () => true }
|
||||
captureStreamWrites(buffer, stream, 'stdout')
|
||||
const calls: (Error | null | undefined)[] = []
|
||||
stream.write('two-arg', (error?: Error | null) => calls.push(error))
|
||||
stream.write('three-arg', 'utf8', (error?: Error | null) => calls.push(error))
|
||||
// Node's contract: the callback fires after the write call returns.
|
||||
expect(calls).toEqual([])
|
||||
await new Promise<void>(resolve => stream.write('awaited flush', resolve))
|
||||
expect(calls).toEqual([null, null])
|
||||
})
|
||||
|
||||
it('still fires the callback for a write the exhausted budget drops', async () => {
|
||||
const buffer = new LogBuffer(4, () => {})
|
||||
const stream: PatchableStream = { write: () => true }
|
||||
captureStreamWrites(buffer, stream, 'stdout')
|
||||
stream.write('this write overflows the budget and is dropped')
|
||||
await new Promise<void>(resolve => stream.write('also dropped', resolve))
|
||||
})
|
||||
})
|
||||
|
||||
describe('prepareValue', () => {
|
||||
it('omits undefined, passes small cloneable values raw', () => {
|
||||
expect(prepareValue(undefined, 100)).toEqual({})
|
||||
expect(prepareValue({ a: [1, 'two'] }, 100)).toEqual({ value: { a: [1, 'two'] } })
|
||||
})
|
||||
|
||||
it('replaces a non-cloneable value with its rendering', () => {
|
||||
const { value } = prepareValue({ fn: () => 1 }, 1_000)
|
||||
expect(typeof value).toBe('string')
|
||||
expect(value).toContain('fn')
|
||||
})
|
||||
|
||||
it('replaces an oversized value with a truncation-marked capped rendering', () => {
|
||||
const { value } = prepareValue('x'.repeat(50), 10)
|
||||
expect(value).toBe(`${'x'.repeat(10)}… [truncated]`)
|
||||
})
|
||||
|
||||
it('measures a container by its structured-clone wire size, not its bounded rendering', () => {
|
||||
// The bounded inspect rendering of a huge array is tiny ("... N more
|
||||
// items"), but its real cross-boundary size is not — the cap must catch
|
||||
// it, replacing the value with that bounded rendering.
|
||||
const huge = new Array(50_000).fill(7)
|
||||
const { value } = prepareValue(huge, 1_000)
|
||||
expect(typeof value).toBe('string')
|
||||
expect(value).toContain('more items')
|
||||
})
|
||||
|
||||
it('caps a multibyte string by UTF-8 bytes, not UTF-16 length', () => {
|
||||
// 4 code units but 12 UTF-8 bytes: a length-counting cap would pass the
|
||||
// full string through untruncated.
|
||||
expect(prepareValue('€€€€', 4)).toEqual({ value: '€… [truncated]' })
|
||||
})
|
||||
|
||||
it('caps a multibyte rendering by UTF-8 bytes too', () => {
|
||||
// Wire size (24-byte string inside an array) exceeds the cap, so the
|
||||
// value crosses as its rendering — whose truncation must also be
|
||||
// byte-exact: "[ '" (3 bytes) + two € (6 bytes) = 9; a third € would
|
||||
// overflow the 10-byte budget.
|
||||
expect(prepareValue(['€€€€€€€€'], 10)).toEqual({ value: "[ '€€… [truncated]" })
|
||||
})
|
||||
})
|
||||
|
||||
describe('truncateUtf8Bytes', () => {
|
||||
it('returns a fitting string whole', () => {
|
||||
expect(truncateUtf8Bytes('fits', 4)).toBe('fits')
|
||||
})
|
||||
|
||||
it('cuts at a code-point boundary, never mid-surrogate-pair', () => {
|
||||
// Each 😀 is one code point, two code units, four UTF-8 bytes: a 5-byte
|
||||
// budget fits exactly one — and never leaves a lone surrogate behind.
|
||||
const cut = truncateUtf8Bytes('😀😀', 5)
|
||||
expect(cut).toBe('😀')
|
||||
expect(Buffer.byteLength(truncateUtf8Bytes('😀😀', 3), 'utf8')).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('makeNamespaces', () => {
|
||||
it('exposes prototype-colliding names as ordinary own properties', async () => {
|
||||
const port = new FakePort()
|
||||
port.respond = message => message.type === 'call' ? { type: 'reply', id: message.id, ok: true, value: `${message.name}-ok` } : undefined
|
||||
const pending = new Map<number, PendingCall>()
|
||||
wireReplies(port, pending)
|
||||
const [tools] = makeNamespaces({ namespaces: [{ global: 'tools', names: ['__proto__', 'constructor', 'toString'] }] }, port, pending, { value: 1 }) as [Record<string, (args: unknown) => Promise<unknown>>]
|
||||
expect(Object.getPrototypeOf(tools)).toBeNull()
|
||||
await expect(tools['__proto__']?.({})).resolves.toBe('__proto__-ok')
|
||||
await expect(tools['constructor']?.({})).resolves.toBe('constructor-ok')
|
||||
await expect(tools['toString']?.({})).resolves.toBe('toString-ok')
|
||||
})
|
||||
|
||||
it('rejects a non-cloneable argument without leaking the pending entry', async () => {
|
||||
let firstCall = true
|
||||
const throwingPort: BootstrapPort = {
|
||||
// First call throws an Error (the real DataCloneError shape), the
|
||||
// second a bare string — the rejection renders both.
|
||||
postMessage: () => {
|
||||
if (firstCall) { firstCall = false; throw new Error('DataCloneError-ish') }
|
||||
throw 'raw-clone-failure'
|
||||
},
|
||||
on: () => {},
|
||||
}
|
||||
const pending = new Map<number, PendingCall>()
|
||||
const [tools] = makeNamespaces({ namespaces: [{ global: 'tools', names: ['x'] }] }, throwingPort, pending, { value: 1 }) as [Record<string, (args: unknown) => Promise<unknown>>]
|
||||
await expect(tools.x?.(() => 1)).rejects.toThrow(/structured-cloneable: DataCloneError-ish/)
|
||||
await expect(tools.x?.(() => 1)).rejects.toThrow(/structured-cloneable: raw-clone-failure/)
|
||||
expect(pending.size).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('runWorkerMain', () => {
|
||||
it('runs a program end-to-end: bindings, console, return value', async () => {
|
||||
const port = new FakePort()
|
||||
port.respond = message => message.type === 'call' ? { type: 'reply', id: message.id, ok: true, value: (message.args as { n: number }).n * 2 } : undefined
|
||||
await runWorkerMain(port, {
|
||||
...BOOT,
|
||||
code: 'const doubled = await tools.double({ n: 21 }); console.log("got", doubled); return { doubled };',
|
||||
namespaces: [{ global: 'tools', names: ['double'] }],
|
||||
}, fakeStreams())
|
||||
expect(port.logs()).toEqual([{ source: 'console', level: 'log', text: 'got 42' }])
|
||||
expect(port.done()).toEqual({ type: 'done', value: { doubled: 42 } })
|
||||
})
|
||||
|
||||
it('reports a thrown program error on the done message', async () => {
|
||||
const port = new FakePort()
|
||||
await runWorkerMain(port, { ...BOOT, code: 'throw new Error("boom")', namespaces: [] }, fakeStreams())
|
||||
const done = port.done()
|
||||
expect(done?.type).toBe('done')
|
||||
expect(done?.type === 'done' ? done.error?.message : undefined).toContain('boom')
|
||||
expect(done?.type === 'done' ? done.value : undefined).toBeUndefined()
|
||||
})
|
||||
|
||||
it('renders non-Error throws and stack-less Errors on the done message', async () => {
|
||||
const rawPort = new FakePort()
|
||||
await runWorkerMain(rawPort, { ...BOOT, code: 'throw "raw-throw"', namespaces: [] }, fakeStreams())
|
||||
expect(rawPort.done()).toEqual({ type: 'done', error: { message: 'raw-throw' } })
|
||||
|
||||
const barePort = new FakePort()
|
||||
await runWorkerMain(barePort, { ...BOOT, code: 'const e = new Error("bare"); e.stack = undefined; throw e', namespaces: [] }, fakeStreams())
|
||||
expect(barePort.done()).toEqual({ type: 'done', error: { message: 'bare' } })
|
||||
})
|
||||
|
||||
it('surfaces a host failure reply as a program-side rejection it can catch', async () => {
|
||||
const port = new FakePort()
|
||||
port.respond = message => message.type === 'call' ? { type: 'reply', id: message.id, ok: false, message: 'denied by host' } : undefined
|
||||
await runWorkerMain(port, {
|
||||
...BOOT,
|
||||
code: 'try { await tools.x({}) } catch (error) { return `caught: ${error.message}` }',
|
||||
namespaces: [{ global: 'tools', names: ['x'] }],
|
||||
}, fakeStreams())
|
||||
expect(port.done()).toEqual({ type: 'done', value: 'caught: denied by host' })
|
||||
})
|
||||
|
||||
it('ignores replies for unknown pending ids', async () => {
|
||||
const port = new FakePort()
|
||||
port.respond = (message) => {
|
||||
if (message.type !== 'call') return undefined
|
||||
// Deliver a stray reply first; the real one follows.
|
||||
port.deliver({ type: 'reply', id: 9_999, ok: true, value: 'stray' })
|
||||
return { type: 'reply', id: message.id, ok: true, value: 'real' }
|
||||
}
|
||||
await runWorkerMain(port, {
|
||||
...BOOT,
|
||||
code: 'return await tools.x({})',
|
||||
namespaces: [{ global: 'tools', names: ['x'] }],
|
||||
}, fakeStreams())
|
||||
expect(port.done()).toEqual({ type: 'done', value: 'real' })
|
||||
})
|
||||
|
||||
it('captures raw stream writes through the patched process streams', async () => {
|
||||
const port = new FakePort()
|
||||
const streams = fakeStreams()
|
||||
await runWorkerMain(port, { ...BOOT, code: 'return 1', namespaces: [] }, streams)
|
||||
streams.stdout.write('never seen — already restored? no: patch persists in worker')
|
||||
// The patch stays installed for the worker's lifetime; writes during the
|
||||
// program landed in order. Here the program wrote nothing via streams, so
|
||||
// only the post-run write above went through the patched slot.
|
||||
expect(port.logs().at(-1)).toMatchObject({ source: 'stdout' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,55 @@
|
||||
import { spawn } from 'node:child_process'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
/**
|
||||
* BUILT-ARTIFACT smoke for the published package (the real-load-path guard
|
||||
* from docs/testing.md): the unit suite runs `src/` under vitest, where the
|
||||
* worker entry resolves to `src/worker.ts` — a consumer runs `lib/index.js`
|
||||
* under plain `node`, where it must resolve the sibling `lib/worker.js`
|
||||
* bundle instead. This spawns plain `node` (NOT tsx) from inside the package
|
||||
* directory and imports the package BY NAME, so resolution flows through the
|
||||
* real `exports` map exactly as it would from a downstream install; the
|
||||
* program exercises the type-strip, the worker spawn, the binding bridge,
|
||||
* and log capture end-to-end through the built bundles.
|
||||
*
|
||||
* It build-gates: SKIPS when the built artifacts are absent (suite run
|
||||
* without `pnpm run build`); CI runs it after the build step. KEYLESS — no
|
||||
* model is involved.
|
||||
*/
|
||||
|
||||
const pkgDir = fileURLToPath(new URL('..', import.meta.url))
|
||||
const built = ['lib/index.js', 'lib/worker.js'].every(file => existsSync(join(pkgDir, file)))
|
||||
&& existsSync(join(pkgDir, '../code-runtime/lib/index.js'))
|
||||
|
||||
describe.skipIf(!built)('built lib real load path (plain node)', () => {
|
||||
it('runs a TypeScript program with a binding through lib/index.js and its lib/worker.js entry', async () => {
|
||||
const script = `
|
||||
const { Context } = await import('cordis')
|
||||
const { WorkerCodeRuntime } = await import('@deepseek-ai/dsh-code-runtime-worker')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WorkerCodeRuntime, {})
|
||||
const result = await ctx.codeRuntime.run({
|
||||
program: 'const doubled: number = await tools.double({ n: 21 }); console.log("halfway", doubled); return doubled;',
|
||||
bindings: [{ global: 'tools', functions: { double: async args => args.n * 2 } }],
|
||||
})
|
||||
console.log(JSON.stringify(result))
|
||||
process.exit(0)
|
||||
`
|
||||
const child = spawn(process.execPath, ['--input-type=module', '-e', script], { cwd: pkgDir, stdio: ['ignore', 'pipe', 'pipe'] })
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
child.stdout.on('data', (chunk: Buffer) => { stdout += chunk.toString('utf8') })
|
||||
child.stderr.on('data', (chunk: Buffer) => { stderr += chunk.toString('utf8') })
|
||||
const exitCode = await new Promise<number | null>(resolve => child.on('close', resolve))
|
||||
|
||||
expect(exitCode, `stderr:\n${stderr}`).toBe(0)
|
||||
const lastLine = stdout.trim().split('\n').at(-1) ?? ''
|
||||
const result = JSON.parse(lastLine) as { value?: unknown; logs: { source: string; level?: string; text: string }[]; error?: unknown }
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe(42)
|
||||
expect(result.logs).toContainEqual({ source: 'console', level: 'log', text: 'halfway 42' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,451 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { WorkerCodeRuntime } from '@deepseek-ai/dsh-code-runtime-worker'
|
||||
import type { Config } from '@deepseek-ai/dsh-code-runtime-worker'
|
||||
import type { CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
|
||||
|
||||
/**
|
||||
* Integration suite over REAL worker threads (no mocks — workers are cheap
|
||||
* and local, per docs/testing.md's real-over-mock policy). Each test builds
|
||||
* a fresh context so budgets can be tuned per case.
|
||||
*/
|
||||
async function setup(config: Config = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WorkerCodeRuntime, config)
|
||||
const runtime = ctx.codeRuntime as WorkerCodeRuntime
|
||||
return { ctx, runtime }
|
||||
}
|
||||
|
||||
/** Convenience: one namespace `tools` with the given functions. */
|
||||
function tools(functions: Record<string, (args: unknown) => Promise<unknown>>) {
|
||||
return [{ global: 'tools', functions }]
|
||||
}
|
||||
|
||||
describe('WorkerCodeRuntime — programs and bindings (real workers)', () => {
|
||||
it('registers with the seam descriptors', async () => {
|
||||
const { runtime } = await setup()
|
||||
expect(runtime.language).toBe('typescript')
|
||||
expect(runtime.isolation).toBe('worker-thread')
|
||||
})
|
||||
|
||||
it('runs TypeScript (erasable syntax), captures console/stdout in order, returns the value', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: `
|
||||
interface Point { x: number; y: number }
|
||||
const p: Point = { x: 1, y: 2 } as Point;
|
||||
console.log('point', p);
|
||||
process.stdout.write('raw-out\\n');
|
||||
console.warn('careful');
|
||||
return p.x + p.y;
|
||||
`,
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe(3)
|
||||
expect(result.logs.map(entry => [entry.source, entry.level ?? null])).toEqual([
|
||||
['console', 'log'],
|
||||
['stdout', null],
|
||||
['console', 'warn'],
|
||||
])
|
||||
expect(result.logs[0]?.text).toBe('point { x: 1, y: 2 }')
|
||||
})
|
||||
|
||||
it('bridges binding calls both ways and rejects the program-side call on a host rejection', async () => {
|
||||
const { runtime } = await setup()
|
||||
const calls: unknown[] = []
|
||||
const result = await runtime.run({
|
||||
program: `
|
||||
const first = await tools.echo({ n: 1 });
|
||||
let caught = '';
|
||||
try { await tools.fail({}) } catch (error) { caught = error.message }
|
||||
let caughtRaw = '';
|
||||
try { await tools.failRaw({}) } catch (error) { caughtRaw = error.message }
|
||||
return { first, caught, caughtRaw };
|
||||
`,
|
||||
bindings: tools({
|
||||
echo: async (args) => { calls.push(args); return { echoed: args } },
|
||||
fail: async () => { throw new Error('nope') },
|
||||
// A non-Error throw: the host renders it, the program still catches.
|
||||
failRaw: async () => { throw 'raw-nope' },
|
||||
}),
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toEqual({ first: { echoed: { n: 1 } }, caught: 'nope', caughtRaw: 'raw-nope' })
|
||||
expect(calls).toEqual([{ n: 1 }])
|
||||
})
|
||||
|
||||
it('reports non-erasable syntax as an exception without spawning a worker', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({ program: 'enum E { A }\nreturn 1', bindings: [] })
|
||||
expect(result.error?.kind).toBe('exception')
|
||||
expect(result.error?.message).toMatch(/enum|strip/i)
|
||||
})
|
||||
|
||||
it('reports a runtime throw as an exception with the message', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({ program: 'throw new Error("kaboom")', bindings: [] })
|
||||
expect(result.error?.kind).toBe('exception')
|
||||
expect(result.error?.message).toContain('kaboom')
|
||||
})
|
||||
|
||||
it('gives the program an EMPTY environment', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({ program: 'return JSON.stringify(process.env)', bindings: [] })
|
||||
expect(result.value).toBe('{}')
|
||||
})
|
||||
|
||||
it('replaces a non-cloneable return value with a string rendering', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({ program: 'return { f: () => 1 }', bindings: [] })
|
||||
expect(typeof result.value).toBe('string')
|
||||
})
|
||||
|
||||
it('completes a program that returns nothing with no value at all', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({ program: 'const x = 1', bindings: [] })
|
||||
expect(result.error).toBeUndefined()
|
||||
expect('value' in result).toBe(false)
|
||||
})
|
||||
|
||||
it('keeps logs streamed before a failure', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: 'console.log("before"); throw new Error("after-log")',
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error?.kind).toBe('exception')
|
||||
expect(result.logs.map(entry => entry.text)).toContain('before')
|
||||
})
|
||||
})
|
||||
|
||||
describe('WorkerCodeRuntime — budgets and containment (real workers)', () => {
|
||||
it('ends a hot loop at the compute budget — including behind a pending decoy dispatch', async () => {
|
||||
const { runtime } = await setup({ computeMs: 300, maxWallMs: 30_000 })
|
||||
const result = await runtime.run({
|
||||
// The decoy: fire a call at a never-resolving binding WITHOUT awaiting,
|
||||
// then spin. Host-side pending-call bookkeeping would pause a naive
|
||||
// budget here; measured busy time cannot be fooled.
|
||||
program: 'void tools.slow({}); for (;;) {}',
|
||||
bindings: tools({ slow: () => new Promise(() => {}) }),
|
||||
})
|
||||
expect(result.error?.kind).toBe('timeout')
|
||||
expect(result.error?.message).toContain('compute budget')
|
||||
}, 15_000)
|
||||
|
||||
it('does not charge time spent awaiting a slow binding against the compute budget', async () => {
|
||||
const { runtime } = await setup({ computeMs: 250, maxWallMs: 30_000 })
|
||||
const result = await runtime.run({
|
||||
program: 'return await tools.slow({})',
|
||||
bindings: tools({ slow: () => new Promise(resolve => setTimeout(() => { resolve('slow-done') }, 700)) }),
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('slow-done')
|
||||
}, 15_000)
|
||||
|
||||
it('ends an idle-forever run at the wall-clock ceiling', async () => {
|
||||
const { runtime } = await setup({ computeMs: 30_000, maxWallMs: 400 })
|
||||
const result = await runtime.run({
|
||||
program: 'await tools.never({}); return 1',
|
||||
bindings: tools({ never: () => new Promise(() => {}) }),
|
||||
})
|
||||
expect(result.error?.kind).toBe('timeout')
|
||||
expect(result.error?.message).toContain('wall-clock ceiling')
|
||||
}, 15_000)
|
||||
|
||||
it('reports an abort mid-run and stops the worker', async () => {
|
||||
const { runtime } = await setup()
|
||||
const controller = new AbortController()
|
||||
setTimeout(() => { controller.abort('user-cancel') }, 150)
|
||||
const result = await runtime.run({ program: 'for (;;) {}', bindings: [], signal: controller.signal })
|
||||
expect(result.error).toEqual({ kind: 'abort', message: 'user-cancel' })
|
||||
}, 15_000)
|
||||
|
||||
it('reports a pre-aborted signal without spawning', async () => {
|
||||
const { runtime } = await setup()
|
||||
const controller = new AbortController()
|
||||
controller.abort('too-late')
|
||||
const result = await runtime.run({ program: 'return 1', bindings: [], signal: controller.signal })
|
||||
expect(result.error).toEqual({ kind: 'abort', message: 'too-late' })
|
||||
})
|
||||
|
||||
it('drops a binding resolution that lands after the run settled', async () => {
|
||||
const { runtime } = await setup()
|
||||
const controller = new AbortController()
|
||||
let replyDelivered!: Promise<void>
|
||||
const result = await runtime.run({
|
||||
program: 'void tools.late({}); for (;;) {}',
|
||||
bindings: tools({
|
||||
// Anchored on invocation: abort 100ms after the call reaches the
|
||||
// host, resolve 400ms after — by then the run has settled, so the
|
||||
// resolution's reply hits the post-settlement drop.
|
||||
late: () => new Promise((resolve) => {
|
||||
setTimeout(() => { controller.abort('cancel-now') }, 100)
|
||||
replyDelivered = new Promise(done => setTimeout(() => { resolve('too-late'); done() }, 400))
|
||||
}),
|
||||
}),
|
||||
signal: controller.signal,
|
||||
})
|
||||
expect(result.error).toEqual({ kind: 'abort', message: 'cancel-now' })
|
||||
// Let the late resolution actually fire so its reply executes instead of
|
||||
// being cancelled with the test.
|
||||
await replyDelivered
|
||||
}, 15_000)
|
||||
|
||||
it('contains an OOM under resourceLimits as worker-exit, host process healthy', async () => {
|
||||
const { runtime } = await setup({ maxOldGenerationSizeMb: 32 })
|
||||
const result = await runtime.run({
|
||||
program: 'const hog = []; for (;;) hog.push(new Array(1e6).fill(1));',
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error?.kind).toBe('worker-exit')
|
||||
// And the host is fine: run something else.
|
||||
const after = await runtime.run({ program: 'return "alive"', bindings: [] })
|
||||
expect(after.value).toBe('alive')
|
||||
}, 30_000)
|
||||
|
||||
it('truncates runaway log output at the byte budget with an in-band marker', async () => {
|
||||
const { runtime } = await setup({ maxLogBytes: 300 })
|
||||
const result = await runtime.run({
|
||||
program: 'for (let i = 0; i < 1000; i++) console.log("spam line", i); return 1',
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.logs.at(-1)?.text).toContain('truncated at 300 bytes')
|
||||
const total = result.logs.reduce((sum, entry) => sum + Buffer.byteLength(entry.text, 'utf8'), 0)
|
||||
expect(total).toBeLessThan(1_000)
|
||||
})
|
||||
|
||||
it('caps an oversized return value with a truncation marker', async () => {
|
||||
const { runtime } = await setup({ maxValueBytes: 64 })
|
||||
const result = await runtime.run({ program: 'return "y".repeat(10_000)', bindings: [] })
|
||||
expect(result.value).toBe(`${'y'.repeat(64)}… [truncated]`)
|
||||
})
|
||||
|
||||
it('caps a multibyte return value by UTF-8 bytes, not string length', async () => {
|
||||
// 4 code units, 12 UTF-8 bytes: a length-counting cap would let the full
|
||||
// string cross. The worker's byte-exact capped rendering then passes the
|
||||
// host re-cap unchanged (cap + marker is exactly the granted slack).
|
||||
const { runtime } = await setup({ maxValueBytes: 4 })
|
||||
const result = await runtime.run({ program: 'return "€€€€"', bindings: [] })
|
||||
expect(result.value).toBe('€… [truncated]')
|
||||
})
|
||||
|
||||
it('completes a program that awaits its write callback, capturing the chunk', async () => {
|
||||
// Node's write(chunk[, encoding][, callback]) contract: dropping the
|
||||
// callback would leave this promise pending until the wall ceiling and
|
||||
// misreport a completed program as a timeout.
|
||||
const { runtime } = await setup({ maxWallMs: 2_000 })
|
||||
const result = await runtime.run({
|
||||
program: 'await new Promise(resolve => process.stdout.write("flushed", resolve)); return "done"',
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('done')
|
||||
expect(result.logs).toContainEqual({ source: 'stdout', text: 'flushed' })
|
||||
})
|
||||
|
||||
it('caps a huge container whose bounded rendering is small (wire size, not rendering, is what counts)', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({ program: 'return new Array(50_000).fill(7)', bindings: [] })
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(typeof result.value).toBe('string')
|
||||
expect(result.value).toContain('more items')
|
||||
})
|
||||
|
||||
it('captures pipe writes that bypass the patched write slot as stray logs, capped by the same budget', async () => {
|
||||
const { runtime } = await setup({ maxLogBytes: 4 })
|
||||
const result = await runtime.run({
|
||||
// The bootstrap patches the stream instance's own `write`; going
|
||||
// through the prototype's slot reaches the real pipe underneath, so
|
||||
// the bytes arrive host-side as stray data. The pauses keep the two
|
||||
// writes in separate pipe chunks and let them land before settlement.
|
||||
program: `
|
||||
const write = (text) => Object.getPrototypeOf(process.stdout).write.call(process.stdout, text);
|
||||
write('abcd');
|
||||
await new Promise(resolve => setTimeout(resolve, 150));
|
||||
write('ef');
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
return 1;
|
||||
`,
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs).toContainEqual({ source: 'stdout', text: 'abcd' })
|
||||
expect(result.logs.map(entry => entry.text)).not.toContain('ef')
|
||||
}, 15_000)
|
||||
})
|
||||
|
||||
describe('WorkerCodeRuntime — hostile programs (real workers)', () => {
|
||||
it('survives forged port traffic: unknown binding names, duplicate ids, junk shapes', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: `
|
||||
const { parentPort } = await import('node:worker_threads');
|
||||
parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} });
|
||||
parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} });
|
||||
parentPort.postMessage({ type: 'call', id: 7778, global: 'tools', name: 'constructor', args: {} });
|
||||
parentPort.postMessage({ type: 'junk' });
|
||||
return await tools.real({});
|
||||
`,
|
||||
bindings: tools({ real: async () => 'still-works' }),
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('still-works')
|
||||
})
|
||||
|
||||
it('survives arbitrary junk on the port: non-objects, junk types, malformed calls, logs, and dones', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: `
|
||||
const { parentPort } = await import('node:worker_threads');
|
||||
for (const junk of [
|
||||
null, 42, 'junk', [],
|
||||
{ type: 'nope' },
|
||||
{ type: 'call' },
|
||||
{ type: 'call', id: 'x', global: 'tools', name: 'real', args: {} },
|
||||
{ type: 'call', id: 1e9, global: 7, name: 'real', args: {} },
|
||||
{ type: 'call', id: 1e9, global: 'tools', name: 7, args: {} },
|
||||
{ type: 'log' },
|
||||
{ type: 'log', entry: null },
|
||||
{ type: 'log', entry: { source: 'stdout', text: 7 } },
|
||||
{ type: 'log', entry: { source: 'nope', text: 'x' } },
|
||||
{ type: 'log', entry: { source: 'console', level: 'nope', text: 'x' } },
|
||||
{ type: 'log', entry: { source: 'console', level: 7, text: 'x' } },
|
||||
{ type: 'done', error: 5 },
|
||||
{ type: 'done', error: { message: 5 } },
|
||||
]) parentPort.postMessage(junk);
|
||||
return await tools.real({});
|
||||
`,
|
||||
bindings: tools({ real: async () => 'still-works' }),
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('still-works')
|
||||
expect(result.logs).toEqual([])
|
||||
})
|
||||
|
||||
it('caps forged log floods and forged done values at the configured budgets, dropping forged extra fields', async () => {
|
||||
const { runtime } = await setup({ maxLogBytes: 200, maxValueBytes: 64 })
|
||||
const result = await runtime.run({
|
||||
// Forged messages bypass the worker-side LogBuffer and prepareValue
|
||||
// entirely — only the host-side ledger and re-cap stand between model
|
||||
// code and an unbounded result.
|
||||
program: `
|
||||
const { parentPort } = await import('node:worker_threads');
|
||||
for (let i = 0; i < 50; i++) parentPort.postMessage({ type: 'log', entry: { source: 'stdout', text: 'F'.repeat(100), forged: true } });
|
||||
parentPort.postMessage({ type: 'done', value: 'V'.repeat(100000) });
|
||||
for (;;) {}
|
||||
`,
|
||||
bindings: [],
|
||||
})
|
||||
expect(typeof result.value).toBe('string')
|
||||
const value = result.value as string
|
||||
expect(value.startsWith('V'.repeat(64))).toBe(true)
|
||||
expect(value.endsWith('… [truncated]')).toBe(true)
|
||||
expect(value.length).toBeLessThan(120)
|
||||
const marker = '[dsh-code-runtime-worker] log capture truncated at 200 bytes'
|
||||
const total = result.logs.reduce((sum, entry) => sum + Buffer.byteLength(entry.text, 'utf8'), 0)
|
||||
expect(total).toBeLessThanOrEqual(200 + Buffer.byteLength(marker, 'utf8'))
|
||||
expect(result.logs.at(-1)?.text).toBe(marker)
|
||||
expect(result.logs.every(entry => !('forged' in entry))).toBe(true)
|
||||
})
|
||||
|
||||
it('accepts a forged done carrying both value and error (self-sabotage, contained)', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: `
|
||||
const { parentPort } = await import('node:worker_threads');
|
||||
parentPort.postMessage({ type: 'done', value: 'lied', error: { message: 'fake failure' } });
|
||||
for (;;) {}
|
||||
`,
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.value).toBe('lied')
|
||||
expect(result.error).toEqual({ kind: 'exception', message: 'fake failure' })
|
||||
})
|
||||
|
||||
it('byte-bounds forged multibyte error text at the host', async () => {
|
||||
// Forged error text bypasses the worker entirely; the host bound is a
|
||||
// BYTE bound (two € = 6 bytes fit an 8-byte cap, a third would not).
|
||||
const { runtime } = await setup({ maxValueBytes: 8 })
|
||||
const result = await runtime.run({
|
||||
program: `
|
||||
const { parentPort } = await import('node:worker_threads');
|
||||
parentPort.postMessage({ type: 'done', error: { message: '€'.repeat(1000) } });
|
||||
for (;;) {}
|
||||
`,
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toEqual({ kind: 'exception', message: '€€' })
|
||||
})
|
||||
|
||||
it('answers a binding whose resolution cannot be cloned with a failure reply', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: 'try { await tools.bad({}) } catch (error) { return error.message }',
|
||||
bindings: tools({ bad: async () => (() => 1) }),
|
||||
})
|
||||
expect(result.value).toContain('not structured-cloneable')
|
||||
})
|
||||
|
||||
it('exposes binding names that collide with Object.prototype as ordinary functions', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: 'return [await tools["__proto__"]({}), await tools["constructor"]({}), typeof tools["hasOwnProperty"]]',
|
||||
// Computed keys: a literal `'__proto__': …` entry would SET the record's
|
||||
// prototype instead of declaring a binding of that name.
|
||||
bindings: tools({ ['__proto__']: async () => 'proto-ok', ['constructor']: async () => 'ctor-ok' }),
|
||||
})
|
||||
expect(result.value).toEqual(['proto-ok', 'ctor-ok', 'undefined'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('WorkerCodeRuntime — seam misuse and lifecycle', () => {
|
||||
it('rejects invalid binding globals loudly (identifier, reserved word, duplicate, console)', async () => {
|
||||
const { runtime } = await setup()
|
||||
const cases: [string, RegExp][] = [
|
||||
['not valid!', /not a usable identifier/],
|
||||
['await', /not a usable identifier/],
|
||||
['console', /duplicate binding global/],
|
||||
]
|
||||
for (const [global, message] of cases) {
|
||||
await expect(runtime.run({ program: 'return 1', bindings: [{ global, functions: {} }] })).rejects.toThrow(message)
|
||||
}
|
||||
await expect(runtime.run({
|
||||
program: 'return 1',
|
||||
bindings: [{ global: 'tools', functions: {} }, { global: 'tools', functions: {} }],
|
||||
})).rejects.toThrow(/duplicate binding global/)
|
||||
})
|
||||
|
||||
it('rejects config values that are not positive numbers', async () => {
|
||||
const ctx = new Context()
|
||||
await expect(ctx.plugin(WorkerCodeRuntime, { computeMs: -1 })).rejects.toThrow(/positive number/)
|
||||
})
|
||||
|
||||
it('keeps runs isolated: no state survives from one run to the next', async () => {
|
||||
const { runtime } = await setup()
|
||||
await runtime.run({ program: 'globalThis.leak = "value"; return 1', bindings: [] })
|
||||
const second = await runtime.run({ program: 'return typeof globalThis.leak', bindings: [] })
|
||||
expect(second.value).toBe('undefined')
|
||||
})
|
||||
|
||||
it('disposal aborts in-flight runs, awaits worker exit, and rejects later runs', async () => {
|
||||
const ctx = new Context()
|
||||
const fiber = await ctx.plugin(WorkerCodeRuntime)
|
||||
const runtime = ctx.codeRuntime as WorkerCodeRuntime
|
||||
const inflight: Promise<CodeRunResult> = runtime.run({ program: 'for (;;) {}', bindings: [] })
|
||||
// Give the worker a moment to actually start spinning.
|
||||
await new Promise(resolve => setTimeout(resolve, 200))
|
||||
await fiber.dispose()
|
||||
const result = await inflight
|
||||
expect(result.error).toEqual({ kind: 'abort', message: 'runtime disposed' })
|
||||
await expect(runtime.run({ program: 'return 1', bindings: [] })).rejects.toThrow(/after disposal/)
|
||||
}, 15_000)
|
||||
|
||||
it('removes ctx.codeRuntime when the providing fiber disposes (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
const fiber = await ctx.plugin(WorkerCodeRuntime)
|
||||
expect(ctx.get('codeRuntime')).toBeInstanceOf(WorkerCodeRuntime)
|
||||
await fiber.dispose()
|
||||
expect(ctx.get('codeRuntime')).toBeUndefined()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../code-runtime"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/**
|
||||
* Package-shape override (see the root tsdown.config.ts): besides the
|
||||
* default lib/index.js bundle, the worker BOOTSTRAP ships as its own
|
||||
* sibling entry — `new Worker(new URL('./worker.js', import.meta.url))`
|
||||
* loads it as a file, so it cannot be part of the index bundle. TWO
|
||||
* single-entry builds, not one two-entry build: a multi-entry build emits
|
||||
* the shared bootstrap module as a `lib/bootstrap-*.js` chunk both bundles
|
||||
* import, which the package.json `files` whitelist (deliberately exact)
|
||||
* would omit from the packed artifact — each single-entry build inlines its
|
||||
* own bootstrap copy instead, keeping every shipped file self-contained.
|
||||
*/
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/index.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/worker.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
])
|
||||
@@ -0,0 +1,9 @@
|
||||
# guard/ — loop-hygiene guard family
|
||||
|
||||
Behavioral guard plugins that watch the agent loop for unproductive patterns and nudge the model back on course. A single **product** package — there is no interface/implementation seam here, because a guard is a self-contained consumer of existing core seams (`tools/post-execute`, `agent/prompt-submit`, `agent/status`), not a swappable capability.
|
||||
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
| `repeat-tool-guard/` | Advisory reminders when an agent loops on identical tool calls | (listens on `ctx.tools`' waterfalls) |
|
||||
|
||||
Reminders travel as `additionalContext` on the `tools/post-execute` decision; the agent loop appends them as logged `context/message` events after the step's tool results (see [the tools package](../core/tools)), so everything a guard says to the model is reconstructable from the session log.
|
||||
@@ -0,0 +1,37 @@
|
||||
# @deepseek-ai/dsh-repeat-tool-guard
|
||||
|
||||
An advisory loop-breaker, not a model-facing tool: it never appears in the tool list, never vetoes or rewrites a call, and adds exactly one behavior — it watches each agent's stream of tool calls, counts runs of consecutive calls to the same tool with identical canonicalized arguments, and at configured run lengths injects an escalating advisory reminder telling the model to stop repeating itself, re-read the last result, and either change approach or conclude. The decision (retry differently, gather more evidence, or finish) stays entirely with the model: a legitimately repeated call is delayed by nothing and blocked by nothing. Decision record: [the repeat-tool-guard RFC](../../../docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md).
|
||||
|
||||
## Config
|
||||
|
||||
```yaml
|
||||
- id: repeat-tool-guard
|
||||
name: '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
config:
|
||||
thresholds: [3, 5, 8] # default; consecutive counts that trigger a reminder
|
||||
include: [] # tool-name patterns to track; empty ⇒ all tools
|
||||
exclude: [todo_write] # tool-name patterns transparent to the chain
|
||||
argumentsPreviewChars: 500 # default; cap on arguments quoted in the detailed reminder
|
||||
```
|
||||
|
||||
`thresholds` fails loud at plugin load: an empty list, a non-integer, a value below 2, or a duplicate throws, never a silent fall-back to defaults; `argumentsPreviewChars` equally rejects anything but an integer >= 1. The list is normalized to ascending order; the FIRST threshold delivers a short generic nudge, every later threshold delivers the detailed form naming the tool, the run length, and the canonical arguments — head-truncated at `argumentsPreviewChars` with an omitted-count marker, so a looping `write`/`edit` payload cannot ride into the next request unbounded (the chain key always compares the FULL canonical string; the cap bounds the reminder, never the detection).
|
||||
|
||||
`include`/`exclude` entries support `*` wildcards and are predicates over whatever tools exist at call time, not references to registry entries — a pattern matching no currently registered tool is NOT an error (`exclude: [mcp_*]` stays valid in a deployment that loads no MCP tools), unlike `toolOrder`'s referent check.
|
||||
|
||||
## Chain semantics
|
||||
|
||||
The chain key is `(tool name, canonical arguments)` — canonicalization is a deep key-sort plus `JSON.stringify`, so argument objects differing only in property order count as identical. A call identical to the previous tracked call increments the agent's consecutive counter; a different tracked call resets it to 1.
|
||||
|
||||
- **Untracked calls are transparent to the chain.** A call excluded by `include`/`exclude` neither increments nor resets the counter, so `grep X → todo_write → grep X` still counts as two consecutive `grep X` when `todo_write` is excluded. This is what makes exclusion useful: bookkeeping tools interleaved into a loop must not launder it.
|
||||
- **Denied calls count.** Detection sits on `tools/post-execute`, which also runs for calls a `tools/pre-execute` listener denied — a model hammering a denied call is exactly the loop worth breaking.
|
||||
- **Calls without an agent are ignored.** A direct `ctx.tools.execute()` caller has no model to remind and no `AgentId` to key on.
|
||||
- **Per-agent keying.** The tool registry is context-level and subagents interleave through the same waterfall, so chains are keyed by `AgentId`; one agent's repetition never trips another's reminder. A user prompt (`agent/prompt-submit`) resets the submitting agent's chain; agent disposal drops its state.
|
||||
- **In-memory only.** A session resumed from persistence starts with a fresh chain — the guard is a heuristic nudge, not a logged invariant, later reminders are the accepted cost.
|
||||
|
||||
## Reminder delivery
|
||||
|
||||
Reminders ride the post-execute decision's `additionalContext` (source `{kind: 'plugin', plugin: 'repeat-tool-guard'}`), never a `content` replacement: the `tool/result` event stays the tool's own output for audit. The loop buffers the context and appends it as a `context/message` after the step's tool results, which the session renders as the tagged synthetic-user envelope — so the reminder is model-visible, source-attributed, and reconstructable from the session log with no new session event. The guard always delegates via `next()` and folds its reminder onto the downstream decision (both variants — a blocked call still gets the nudge); when a downstream listener attached its own `additionalContext`, the fold concatenates content and carries the guard's `source` (a `HookContext` holds one `MessageSource`; `source.kind` is what framing depends on).
|
||||
|
||||
## Testing
|
||||
|
||||
Unit suites drive a real agent loop against a mock adapter (no network) and cover the chain semantics above to per-file 100%. The snapshot tier owns the transcript surface: a scripted-replay scenario repeats a call five times and pins both reminder tiers (gentle at 3, detailed at 5) as `context/message`s in the ACP transcript.
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-repeat-tool-guard",
|
||||
"description": "Repeat-tool-call guard plugin: advisory reminders when an agent loops on identical tool calls",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
/**
|
||||
* Repeat-tool-call guard: advisory loop-breaker for agents stuck re-issuing
|
||||
* the same tool call with identical arguments.
|
||||
*
|
||||
* Not a model-facing tool — it registers no tool, never vetoes or rewrites a
|
||||
* call, and adds exactly one behavior: watch each agent's stream of tool calls
|
||||
* through the `tools/post-execute` waterfall, count runs of consecutive calls
|
||||
* to the same tool with identical canonicalized arguments, and at configured
|
||||
* run lengths fold an escalating advisory reminder onto the decision's
|
||||
* `additionalContext`. The loop appends that context as a logged
|
||||
* `context/message` after the step's tool results, so the reminder is
|
||||
* model-visible, source-attributed, and reconstructable from the session log
|
||||
* with no new session event. Decision record:
|
||||
* docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md.
|
||||
*
|
||||
* ```yaml
|
||||
* - id: repeat-tool-guard
|
||||
* name: '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
* config:
|
||||
* thresholds: [3, 5, 8] # consecutive counts that trigger a reminder
|
||||
* include: [] # tool-name patterns to track; empty = all tools
|
||||
* exclude: [todo_write] # tool-name patterns transparent to the chain
|
||||
* ```
|
||||
*
|
||||
* Chain state is keyed per {@link AgentId} — the tool registry is a
|
||||
* context-level singleton whose waterfalls interleave every agent's calls, so
|
||||
* a shared counter would let one agent's repetition trip another's reminder.
|
||||
* State is in-memory only: a session resumed from persistence starts with a
|
||||
* fresh chain (the guard is a heuristic nudge, not a logged invariant).
|
||||
*
|
||||
* Plugin export shape: named exports, NO default. The cordis Loader's
|
||||
* `unwrapExports` does `exports.default ?? exports`, so a stray default would
|
||||
* collapse the module to the bare `apply` (see docs/postmortem/0001).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-repeat-tool-guard
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { AgentId, HookContext, PromptDecision } from '@deepseek-ai/dsh-agent'
|
||||
import type { MessageSource } from '@deepseek-ai/dsh-llm'
|
||||
import type { PostToolDecision, ToolExecution } from '@deepseek-ai/dsh-tools'
|
||||
|
||||
export const name = 'repeat-tool-guard'
|
||||
|
||||
/**
|
||||
* Plugin config, validated by the same-named schemastery schema plus the
|
||||
* load-time checks in `apply` (misconfiguration fails loud: an empty
|
||||
* `thresholds` list, a non-integer, a value below 2, or a duplicate throws at
|
||||
* plugin load, never a silent fall-back). `include`/`exclude` entries are
|
||||
* `*`-wildcard predicates over tool names at call time, not references to
|
||||
* registry entries — a pattern matching no currently registered tool is valid
|
||||
* (`exclude: [mcp_*]` must stay legal in a deployment that loads no MCP tools).
|
||||
*/
|
||||
export interface Config {
|
||||
/** Consecutive-repeat counts that trigger a reminder (default `[3, 5, 8]`). */
|
||||
thresholds?: number[]
|
||||
/** Tool-name patterns to track; empty means every tool is tracked. */
|
||||
include?: string[]
|
||||
/** Tool-name patterns transparent to the chain (neither count nor reset). */
|
||||
exclude?: string[]
|
||||
/**
|
||||
* Maximum characters of canonical arguments quoted in the DETAILED reminder
|
||||
* (default 500). Large payloads (a `write` body, a long command) would
|
||||
* otherwise ride into the next request unbounded — precisely in a loop
|
||||
* scenario; the cap bounds the reminder, never the detection (the chain key
|
||||
* always compares the FULL canonical string).
|
||||
*/
|
||||
argumentsPreviewChars?: number
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
thresholds: z.array(z.number()).default([3, 5, 8]),
|
||||
include: z.array(z.string()).default([]),
|
||||
exclude: z.array(z.string()).default([]),
|
||||
argumentsPreviewChars: z.number().default(500),
|
||||
})
|
||||
|
||||
/**
|
||||
* The `{kind:'plugin'}` source stamped on every reminder this guard injects —
|
||||
* the label is load-bearing (an unlabeled context would render as a user
|
||||
* prompt in derived history).
|
||||
*/
|
||||
const PLUGIN_SOURCE: MessageSource = { kind: 'plugin', plugin: 'repeat-tool-guard' }
|
||||
|
||||
/**
|
||||
* The gentle first-threshold reminder. Keyed to `thresholds[0]`, not a literal
|
||||
* count, so a custom first threshold keeps the gentle-then-detailed escalation.
|
||||
*/
|
||||
const GENTLE_REMINDER =
|
||||
'You are repeating the exact same tool call with identical arguments. '
|
||||
+ 'Carefully analyze the previous result before calling again: if the task is '
|
||||
+ 'not complete, try a different approach or different arguments instead of '
|
||||
+ 'repeating the call.'
|
||||
|
||||
/** The detailed later-threshold reminder naming the tool, the run length, and the canonical arguments. */
|
||||
function detailedReminder(toolName: string, count: number, canonicalArguments: string): string {
|
||||
return 'Repeated tool call detected:\n'
|
||||
+ `- tool: ${toolName}\n`
|
||||
+ `- consecutive_calls: ${count}\n`
|
||||
+ `- arguments: ${canonicalArguments}\n`
|
||||
+ 'The repeated calls are not making progress. Do not call this tool with '
|
||||
+ 'these exact arguments again. Inspect the latest result and choose a '
|
||||
+ 'different action, different arguments, or finish the task if enough '
|
||||
+ 'evidence has been gathered.'
|
||||
}
|
||||
|
||||
/**
|
||||
* Deep key-sort of a parsed-JSON value so two argument objects that differ
|
||||
* only in property order canonicalize identically. Arguments reach the guard
|
||||
* as the loop's `JSON.parse` output (or its raw-string fallback for malformed
|
||||
* argument JSON), so JSON's value domain is the whole input domain — no
|
||||
* bigint, cycle, or `undefined` handling exists because no input path can
|
||||
* produce them.
|
||||
*/
|
||||
function sortJsonValue(value: unknown): unknown {
|
||||
if (Array.isArray(value)) return value.map(sortJsonValue)
|
||||
if (value !== null && typeof value === 'object') {
|
||||
const record = value as Record<string, unknown>
|
||||
const sorted: Record<string, unknown> = {}
|
||||
for (const key of Object.keys(record).sort()) {
|
||||
sorted[key] = sortJsonValue(record[key])
|
||||
}
|
||||
return sorted
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** Canonical string form of a call's arguments: deep key-sort, then stringify. */
|
||||
function canonicalize(argumentsValue: unknown): string {
|
||||
return JSON.stringify(sortJsonValue(argumentsValue))
|
||||
}
|
||||
|
||||
/** Compile one `*`-wildcard pattern to an anchored RegExp (every other regex metacharacter is matched literally). */
|
||||
function wildcardToRegExp(pattern: string): RegExp {
|
||||
const escaped = pattern.replace(/[|\\{}()[\]^$+?.]/g, String.raw`\$&`)
|
||||
return new RegExp(`^${escaped.replaceAll('*', '.*')}$`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Head-truncate the canonical arguments for quoting in the detailed reminder,
|
||||
* marking how much was omitted. Bounds only the model-visible text — the
|
||||
* chain key always uses the full canonical string.
|
||||
*/
|
||||
function previewArguments(canonical: string, cap: number): string {
|
||||
if (canonical.length <= cap) return canonical
|
||||
return `${canonical.slice(0, cap)}… (+${canonical.length - cap} more chars)`
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate `thresholds` per the fail-loud contract and return them sorted
|
||||
* ascending (the escalation rule reads `thresholds[0]` as the gentle tier, so
|
||||
* order is normalized here, once).
|
||||
*/
|
||||
function validateThresholds(values: number[]): number[] {
|
||||
if (values.length === 0) {
|
||||
throw new Error('repeat-tool-guard: `thresholds` must not be empty')
|
||||
}
|
||||
for (const value of values) {
|
||||
if (!Number.isInteger(value) || value < 2) {
|
||||
throw new Error(`repeat-tool-guard: invalid threshold ${value} — every threshold must be an integer >= 2`)
|
||||
}
|
||||
}
|
||||
if (new Set(values).size !== values.length) {
|
||||
throw new Error('repeat-tool-guard: `thresholds` must not contain duplicates')
|
||||
}
|
||||
return [...values].sort((a, b) => a - b)
|
||||
}
|
||||
|
||||
/**
|
||||
* Concatenate the guard's reminder context with a downstream listener's
|
||||
* optional one so folding drops neither. The merged block carries the guard's
|
||||
* `source` — a `HookContext` holds one `MessageSource` and the seam cannot
|
||||
* represent mixed provenance; the rendered `context/message` only
|
||||
* distinguishes by `source.kind`, so a downstream plugin's text is still
|
||||
* correctly framed as plugin context.
|
||||
*/
|
||||
function concatContext(ours: HookContext, theirs: HookContext | undefined): HookContext {
|
||||
if (!theirs) return ours
|
||||
return { content: [...ours.content, ...theirs.content], source: ours.source }
|
||||
}
|
||||
|
||||
/** One agent's consecutive-repeat chain: the last tracked call's identity key and its run length. */
|
||||
interface Chain {
|
||||
key: string
|
||||
count: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Install the guard's listeners.
|
||||
* @param ctx - plugin context; listeners are scoped to it and disposed with it.
|
||||
* @param config - validated {@link Config}; `thresholds` is re-checked fail-loud here.
|
||||
*/
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
// schemastery's .default() guarantees the fields are set after validation.
|
||||
const thresholds = validateThresholds(config.thresholds as number[])
|
||||
const thresholdSet = new Set(thresholds)
|
||||
const includePatterns = (config.include as string[]).map(wildcardToRegExp)
|
||||
const excludePatterns = (config.exclude as string[]).map(wildcardToRegExp)
|
||||
const argumentsPreviewChars = config.argumentsPreviewChars as number
|
||||
if (!Number.isInteger(argumentsPreviewChars) || argumentsPreviewChars < 1) {
|
||||
throw new Error(`repeat-tool-guard: invalid argumentsPreviewChars ${argumentsPreviewChars} — must be an integer >= 1`)
|
||||
}
|
||||
|
||||
const chains = new Map<AgentId, Chain>()
|
||||
|
||||
/** Whether a tool participates in the chain (untracked calls are transparent: they neither count nor reset). */
|
||||
function tracked(toolName: string): boolean {
|
||||
if (includePatterns.length > 0 && !includePatterns.some(pattern => pattern.test(toolName))) return false
|
||||
return !excludePatterns.some(pattern => pattern.test(toolName))
|
||||
}
|
||||
|
||||
/**
|
||||
* Advance the calling agent's chain for one attempt and return the reminder
|
||||
* to deliver, if this attempt's run length hits a configured threshold.
|
||||
* Counting happens here — in post-execute — because denied calls also flow
|
||||
* through this waterfall (`ToolRegistry.execute` routes a deny through the
|
||||
* same pipeline), and a model hammering a denied call is exactly the loop
|
||||
* worth breaking.
|
||||
*/
|
||||
function observe(exec: ToolExecution): HookContext | undefined {
|
||||
// A direct `ctx.tools.execute()` caller has no model to remind and no id
|
||||
// to key on; only agent-loop calls participate.
|
||||
if (!exec.agent) return undefined
|
||||
if (!tracked(exec.name)) return undefined
|
||||
const canonical = canonicalize(exec.arguments)
|
||||
const key = JSON.stringify([exec.name, canonical])
|
||||
const chain = chains.get(exec.agent.id)
|
||||
const count = chain !== undefined && chain.key === key ? chain.count + 1 : 1
|
||||
chains.set(exec.agent.id, { key, count })
|
||||
if (!thresholdSet.has(count)) return undefined
|
||||
const text = count === thresholds[0]
|
||||
? GENTLE_REMINDER
|
||||
: detailedReminder(exec.name, count, previewArguments(canonical, argumentsPreviewChars))
|
||||
return { content: [{ type: 'text', text }], source: PLUGIN_SOURCE }
|
||||
}
|
||||
|
||||
// Observe-and-enrich, never veto: count first (state advances regardless of
|
||||
// the downstream outcome), DELEGATE so a later listener can still block or
|
||||
// replace, then fold the reminder onto whatever came back — additionalContext
|
||||
// rides both decision variants, so a blocked call still gets the nudge.
|
||||
ctx.on('tools/post-execute', async (exec, _result, next): Promise<PostToolDecision> => {
|
||||
const reminder = observe(exec)
|
||||
const downstream = await next()
|
||||
if (!reminder) return downstream
|
||||
if (downstream.kind === 'block') {
|
||||
return { kind: 'block', feedback: downstream.feedback, additionalContext: concatContext(reminder, downstream.additionalContext) }
|
||||
}
|
||||
return {
|
||||
kind: 'accept',
|
||||
...downstream.content !== undefined ? { content: downstream.content } : {},
|
||||
additionalContext: concatContext(reminder, downstream.additionalContext),
|
||||
}
|
||||
})
|
||||
|
||||
// A user interjection changes the context; repetition across it is not a
|
||||
// loop. Pure reset hook: always delegates (attaching nothing, vetoing
|
||||
// nothing).
|
||||
ctx.on('agent/prompt-submit', (agent, _content, _source, next): Promise<PromptDecision> => {
|
||||
chains.delete(agent.id)
|
||||
return next()
|
||||
})
|
||||
|
||||
// Drop state when an agent goes away, bounding the map over harness lifetime.
|
||||
ctx.on('agent/status', (agent, status) => {
|
||||
if (status === 'disposed') chains.delete(agent.id)
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,401 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { type SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop, { type ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop'
|
||||
import * as RepeatToolGuard from '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
import type { Config } from '@deepseek-ai/dsh-repeat-tool-guard'
|
||||
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
|
||||
|
||||
/**
|
||||
* Behavior suite for the repeat-tool-call guard: chain semantics (identical /
|
||||
* different-tracked / untracked-transparent / per-agent / resets), threshold
|
||||
* escalation incl. the `thresholds[0]` gentle-text rule, canonicalization,
|
||||
* fold-onto-downstream-decision, and fail-loud config validation — all driven
|
||||
* through a real agent loop against a scripted mock adapter (no network).
|
||||
*/
|
||||
|
||||
/** Boot the core spine + the guard; the caller registers adapters and extra listeners. */
|
||||
async function harness(config: Config = {}): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(RepeatToolGuard, config)
|
||||
ctx.tools.register(defineTool({ name: 'probe', description: 'p', parameters: {}, async execute() { return [{ type: 'text', text: 'ok' }] } }))
|
||||
ctx.tools.register(defineTool({ name: 'other', description: 'o', parameters: {}, async execute() { return [{ type: 'text', text: 'ok' }] } }))
|
||||
return ctx
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise<void> {
|
||||
return new Promise((resolve) => { const d = ctx.on('agent/status', (s, st) => { if (s === agent && st === 'idle') { d(); resolve() } }) })
|
||||
}
|
||||
|
||||
/** Every `context/message` in the agent's log, flattened to joined text + source for terse assertions. */
|
||||
function reminders(agent: ReactLoopAgent): { text: string; source: unknown }[] {
|
||||
return [...agent.session.events]
|
||||
.filter((e): e is SessionEvent<'context/message'> => e.type === 'context/message')
|
||||
.map(e => ({
|
||||
text: e.data.content.map(block => block.type === 'text' ? block.text : '').join('|'),
|
||||
source: e.data.source,
|
||||
}))
|
||||
}
|
||||
|
||||
const GUARD_SOURCE = { kind: 'plugin', plugin: 'repeat-tool-guard' }
|
||||
|
||||
describe('threshold escalation', () => {
|
||||
it('reminds gently at the first default threshold (3) and in detail at the second (5)', async () => {
|
||||
const ctx = await harness()
|
||||
const adapter = new MockAdapter([
|
||||
...Array.from({ length: 5 }, (_, i) => toolCallResponse(`c${i}`, 'probe', { q: 'same' })),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(2)
|
||||
expect(found[0]!.text).toContain('repeating the exact same tool call')
|
||||
expect(found[0]!.source).toEqual(GUARD_SOURCE)
|
||||
expect(found[1]!.text).toContain('consecutive_calls: 5')
|
||||
expect(found[1]!.text).toContain('- tool: probe')
|
||||
expect(found[1]!.text).toContain('{"q":"same"}')
|
||||
expect(found[1]!.source).toEqual(GUARD_SOURCE)
|
||||
})
|
||||
|
||||
it('keys the gentle text to thresholds[0], not the literal 3', async () => {
|
||||
const ctx = await harness({ thresholds: [4, 2] }) // unsorted on purpose: normalized ascending
|
||||
const adapter = new MockAdapter([
|
||||
...Array.from({ length: 4 }, (_, i) => toolCallResponse(`c${i}`, 'probe', {})),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(2)
|
||||
expect(found[0]!.text).toContain('repeating the exact same tool call') // gentle at 2
|
||||
expect(found[1]!.text).toContain('consecutive_calls: 4') // detailed at 4
|
||||
})
|
||||
})
|
||||
|
||||
describe('chain semantics', () => {
|
||||
it('caps the detailed reminder arguments at argumentsPreviewChars (detection still keys on the full string)', async () => {
|
||||
const ctx = await harness({ thresholds: [2, 3], argumentsPreviewChars: 24 })
|
||||
const bigPayload = 'x'.repeat(400)
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { body: bigPayload }),
|
||||
toolCallResponse('c2', 'probe', { body: bigPayload }),
|
||||
toolCallResponse('c3', 'probe', { body: bigPayload }),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(2) // gentle at 2, detailed at 3 — full-key matching survived the cap
|
||||
const detailed = found[1]!.text
|
||||
expect(detailed).toContain('- arguments: {"body":"xxxxxxxxxxxxxx') // 24-char head
|
||||
expect(detailed).toContain('… (+387 more chars)')
|
||||
expect(detailed).not.toContain(bigPayload)
|
||||
})
|
||||
|
||||
it('a different tracked call resets the chain', async () => {
|
||||
const ctx = await harness()
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
toolCallResponse('c2', 'probe', { q: 1 }),
|
||||
toolCallResponse('c3', 'other', {}), // tracked, different → reset
|
||||
toolCallResponse('c4', 'probe', { q: 1 }),
|
||||
toolCallResponse('c5', 'probe', { q: 1 }),
|
||||
toolCallResponse('c6', 'probe', { q: 1 }), // 3rd consecutive AFTER the reset
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reminders(agent)).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('excluded calls are transparent: they neither count nor reset', async () => {
|
||||
const ctx = await harness({ exclude: ['other'] })
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
toolCallResponse('c2', 'other', {}), // excluded → invisible to the chain
|
||||
toolCallResponse('c3', 'probe', { q: 1 }),
|
||||
toolCallResponse('c4', 'other', {}),
|
||||
toolCallResponse('c5', 'probe', { q: 1 }), // 3rd consecutive probe
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(1)
|
||||
expect(found[0]!.text).toContain('repeating the exact same tool call')
|
||||
})
|
||||
|
||||
it('include patterns track only matching tools (wildcard star)', async () => {
|
||||
const ctx = await harness({ include: ['pro*'] })
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'other', {}),
|
||||
toolCallResponse('c2', 'other', {}),
|
||||
toolCallResponse('c3', 'other', {}), // 3 identical, but untracked
|
||||
toolCallResponse('c4', 'probe', {}),
|
||||
toolCallResponse('c5', 'probe', {}),
|
||||
toolCallResponse('c6', 'probe', {}), // 3 identical, tracked
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(1)
|
||||
expect(found[0]!.text).toContain('repeating the exact same tool call')
|
||||
})
|
||||
|
||||
it('escapes regex metacharacters in patterns (a dot matches only a literal dot)', async () => {
|
||||
const ctx = await harness({ exclude: ['pr.be'] }) // would match 'probe' as a regex; must not as a wildcard
|
||||
const adapter = new MockAdapter([
|
||||
...Array.from({ length: 3 }, (_, i) => toolCallResponse(`c${i}`, 'probe', {})),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reminders(agent)).toHaveLength(1) // probe was NOT excluded
|
||||
})
|
||||
|
||||
it('canonicalization ignores property order, deeply', async () => {
|
||||
const ctx = await harness()
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { a: 1, nested: { x: [1, 2], y: null } }),
|
||||
toolCallResponse('c2', 'probe', { nested: { y: null, x: [1, 2] }, a: 1 }),
|
||||
toolCallResponse('c3', 'probe', { a: 1, nested: { x: [1, 2], y: null } }),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reminders(agent)).toHaveLength(1) // all three canonicalize identically
|
||||
})
|
||||
|
||||
it('keys chains per agent: one agent repeating never trips another', async () => {
|
||||
const ctx = await harness()
|
||||
ctx.llm.registerAdapter(['mock-a'], new MockAdapter([
|
||||
toolCallResponse('a1', 'probe', { q: 1 }),
|
||||
toolCallResponse('a2', 'probe', { q: 1 }),
|
||||
textResponse('done'),
|
||||
]))
|
||||
ctx.llm.registerAdapter(['mock-b'], new MockAdapter([
|
||||
toolCallResponse('b1', 'probe', { q: 1 }),
|
||||
toolCallResponse('b2', 'probe', { q: 1 }),
|
||||
toolCallResponse('b3', 'probe', { q: 1 }),
|
||||
textResponse('done'),
|
||||
]))
|
||||
const agentA = ctx.agentLoop.create(AgentId('a'), { model: 'mock-a' })
|
||||
const agentB = ctx.agentLoop.create(AgentId('b'), { model: 'mock-b' })
|
||||
agentA.send([{ type: 'text', text: 'go' }])
|
||||
agentB.send([{ type: 'text', text: 'go' }])
|
||||
await Promise.all([waitForIdle(ctx, agentA), waitForIdle(ctx, agentB)])
|
||||
|
||||
expect(reminders(agentA)).toHaveLength(0) // 2 repeats < 3, despite B's 3 in the same registry
|
||||
expect(reminders(agentB)).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('a new user prompt resets the chain', async () => {
|
||||
const ctx = await harness()
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
toolCallResponse('c2', 'probe', { q: 1 }),
|
||||
textResponse('turn one done'),
|
||||
toolCallResponse('c3', 'probe', { q: 1 }), // without the reset this would be the 3rd
|
||||
textResponse('turn two done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
agent.send([{ type: 'text', text: 'again' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reminders(agent)).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('drops an agent chain on disposal', async () => {
|
||||
const ctx = await harness({ thresholds: [2] })
|
||||
ctx.llm.registerAdapter(['mock'], new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
textResponse('done'),
|
||||
toolCallResponse('c2', 'probe', { q: 1 }), // same id, fresh agent: count 1, not 2
|
||||
textResponse('done'),
|
||||
]))
|
||||
// Loop agents are torn down by disposing the scope that created them
|
||||
// (the loop.spec pattern): a child plugin fiber owns `first`.
|
||||
let first!: ReactLoopAgent
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.agentLoop.create(AgentId('reused'), { model: 'mock' })
|
||||
}, { inject: ['agentLoop'] }))
|
||||
first.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, first)
|
||||
await fiber.dispose()
|
||||
await first.done
|
||||
|
||||
const second = ctx.agentLoop.create(AgentId('reused'), { model: 'mock' })
|
||||
second.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, second)
|
||||
|
||||
expect(reminders(second)).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('counts denied calls: hammering a denied tool still draws the reminder', async () => {
|
||||
const ctx = await harness({ thresholds: [2] })
|
||||
ctx.on('tools/pre-execute', async () => ({ kind: 'deny' as const, reason: 'sealed' }))
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
toolCallResponse('c2', 'probe', { q: 1 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reminders(agent)).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('ignores direct executes with no agent (they neither crash nor advance any chain)', async () => {
|
||||
const ctx = await harness({ thresholds: [2] })
|
||||
const direct = await ctx.tools.execute({ callId: CallId('d1'), name: 'probe', arguments: { q: 1 } })
|
||||
expect(direct.isError).toBe(false)
|
||||
|
||||
ctx.llm.registerAdapter(['mock'], new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }), // if the direct call had counted, this would be #2
|
||||
textResponse('done'),
|
||||
]))
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reminders(agent)).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('fold onto the downstream decision', () => {
|
||||
it('folds the reminder onto a downstream block and keeps its feedback', async () => {
|
||||
const ctx = await harness({ thresholds: [2] })
|
||||
ctx.on('tools/post-execute', async () => ({
|
||||
kind: 'block' as const,
|
||||
feedback: [{ type: 'text' as const, text: 'nope' }],
|
||||
additionalContext: { content: [{ type: 'text' as const, text: 'downstream-ctx' }], source: { kind: 'plugin' as const, plugin: 'test' } },
|
||||
}))
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
toolCallResponse('c2', 'probe', { q: 1 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(2)
|
||||
// Call 1: below threshold — the downstream context passes through untouched.
|
||||
expect(found[0]!.text).toBe('downstream-ctx')
|
||||
expect(found[0]!.source).toEqual({ kind: 'plugin', plugin: 'test' })
|
||||
// Call 2: reminder folded in front, single merged context, the guard's source.
|
||||
expect(found[1]!.text).toContain('repeating the exact same tool call')
|
||||
expect(found[1]!.text).toContain('|downstream-ctx')
|
||||
expect(found[1]!.source).toEqual(GUARD_SOURCE)
|
||||
// The block's feedback reached the tool result unchanged.
|
||||
const results = [...agent.session.events].filter((e): e is SessionEvent<'tool/result'> => e.type === 'tool/result')
|
||||
expect(results.every(r => r.data.isError)).toBe(true)
|
||||
expect(results[1]!.data.content).toEqual([{ type: 'text', text: 'nope' }])
|
||||
})
|
||||
|
||||
it('preserves a downstream accept content replacement while folding', async () => {
|
||||
const ctx = await harness({ thresholds: [2] })
|
||||
ctx.on('tools/post-execute', async () => ({
|
||||
kind: 'accept' as const,
|
||||
content: [{ type: 'text' as const, text: 'replaced' }],
|
||||
}))
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'probe', { q: 1 }),
|
||||
toolCallResponse('c2', 'probe', { q: 1 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const found = reminders(agent)
|
||||
expect(found).toHaveLength(1)
|
||||
expect(found[0]!.text).toContain('repeating the exact same tool call')
|
||||
const results = [...agent.session.events].filter((e): e is SessionEvent<'tool/result'> => e.type === 'tool/result')
|
||||
expect(results[1]!.data.content).toEqual([{ type: 'text', text: 'replaced' }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('config validation fails loud', () => {
|
||||
async function spine(): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
return ctx
|
||||
}
|
||||
|
||||
it('rejects an empty thresholds list', async () => {
|
||||
const ctx = await spine()
|
||||
await expect(ctx.plugin(RepeatToolGuard, { thresholds: [] })).rejects.toThrow(/must not be empty/)
|
||||
})
|
||||
|
||||
it('rejects a threshold below 2', async () => {
|
||||
const ctx = await spine()
|
||||
await expect(ctx.plugin(RepeatToolGuard, { thresholds: [1, 3] })).rejects.toThrow(/integer >= 2/)
|
||||
})
|
||||
|
||||
it('rejects a non-integer threshold', async () => {
|
||||
const ctx = await spine()
|
||||
await expect(ctx.plugin(RepeatToolGuard, { thresholds: [2.5] })).rejects.toThrow(/integer >= 2/)
|
||||
})
|
||||
|
||||
it('rejects duplicate thresholds', async () => {
|
||||
const ctx = await spine()
|
||||
await expect(ctx.plugin(RepeatToolGuard, { thresholds: [3, 3] })).rejects.toThrow(/duplicates/)
|
||||
})
|
||||
|
||||
it('rejects a non-positive or fractional argumentsPreviewChars', async () => {
|
||||
const ctx = await spine()
|
||||
await expect(ctx.plugin(RepeatToolGuard, { argumentsPreviewChars: 0 })).rejects.toThrow(/argumentsPreviewChars/)
|
||||
const ctx2 = await spine()
|
||||
await expect(ctx2.plugin(RepeatToolGuard, { argumentsPreviewChars: 12.5 })).rejects.toThrow(/argumentsPreviewChars/)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/tools"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -52,12 +52,22 @@ export interface Scenario {
|
||||
/**
|
||||
* Whether `test:snapshot:record` regenerates this scenario's `session.jsonl`
|
||||
* from the LIVE API. `recorded` scenarios are model-driven and reproducible;
|
||||
* `authored` scenarios (a hand-written `replay.override.json` sidecar drives
|
||||
* replay — e.g. a provider error or a cancel, which the live API can't be
|
||||
* coaxed into deterministically — or a deterministic hook scenario whose
|
||||
* derived empty script needs no sidecar) are NEVER re-recorded.
|
||||
* `authored` scenarios (fixtures hand-written or hand-harvested — e.g. a
|
||||
* provider error or a cancel the live API can't be coaxed into
|
||||
* deterministically, a deterministic hook scenario, or a scripted repetition
|
||||
* a live model won't reproduce) are NEVER re-recorded.
|
||||
*/
|
||||
recorded: boolean
|
||||
/**
|
||||
* Whether replay is driven by a hand-written `replay.override.json` sidecar
|
||||
* (a `ReplayEntry[]` that REPLACES the script derived from `session.jsonl`)
|
||||
* — the throw/hang cases chunks cannot express. The fixture guard requires
|
||||
* the sidecar exactly when this is set: the harness forwards the file purely
|
||||
* on existence, so an unregistered stray sidecar would silently replace the
|
||||
* derived script — the guard fails loud on either mismatch. Defaults to
|
||||
* false (replay derives from the fixture's `assistant/chunk` events).
|
||||
*/
|
||||
overridden?: boolean
|
||||
/**
|
||||
* How many SUBAGENT child sessions this scenario records beyond the top-level
|
||||
* one (0 for a single-session scenario). Each child rides in a sibling fixture
|
||||
@@ -317,17 +327,18 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
// throws "fixture not found" when it is absent and no override replaces it.
|
||||
// A no-model scenario ships a header-only `session.jsonl` (it derives to an
|
||||
// empty script — no model call is made); a model scenario's fixture also
|
||||
// doubles as the expected-log artifact the run is diffed against. An authored
|
||||
// (non-`recorded`) model scenario additionally ships a `replay.override.json`
|
||||
// sidecar for the throw/hang cases a derived script cannot express.
|
||||
for (const { name, hasModelTurn, recorded, childSessions } of scenarios) {
|
||||
// doubles as the expected-log artifact the run is diffed against. The
|
||||
// `replay.override.json` sidecar is matched BOTH ways against the table's
|
||||
// `overridden` flag: required when set, forbidden when not — the harness
|
||||
// forwards the file purely on existence, so an unregistered stray sidecar
|
||||
// would silently replace the derived script.
|
||||
for (const { name, overridden, childSessions } of scenarios) {
|
||||
const dir = join(snapshotsDir, name)
|
||||
expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true)
|
||||
expect(existsSync(join(dir, 'stdout.golden.jsonl')), `${name}/stdout.golden.jsonl`).toBe(true)
|
||||
expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true)
|
||||
if (hasModelTurn && !recorded) {
|
||||
expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json`).toBe(true)
|
||||
}
|
||||
expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json presence must match \`overridden\``)
|
||||
.toBe(overridden === true)
|
||||
// A nested-agent scenario ships one child fixture per recorded subagent
|
||||
// session (`session.1.jsonl` …), the replay source for that child session.
|
||||
for (const childFixture of childFixturePaths(dir, childSessions ?? 0)) {
|
||||
|
||||
@@ -39,14 +39,14 @@ const REPLAY_SCENARIOS: Scenario[] = [
|
||||
{ name: 'plain-turn', hasModelTurn: true, recorded: true, childSessions: 1 },
|
||||
{ name: 'no-model', hasModelTurn: false, recorded: false },
|
||||
{ name: 'blocked-log', hasModelTurn: false, comparesLog: true, recorded: false },
|
||||
{ name: 'authored-error', hasModelTurn: true, recorded: false },
|
||||
{ name: 'authored-error', hasModelTurn: true, recorded: false, overridden: true },
|
||||
]
|
||||
|
||||
const RECORD_SCENARIOS: Scenario[] = [
|
||||
{ name: 'rec-pin', hasModelTurn: true, recorded: true, pinsHeader: true },
|
||||
{ name: 'rec-child', hasModelTurn: true, recorded: true, childSessions: 1 },
|
||||
// recorded:false in record mode → registered but skipped (never re-recorded).
|
||||
{ name: 'rec-skip', hasModelTurn: true, recorded: false },
|
||||
{ name: 'rec-skip', hasModelTurn: true, recorded: false, overridden: true },
|
||||
]
|
||||
|
||||
// Record mode mutates its snapshots dir, so run it on a throwaway copy —
|
||||
|
||||
Generated
+41
@@ -136,6 +136,19 @@ importers:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/code-runtime/code-runtime-worker:
|
||||
dependencies:
|
||||
schemastery:
|
||||
specifier: ^3.18.0
|
||||
version: 3.18.0
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-code-runtime':
|
||||
specifier: workspace:^
|
||||
version: link:../code-runtime
|
||||
cordis:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/compact/compact:
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-llm':
|
||||
@@ -390,6 +403,34 @@ importers:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/guard/repeat-tool-guard:
|
||||
dependencies:
|
||||
schemastery:
|
||||
specifier: ^3.18.0
|
||||
version: 3.18.0
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-agent':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/agent
|
||||
'@deepseek-ai/dsh-agent-loop':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/agent-loop
|
||||
'@deepseek-ai/dsh-llm':
|
||||
specifier: workspace:^
|
||||
version: link:../../llm/llm
|
||||
'@deepseek-ai/dsh-session':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/session
|
||||
'@deepseek-ai/dsh-system-prompt':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/system-prompt
|
||||
'@deepseek-ai/dsh-tools':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/tools
|
||||
cordis:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/hooks/hook-protocol:
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-bash':
|
||||
|
||||
@@ -105,11 +105,26 @@ const dshBinPackageFiles = [
|
||||
'src',
|
||||
] as const
|
||||
|
||||
// Packages that ship a worker-thread entry as a sibling runtime bundle
|
||||
// (lib/worker.js, its own tsdown entry): the bootstrap is loaded via
|
||||
// `new Worker(new URL('./worker.js', import.meta.url))`, so it cannot live
|
||||
// inside the index bundle and must be published alongside it.
|
||||
const workerEntryPackages = new Set(['@deepseek-ai/dsh-code-runtime-worker'])
|
||||
|
||||
const dshWorkerPackageFiles = [
|
||||
'lib/index.js',
|
||||
'lib/worker.js',
|
||||
'lib/types/**/*.d.ts',
|
||||
'lib/types/**/*.d.ts.map',
|
||||
'src',
|
||||
] as const
|
||||
|
||||
function sameStringList(actual: readonly string[] | undefined, expected: readonly string[]): boolean {
|
||||
return !!actual && actual.length === expected.length && actual.every((value, index) => value === expected[index])
|
||||
}
|
||||
|
||||
function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] {
|
||||
if (manifest.name && workerEntryPackages.has(manifest.name)) return dshWorkerPackageFiles
|
||||
return manifest.bin ? dshBinPackageFiles : dshPackageFiles
|
||||
}
|
||||
|
||||
|
||||
@@ -154,7 +154,7 @@ const SERVICE_ROLES: ServiceRole[] = [
|
||||
pkg: 'code-runtime',
|
||||
title: 'Code-execution seam',
|
||||
mode: 'seam',
|
||||
implementations: [],
|
||||
implementations: ['code-runtime-worker'],
|
||||
consumers: [],
|
||||
note: 'Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the Code Mode RFC specifies the worker-thread backend and the tool-registry consumer).',
|
||||
},
|
||||
|
||||
@@ -310,6 +310,7 @@ function builtBinSmokeGate(): Gate {
|
||||
'vitest.e2e.config.ts',
|
||||
'packages/ui/stdio-agent/tests/built-bin.e2e.ts',
|
||||
'packages/ui/acp-agent/tests/built-bin.e2e.ts',
|
||||
'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts',
|
||||
], {
|
||||
label: 'built-bin smoke',
|
||||
needs: ['build'],
|
||||
|
||||
@@ -46,6 +46,7 @@
|
||||
"./packages/code-runtime/*/src",
|
||||
"./packages/fs/*/src",
|
||||
"./packages/compact/*/src",
|
||||
"./packages/guard/*/src",
|
||||
"./packages/subagent/*/src",
|
||||
"./packages/web/*/src",
|
||||
"./packages/timeout/*/src",
|
||||
|
||||
@@ -24,6 +24,7 @@
|
||||
{ "path": "./packages/core/agent-core" },
|
||||
{ "path": "./packages/bash/bash" },
|
||||
{ "path": "./packages/code-runtime/code-runtime" },
|
||||
{ "path": "./packages/code-runtime/code-runtime-worker" },
|
||||
{ "path": "./packages/compact/compact" },
|
||||
{ "path": "./packages/compact/compact-basic" },
|
||||
{ "path": "./packages/llm/llm-deepseek" },
|
||||
@@ -56,6 +57,7 @@
|
||||
{ "path": "./packages/subagent/subagent-fork" },
|
||||
{ "path": "./packages/subagent/subagent-acp" },
|
||||
{ "path": "./packages/todo/tool-todo" },
|
||||
{ "path": "./packages/guard/repeat-tool-guard" },
|
||||
{ "path": "./packages/hooks/hook-protocol" },
|
||||
{ "path": "./packages/hooks/hooks-claude" },
|
||||
{ "path": "./packages/hooks/hooks-codex" }
|
||||
|
||||
@@ -35,6 +35,7 @@
|
||||
{ "path": "./packages/core/agent-core" },
|
||||
{ "path": "./packages/bash/bash" },
|
||||
{ "path": "./packages/code-runtime/code-runtime" },
|
||||
{ "path": "./packages/code-runtime/code-runtime-worker" },
|
||||
{ "path": "./packages/llm/llm-deepseek" },
|
||||
{ "path": "./packages/llm/llm-pi-ai" },
|
||||
{ "path": "./packages/bash/bash-local" },
|
||||
@@ -67,6 +68,7 @@
|
||||
{ "path": "./packages/subagent/subagent-fork" },
|
||||
{ "path": "./packages/subagent/subagent-acp" },
|
||||
{ "path": "./packages/todo/tool-todo" },
|
||||
{ "path": "./packages/guard/repeat-tool-guard" },
|
||||
{ "path": "./packages/hooks/hook-protocol" },
|
||||
{ "path": "./packages/hooks/hooks-claude" },
|
||||
{ "path": "./packages/hooks/hooks-codex" }
|
||||
|
||||
+6
-1
@@ -31,7 +31,12 @@ export default defineConfig({
|
||||
// can't import one without booting it, so they are driven by the keyless
|
||||
// Loader-path smoke (a real subprocess) instead of the in-process unit
|
||||
// suite — the same reason `examples/start.ts` sat out of coverage scope.
|
||||
exclude: ['packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts'],
|
||||
// `worker.ts` files are the same class as bin.ts: self-executing
|
||||
// worker-thread entrypoints that only ever run inside a spawned isolate
|
||||
// the v8 provider cannot observe. They stay thin glue over in-process-
|
||||
// tested logic (bootstrap.ts) and are pinned by real-worker integration
|
||||
// tests.
|
||||
exclude: ['packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts', 'packages/*/*/src/worker.ts'],
|
||||
// 100% or it doesn't merge (docs/testing.md: excessive tests are welcome).
|
||||
// Per-file so a well-covered big file can't subsidize a bare one.
|
||||
// Every v8 ignore comment must carry a reason — see the quality-gates RFC
|
||||
|
||||
Reference in New Issue
Block a user