diff --git a/.agents/skills/dsh-pre-push-checks/SKILL.md b/.agents/skills/dsh-pre-push-checks/SKILL.md new file mode 100644 index 0000000000..7c0d629fff --- /dev/null +++ b/.agents/skills/dsh-pre-push-checks/SKILL.md @@ -0,0 +1,103 @@ +--- +name: dsh-pre-push-checks +description: Use before pushing, force-pushing, marking ready for review, claiming checks pass, or bypassing a local hook on a deepseek-harness branch, especially after merges, review fixes, package graph changes, docs/catalog updates, snapshots, e2e behavior, or built artifact changes. +--- + +# DSH Pre-Push Checks + +Use this skill to choose and run the smallest sufficient verification set before a `deepseek-harness` push. Do not treat the local pre-push hook as the full CI contract: CI also runs coverage, build, demo smoke, and built-bin smoke. + +## First Steps + +1. Confirm the checkout and branch. + +```sh +git status --short --branch +git rev-parse --show-toplevel +``` + +2. Inspect the outgoing diff. + +```sh +git diff --stat +git diff --name-only origin/$(git branch --show-current)...HEAD +``` + +If the branch has no upstream or the command is not meaningful for the stack shape, use `git diff --name-only origin/master...HEAD` or the PR base branch. + +3. If the branch was just merged with `master`, or the user says master changed, run the gates after resolving the merge and before pushing or marking ready. Do not present a conflict-resolution commit as ready with only typecheck/lint evidence. + +## Required Baseline + +Run these before every non-trivial push: + +```sh +pnpm run typecheck +pnpm run lint +pnpm run test:coverage +``` + +Why `test:coverage`, not only `test`: CI enforces per-file 100% coverage. A branch can pass `pnpm run test` and still fail CI. + +## Add Gates By Touched Surface + +Run `pnpm run doc-sync` and `pnpm run verify-module-graph` when the diff touches Markdown docs, package manifests, package imports/exports, generated catalogs, RFCs, architecture docs, translation pairs, Mermaid diagrams, or comments that cite docs/packages. + +Run `pnpm run build` and `pnpm run hygiene` when the diff touches any package `package.json`, dependency graph, public exports, build config, declaration surface, bundled runtime path, or code that will be consumed from built `lib/`. + +Run snapshot tests when the diff changes ACP/editor-facing transcript behavior: ACP bridge updates, agent-loop observable output, tool call/result presentation, session log rendering, stdout/stderr protocol output, or snapshot fixtures. + +```sh +pnpm run test:snapshot +``` + +Run built-bin smoke tests after `pnpm run build` when app packages, app boot, package runtime imports, bin entries, loader behavior, or published artifact paths change. + +```sh +pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts +``` + +Run real e2e when behavior depends on a real model/API, tool-use loop, ACP integration, prompt injection, or end-to-end agent UX. If `.env` is available, use it; do not print secrets. + +```sh +pnpm run test:e2e +``` + +Run a targeted test first for the changed package, but never use targeted tests as the only push evidence unless the change is test-only and cannot affect shared behavior. + +## Full Local CI Approximation + +Use this before high-risk pushes, after large merges, before asking for review on a major PR, or when prior pushes have caused CI churn. The authoritative command list is the root [AGENTS.md § Run the CI gates locally before marking a PR ready](../../../AGENTS.md#run-the-ci-gates-locally-before-marking-a-pr-ready); run that block rather than copying a local variant into this skill. Add `pnpm run test:e2e` when a key is available and the feature has real-agent behavior. + +## Handling Failures + +If a gate fails, stop and fix or explain the blocker. Do not push and hope CI differs. + +If a failure looks environment-specific, prove it: + +- Record the exact command, failing test, and platform-specific mismatch. +- Confirm the relevant non-platform gates pass. +- Prefer fixing the test for cross-platform determinism if the test is part of the required local gate. +- Bypass a local hook only when the user explicitly asks to push or agrees, and state exactly which hook failed and why it is not expected to fail on CI. + +Known pattern to watch for: Linux CI and macOS local behavior can differ for shell utilities such as `sed -i`. Treat this as evidence to investigate, not as automatic permission to bypass. + +## Push Procedure + +1. Local commits may happen before the full gate set, but do not push, mark ready, or claim checks pass until the relevant gates pass or any blocker is explicitly documented. +2. Let the normal pre-commit hook run. If it changes files, inspect and commit or amend the change intentionally rather than hiding it. +3. Push normally first so the pre-push hook can run. +4. If a local hook is bypassed after user approval, use the narrow bypass and say so in the final response. +5. After push, verify the remote ref matches local HEAD. + +```sh +git rev-parse HEAD origin/$(git branch --show-current) +``` + +For GitHub PRs, check CI after push: + +```sh +gh pr checks +``` + +If checks are pending, say pending. If checks fail, inspect logs before claiming the push is good. diff --git a/.agents/skills/dsh-pre-push-checks/agents/openai.yaml b/.agents/skills/dsh-pre-push-checks/agents/openai.yaml new file mode 100644 index 0000000000..6ad9b63935 --- /dev/null +++ b/.agents/skills/dsh-pre-push-checks/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "DSH Pre-Push Checks" + short_description: "Run the right DeepSeek Harness gates before push" + default_prompt: "Use $dsh-pre-push-checks before pushing this DeepSeek Harness branch." diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index efa4dfc458..acb74c0c26 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -95,13 +95,13 @@ jobs: node-compat: runs-on: ubuntu-latest - name: node 26 + name: node ${{ matrix.node }} env: DSH_GATE_CONCURRENCY: '2' strategy: fail-fast: false matrix: - node: [26] + node: ['22.19', 24, 26] steps: - uses: actions/checkout@v6 @@ -128,3 +128,25 @@ jobs: - name: Run compatibility gates run: pnpm run check:node-compat + + # Single stable required check for branch protection: require "all checks + # passed" instead of enumerating matrix legs whose names change as lanes and + # node versions evolve. Every other job in THIS workflow must be listed in + # `needs` (`needs` cannot reach across workflow files; e2e.yml stays its own + # check). `if: always()` is load-bearing: without it a failed dependency + # would SKIP this job, and GitHub counts a skipped required check as passing + # — so this job always runs and fails on any non-success result, including + # 'cancelled' and 'skipped'. + all-checks-passed: + name: all checks passed + runs-on: ubuntu-latest + needs: [node-24, node-compat] + if: always() + steps: + - name: Fail if any needed job did not succeed + if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'skipped') + run: | + echo "::error::Needed job results: ${{ join(needs.*.result, ', ') }}" + exit 1 + - name: All checks passed + run: echo "All needed jobs succeeded (${{ join(needs.*.result, ', ') }})" diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml index 1ae0733286..c0371947fd 100644 --- a/.github/workflows/e2e.yml +++ b/.github/workflows/e2e.yml @@ -49,6 +49,7 @@ permissions: jobs: e2e: runs-on: ubuntu-latest + name: e2e # Run on every trusted event. Skip untrusted PRs (forks + Dependabot) where # the secret is withheld — they would otherwise hard-fail the preflight. if: >- diff --git a/AGENTS.md b/AGENTS.md index e0c044e824..1ef6af2ba0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # AGENTS.md -This is the monorepo of the DeepSeek Harness group; it hosts **DeepSeek Harness SDK**, a plugin-based SDK for building agent harnesses. The codebase is built on the vendored Cordis framework, microkernel-style: **everything is a plugin**. Read [docs/architecture.md](docs/architecture.md) before changing `packages/`; the documentation standard is [docs/AGENTS.md](docs/AGENTS.md). +This is the DeepSeek Harness group's monorepo; it hosts **DeepSeek Harness SDK**, a plugin-based SDK for building agent harnesses. The codebase is built on the vendored Cordis framework, microkernel-style: **everything is a plugin**. Read [docs/architecture.md](docs/architecture.md) before changing `packages/`; the documentation standard is [docs/AGENTS.md](docs/AGENTS.md). ## Pre-release stance: foundation over blast radius @@ -19,11 +19,12 @@ packages/ Harness packages at packages///, all named @deepseek-ai compact/ compaction seam + basic backend subagent/ subagent seam + spawn/fork/ACP backends + delegation tool todo/ the todo_write tool + guard/ loop-hygiene plugins hooks/ Claude Code / Codex hook bridges + shared wire-protocol library session-persistence/ persistence seam + JSONL/SQLite backends ui/ ACP bridge + app-boot glue + the stdio/ACP app bins - support/ dev/test infrastructure: invariants, llm-replay, subagent-mock - util/ zero-dependency utilities (Branded) + support/ dev/test infrastructure packages + util/ zero-dependency utilities examples/ Runnable demos: thin cordis.yml leaves over the app packages (see examples/AGENTS.md) docs/ architecture, generated catalogs, RFCs, postmortems, cookbook (see docs/AGENTS.md) scripts/ repo gates and generators @@ -34,7 +35,7 @@ Per-package map: the group READMEs, indexed from [packages/README.md](packages/R ## Commands ```sh -pnpm install # pnpm workspaces, node >= 24 +pnpm install # pnpm workspaces, node ^22.19 || >=24 pnpm run test # vitest unit tests pnpm run test:coverage # THE gating test run: per-file 100% coverage on packages/*/*/src pnpm run test:e2e # real-API tests; self-skip without DEEPSEEK_API_KEY @@ -69,7 +70,7 @@ printf '%s\n' "$out" | grep -q '\[tool call\] echo({"text":"ci smoke"})' printf '%s\n' "$out" | grep -q '\[tool result\] ECHO: CI SMOKE' ls .sessions/_no-cwd/main-session-*.jsonl >/dev/null rm -rf .sessions -pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts +pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts ``` `test:coverage`, not `test`, is the gating run ([why](docs/testing.md)); a sign-off counts only for commands actually run. @@ -83,21 +84,22 @@ Real-API tests and demos read `DEEPSEEK_API_KEY` (and optional `DEEPSEEK_BASE_UR - Every npm package is `@deepseek-ai/dsh-`; vendored packages keep upstream names and are `private: true`. `cordis` is a peerDependency (+ dev) of every harness package. - ESM everywhere (`"type": "module"`). Cross-package imports use package names, never relative paths; in-package relative imports use explicit `.ts` extensions. Dev/test/demo run unbuilt via tsx + the root tsconfig `paths` map; builds are for outside consumers only. - **Registrations are effects**: every contribution goes through `ctx.effect()` / `ctx.on()`; a registry's `register()` returns the disposer. -- **Typed events via declaration merging**; extensible unions use the merge-extensible-map pattern (`ContentBlockMap`, `SessionEventMap`, …). Every new event's JSDoc carries an `@mode` tag and a `@param` per payload parameter (`this`/trailing `next` exempt); every public service-class method documents each parameter and non-void return (`@param`/`@returns`) — the catalog generator hard-errors otherwise ([completeness RFC](docs/rfc/implemented/process/2026-07-04-cordis-jsdoc-completeness-gate.md)); mode semantics are in the [generated events catalog](docs/cordis-catalog/events.md) header ([catalog RFC](docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md)). +- **Typed events via declaration merging**; extensible unions use the merge-extensible-map pattern (`ContentBlockMap`, `SessionEventMap`, …). Every new event's JSDoc carries an `@mode` tag and a `@param` per payload parameter (`this`/trailing `next` exempt); every public service-class method documents each parameter and non-void return (`@param`/`@returns`) — the catalog generator hard-errors otherwise; mode semantics are in the [generated events catalog](docs/cordis-catalog/events.md) header. - **Discriminated unions: `switch` on the tag**, not if-chains. Closed unions end with `default: assertNever(...)`; merge-extensible unions must NOT — handle known cases and fall through `default` with a comment. - **Waterfall listeners MUST call `next()`** to delegate; returning without it is the veto ([semantics](docs/cordis-primer.md#cordis-waterfall-semantics)). -- **Model-visible ⟺ logged**: anything that reaches a model request must be reconstructable from the session log; a new model-visible input requires a session event ([reconstructability RFC](docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). +- **Model-visible ⟺ logged**: anything that reaches a model request must be reconstructable from the session log; a new model-visible input requires a session event. - **Plugins, not loop changes**: new behavior goes on the documented extension seams; changing `agent-loop` requires updating docs/architecture.md. -- **Capability seams are three packages** — interface / implementation / consumer ([capability seams](docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)); don't split preemptively. +- **Capability seams are three packages** — interface / implementation / consumer; don't split preemptively. - **Explicit > implicit at package seams**: defaulting is an explicit `resolve(request): Spec` step in the owning implementation, never a hidden `?? default` inside `run()` (the `dsh-bash` request/spec split is the template). - **No hardcoded tunables in plugins**: anything two deployments could want different — timeouts, caps, model names, base URLs — is a defaulted, validated `Config` field, not a literal; a `DEFAULT_*` constant or test-only seam is not configurability. The test: changeable from `cordis.yml`, no code edit. Protocol/wire constants, external-spec values, security invariants stay hardcoded. -- **Opaque cross-boundary ids are branded** (`Branded` from `dsh-brand`), never bare `string` ([branded IDs](docs/rfc/implemented/architecture/2026-06-20-branded-ids.md)). +- **Misconfiguration fails loud**: a config value referencing something that does not exist (a `toolOrder` tool name, a plugin path) throws — at load when the check is self-contained, else at the earliest moment the referent exists (for `toolOrder`, every prompt assembly) — never a silent skip. +- **Opaque cross-boundary ids are branded** (`Branded` from `dsh-brand`), never bare `string`. - **An empty `catch` names what it swallows** and why nothing else can reach it; keep the `try` to one statement. - **Symmetry is usually more correct**: parallel values get parallel form; asymmetry smells of a missed extraction. -- **Tests document behavior, not golden truth**: a green test pins what the code DOES, not what it SHOULD do. Before preserving a behavior solely for its test, ask whether it is load-bearing; an artifact changes together with its test, with the why in the PR ([worked example](docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md)). -- **RFCs are proposals, not golden truth**: validate its premise against current code before implementing; friction is evidence of over-reach — amend on the way to `implemented/` ([worked example](docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md)). -- **Testing policy** — tiers, with-key generosity, real-over-mock, world-verification, real-load-path and published-bin guards: [docs/testing.md](docs/testing.md). A transcript/UX-affecting change needs a snapshot test, or a PR note why none applies. -- **A tool's ACP render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([render-intent RFC](docs/rfc/implemented/architecture/2026-07-02-tool-render-intent-union.md), [cookbook](docs/cookbook/adding-a-tool.md)). +- **Tests document behavior, not golden truth**: a green test pins what the code DOES, not what it SHOULD do. Before preserving a behavior solely for its test, ask whether it is load-bearing; an artifact changes together with its test, with the why in the PR. +- **RFCs are proposals, not golden truth**: validate its premise against current code before implementing; friction is evidence of over-reach — amend on the way to `implemented/`. +- **Testing policy** — [docs/testing.md](docs/testing.md). Transcript/UX changes need snapshots or a PR note. Snapshot fixtures must replay on macOS/Linux; avoid GNU/BSD-only commands (e.g. `sed -i`); fix fixtures, not normalizers. +- **A tool's ACP render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **A new capability seam, lifecycle shape, or transcript surface names its coverage at every tier (unit, e2e, snapshot) at plan time** and verifies the harness can express it — a gap is scheduled work, not a mid-build surprise. - **Merge PRs with merge commits** (`gh pr merge --merge`), never squash/rebase. **Never rewrite a pushed branch**; update a child by merging its parent down. **A review fix lands on the PR that introduced the issue, as a separate commit**, then merges down ([stacked-review guide](docs/cookbook/responding-to-pr-review-on-a-stack.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). @@ -109,13 +111,13 @@ Real-API tests and demos read `DEEPSEEK_API_KEY` (and optional `DEEPSEEK_BASE_UR ## Type safety and documentation -Everything compiles under `strict: true` with `noImplicitAny`; every remaining `any` carries a comment saying why a narrower type is infeasible. Every module has a module-level doc comment; every export (and non-obvious method) has a JSDoc explaining semantics — contracts, disposal, errors — not the name restated; internal helpers only where non-obvious; one-liners when one line suffices. Lean toward the stricter lint rule and the extra mechanical gate: encode invariants in checks (`verify-*` scripts), preferring a narrow justified escape hatch over a rule left off globally. Type gymnastics are acceptable inside core packages when they buy plugin-author DX (the `defineTool` schema DSL is the canonical example). +Everything compiles under `strict: true` with `noImplicitAny`; every remaining `any` carries a comment saying why a narrower type is infeasible. Every module has a module-level doc comment; every export (and non-obvious method) has a JSDoc explaining semantics — contracts, disposal, errors — not the name restated; internal helpers only where non-obvious; one-liners when one line suffices. The export half is mechanical: `verify-export-jsdoc` (in `doc-sync`) requires description prose on every package export plus `@param`/`@returns` (and an annotated return) on function-like ones. Heritage-declared members, plugin-protocol slots, and constructors are exempt — their docs' one home is the seam declaration, the framework protocol, and the class doc respectively. Lean toward the stricter lint rule and the extra mechanical gate: encode invariants in checks (`verify-*` scripts), preferring a narrow justified escape hatch over a rule left off globally. Type gymnastics are acceptable inside core packages when they buy plugin-author DX (the `defineTool` schema DSL is the canonical example). Docs are part of every change: code changes update their README and JSDoc in the SAME change; a bilingual-pair edit updates the counterpart and re-records ([i18n contract](docs/i18n/README.md)). The writing rules — document the current state never the history, one physical line per paragraph, one home per fact — and the word-budget gate live in [docs/AGENTS.md](docs/AGENTS.md). ## Editing these instructions -`AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it (root, `packages/`, `examples/`). Edit `AGENTS.md`, never the symlink. This file is budget-gated (`verify-doc-budgets`): condense first if it is possible without sacrificing clarity; truly needed additions may justify a ceiling raise. +`AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it (root, `packages/`, `examples/`). Edit `AGENTS.md`, never the symlink. Keep it self-contained: state each principle inline instead of citing RFCs (they stay discoverable via the RFC index); linking high-level docs — architecture, testing, cookbooks — is fine. This file is budget-gated (`verify-doc-budgets`): condense first if it is possible without sacrificing clarity; truly needed additions may justify a ceiling raise. ## Vendoring policy diff --git a/docs/AGENTS.md b/docs/AGENTS.md index fed058f60a..771e30dcc6 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -16,7 +16,7 @@ Every fact has exactly one home — the tier whose job it is — and every other | [postmortem/](postmortem/README.md) | Incident stories — the only tier where war-story narrative belongs | — | | [cookbook/](cookbook/adding-a-package.md) | Step-by-step how-tos with numbered verify steps | Design rationale (→ the RFC each guide links) | | Package README | The per-package contract: config, semantics, limitations, extension points | JSDoc restatement, generated-catalog restatement (event/tool tables), other packages' concerns | -| [development.md](development.md) | Human-facing setup and daily workflow; a bilingual pair under the [i18n contract](i18n/README.md) | Gate-by-gate enumerations that drift from `package.json` scripts | +| [development.md](development.md) | First-stop contributor onboarding: local setup, daily workflow, and CI shape at summary level; a bilingual pair under the [i18n contract](i18n/README.md) | Runtime/version rationale (→ RFCs), gate-by-gate enumerations that drift from `package.json` scripts | | Generated catalogs: [cordis events](cordis-catalog/events.md), [cordis services](cordis-catalog/services.md), [tool-catalog](tool-catalog.md), [config-catalog](config-catalog.md), [persistence-catalog](persistence-catalog.md), [module-graph.md](module-graph.md) | Exhaustive enumerations regenerated from source, freshness-gated | Hand edits of any kind | | Skills (`.agents/skills/`) | Workflows: how to carry out a recurring task against the contracts | The contracts themselves (→ docs) | diff --git a/docs/architecture.md b/docs/architecture.md index ff51688513..04144f3d28 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -26,6 +26,7 @@ The default distribution is a composition, not a hierarchy. `packages/core/` is |---|---|---| | `ctx.llm` | [`llm/`](../packages/llm/README.md) | adapter registry and streaming model calls | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | foreground/background command execution | +| `ctx.codeRuntime` | [`code-runtime/`](../packages/code-runtime/README.md) | model-written program execution | | `ctx.fs` | [`fs/`](../packages/fs/README.md) | filesystem provider primitives and policy events | | `ctx.web` | [`web/`](../packages/web/README.md) | search/fetch provider registries | | `ctx.compact` | [`compact/`](../packages/compact/README.md) | session-surface compaction | @@ -78,7 +79,7 @@ forever: 'assistant/message' each tool call: 'tool/call' - tools/pre-execute -> dispatch -> tools/post-execute + tools/pre-execute -> tools/execute -> tools/post-execute 'tool/result' append post-tool context and steering 'step/end' @@ -108,7 +109,7 @@ Every session event is turn-enclosed. Reloading a crashed session preserves the The session log is the source of truth. `deriveMessages()` projects session events into the `Message[]` sent to the model; raw `assistant/chunk` events stay in the log for replay and UI fidelity. Replay, fork, resume, transcript rendering, telemetry, and persistence all derive from the same event stream. -**Model-visible ⟺ logged**: the log reconstructs every conversation request byte-for-byte — messages by derivation at the `step/start` boundary, the header (system prompt, tools, model + sampling) by folding `request/header` events — asserted per request by the dev invariant ([reconstructability RFC](rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). +**Model-visible ⟺ logged**: the log reconstructs every request — messages at `step/start`, headers by folding `request/header` — and dev invariants assert this ([reconstructability RFC](rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). Durability is a plugin concern. Persistence backends buffer synchronous `session/event` notifications and the loop awaits a turn-end checkpoint before moving on. The `SessionPersistence` seam stores `SessionEvent` directly, with metadata in `SessionHeader`; JSONL and SQLite share one contract suite. @@ -143,5 +144,6 @@ New behavior should attach to a documented seam; changing the shipped loop requi | Intercept prompts, requests, tool use, or continuation | listen on the relevant `agent/*` or `tools/*` waterfall | | Add UI or editor integration | drive `ctx.agents` and render from `session/event` | | Add durable session state | add a `SessionEventMap` member and render/replay from the log | +| Fork a live session | use `ctx.sessions.fork(source, boundary?, childSessionId?)` | The [extension cookbook](cookbook/extension-cookbook.md) carries plugin skeletons and the feature-to-seam map; step-by-step guides cover [packages](cookbook/adding-a-package.md), [tools](cookbook/adding-a-tool.md), [LLM adapters](cookbook/adding-an-llm-adapter.md), and [vendored packages](cookbook/adding-a-vendored-package.md). diff --git a/docs/capability-seams.md b/docs/capability-seams.md index 414ee898d1..12ce8033b9 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -42,6 +42,9 @@ flowchart LR pkg_bash_local["bash-local"] pkg_hooks_claude["hooks-claude"] pkg_hooks_codex["hooks-codex"] + pkg_code_runtime["code-runtime"] + svc_codeRuntime["ctx.codeRuntime
Code-execution seam"] + pkg_code_runtime_worker["code-runtime-worker"] pkg_fs["fs"] svc_fs["ctx.fs
Filesystem provider seam"] pkg_fs_local["fs-local"] @@ -64,6 +67,8 @@ flowchart LR pkg_agent_loop --> svc_agentLoop pkg_bash --> svc_bash pkg_bash_local --> svc_bash + pkg_code_runtime --> svc_codeRuntime + pkg_code_runtime_worker --> svc_codeRuntime pkg_compact --> svc_compact pkg_compact_basic --> svc_compact pkg_fs --> svc_fs @@ -134,6 +139,7 @@ flowchart LR | `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/ui/acp), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`stdio-agent`](../packages/ui/stdio-agent), [`invariants`](../packages/support/invariants) | - | Owns live Agent handles and the create/resume factory seam. | | `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-core`](../packages/core/agent-core) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. | | `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local) | [`tool-bash`](../packages/bash/tool-bash), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors can replace bash-local. | +| `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | - | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the Code Mode RFC specifies the worker-thread backend and the tool-registry consumer). | | `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-policy contributes observed-state checks through the fs/* event gate. | | `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend currently consumes the pre-step event directly; a model-facing compact tool remains deferred. | | `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-mock`](../packages/support/subagent-mock) | [`tool-subagent`](../packages/subagent/tool-subagent) | - | Providers implement transports; tool-subagent exposes one configured provider as a model-facing tool name. | diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 66766144a8..2429fe3163 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -40,7 +40,8 @@ Source: [`packages/ui/acp/src/index.ts:115`](../packages/ui/acp/src/index.ts) * App config: the swappable per-deployment values. `model` configures the * agent template the ACP bridge creates each session's agent from (NOT a * pre-created agent — ACP creates agents at `session/new`); `persona` is the - * deployment persona (forwarded to the system-prompt plugin); + * deployment persona (forwarded to the system-prompt plugin); `toolOrder` is + * the explicit model-facing tool order (forwarded to the system-prompt plugin); * `persistenceRoot` is the JSONL backend's directory. */ export interface Config { @@ -48,12 +49,14 @@ export interface Config { model: string /** Deployment persona (the system-prompt plugin's `persona` config). */ persona?: string + /** Explicit model-facing tool order (the system-prompt plugin's `toolOrder` config; see dsh-system-prompt). */ + toolOrder?: string[] /** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */ persistenceRoot?: string } ``` -Source: [`packages/ui/acp-agent/src/index.ts:48`](../packages/ui/acp-agent/src/index.ts) +Source: [`packages/ui/acp-agent/src/index.ts:49`](../packages/ui/acp-agent/src/index.ts) ## `@deepseek-ai/dsh-agent-core` @@ -61,29 +64,36 @@ Source: [`packages/ui/acp-agent/src/index.ts:48`](../packages/ui/acp-agent/src/i /** * Bundle config: each field forwarded verbatim to the child that owns it — * `agents` to the agent loop (an app that pre-creates no agents, like the ACP - * bridge, simply omits it), `persona` to the system-prompt plugin (the - * deployment's persona section). Both are optional INPUT here because each - * owner's schema supplies the default (`[]` / `''`); the schema is the - * INTERSECTION of the owners' own schemas, so validation and defaulting can - * never drift from them. + * bridge, simply omits it), `persona` and `toolOrder` to the system-prompt + * plugin (the deployment's persona section and the explicit model-facing tool + * order). Every field is optional INPUT here because each owner's schema + * supplies the default (`[]` / `''` / absent — lexicographic); the schema is + * the INTERSECTION of the owners' own schemas, so validation and defaulting + * can never drift from them. */ export interface Config { /** The agent-loop `agents` list (see dsh-agent-loop's `Config`). */ agents?: AgentLoopConfig['agents'] /** The deployment persona (see dsh-system-prompt's `Config`). */ persona?: SystemPromptConfig['persona'] + /** The explicit model-facing tool order (see dsh-system-prompt's `Config`). */ + toolOrder?: SystemPromptConfig['toolOrder'] } ``` Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) -Source: [`packages/core/agent-core/src/index.ts:68`](../packages/core/agent-core/src/index.ts) +Source: [`packages/core/agent-core/src/index.ts:69`](../packages/core/agent-core/src/index.ts) ## `@deepseek-ai/dsh-agent-loop` Requires: `agents` · `sessions` · `llm` · `tools` · `systemPrompt` ```ts config-catalog +/** + * Plugin config: the agents to create — or resume, via `resumeSessionId` — + * declaratively at startup, so a cordis.yml deployment needs no code. + */ export interface Config { /** Agents created from configuration at startup. */ agents: (AgentOptions & { @@ -109,7 +119,7 @@ export interface Config { Depends on: [`AgentId`](../packages/core/agent/src/index.ts) · [`AgentOptions`](../packages/core/agent/src/index.ts) · [`SessionId`](../packages/core/session/src/index.ts) -Source: [`packages/core/agent-loop/src/index.ts:32`](../packages/core/agent-loop/src/index.ts) +Source: [`packages/core/agent-loop/src/index.ts:36`](../packages/core/agent-loop/src/index.ts) ## `@deepseek-ai/dsh-bash-local` @@ -129,7 +139,43 @@ export interface Config { } ``` -Source: [`packages/bash/bash-local/src/index.ts:28`](../packages/bash/bash-local/src/index.ts) +Source: [`packages/bash/bash-local/src/index.ts:29`](../packages/bash/bash-local/src/index.ts) + +## `@deepseek-ai/dsh-code-runtime-worker` + +```ts config-catalog +/** Plugin config: every execution cap, changeable from `cordis.yml` (no hardcoded tunables). */ +export interface Config { + /** + * Busy-time budget in milliseconds: the run fails with kind `'timeout'` + * once the worker's MEASURED event-loop active time + * (`worker.performance.eventLoopUtilization()`) exceeds this. Metering + * measured busy time — not wall time, not host-side pending-call + * bookkeeping — is what makes the budget both fair (a program awaiting a + * slow tool accrues nothing) and ungameable (a hot loop accrues whether + * or not a decoy dispatch is in flight). + */ + computeMs?: number + /** + * Wall-clock ceiling in milliseconds; never pauses for anything. The + * backstop for what busy-time cannot see (a program awaiting a promise + * nobody will resolve). + */ + maxWallMs?: number + /** Shared byte budget for captured log text (console + raw stream writes), truncation marked in-band. */ + maxLogBytes?: number + /** + * Byte cap for the completion value, measured by its real cross-boundary + * size (string bytes, or structured-clone wire size); an oversized or + * non-cloneable value crosses as a capped string rendering. + */ + maxValueBytes?: number + /** The worker's max old-generation heap in MiB (`resourceLimits`); overflow kills the worker, surfacing as kind `'worker-exit'`. */ + maxOldGenerationSizeMb?: number +} +``` + +Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:29`](../packages/code-runtime/code-runtime-worker/src/index.ts) ## `@deepseek-ai/dsh-compact-basic` @@ -218,7 +264,7 @@ export interface Config { } ``` -Source: [`packages/hooks/hooks-claude/src/index.ts:55`](../packages/hooks/hooks-claude/src/index.ts) +Source: [`packages/hooks/hooks-claude/src/index.ts:56`](../packages/hooks/hooks-claude/src/index.ts) ## `@deepseek-ai/dsh-hooks-codex` @@ -243,7 +289,7 @@ export interface Config { } ``` -Source: [`packages/hooks/hooks-codex/src/index.ts:42`](../packages/hooks/hooks-codex/src/index.ts) +Source: [`packages/hooks/hooks-codex/src/index.ts:43`](../packages/hooks/hooks-codex/src/index.ts) ## `@deepseek-ai/dsh-invariants` @@ -268,6 +314,12 @@ Source: [`packages/support/invariants/src/index.ts:45`](../packages/support/inva Requires: `llm` ```ts config-catalog +/** + * Plugin config, validated by the same-named schemastery schema. Every field + * is optional in yml: credentials/endpoint fall back to the environment (a + * missing API key fails plugin load, not the first call), and omitted + * thinking fields send nothing on the wire, so the provider default applies. + */ export interface Config { /** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */ apiKey?: string @@ -282,13 +334,18 @@ export interface Config { } ``` -Source: [`packages/llm/llm-deepseek/src/index.ts:37`](../packages/llm/llm-deepseek/src/index.ts) +Source: [`packages/llm/llm-deepseek/src/index.ts:43`](../packages/llm/llm-deepseek/src/index.ts) ## `@deepseek-ai/dsh-llm-pi-ai` Requires: `llm` ```ts config-catalog +/** + * Plugin config, validated by the same-named schemastery schema. Every field + * is optional in yml: credentials/endpoint fall back to the environment (a + * missing API key fails plugin load, not the first call). + */ export interface Config { /** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */ apiKey?: string @@ -308,13 +365,14 @@ export interface Config { export type PiAiReasoning = 'off' | 'high' | 'xhigh' ``` -Source: [`packages/llm/llm-pi-ai/src/index.ts:32`](../packages/llm/llm-pi-ai/src/index.ts) +Source: [`packages/llm/llm-pi-ai/src/index.ts:37`](../packages/llm/llm-pi-ai/src/index.ts) ## `@deepseek-ai/dsh-llm-replay` Requires: `llm` ```ts config-catalog +/** Plugin config: the {@link ReplayConfig} inputs, each defaulting to its `DSH_SNAPSHOT_*` env var in `apply`. */ export interface Config { /** Override the fixture path; defaults to `$DSH_SNAPSHOT_FILE`. */ file?: string @@ -329,13 +387,46 @@ export interface Config { } ``` -Source: [`packages/support/llm-replay/src/index.ts:411`](../packages/support/llm-replay/src/index.ts) +Source: [`packages/support/llm-replay/src/index.ts:429`](../packages/support/llm-replay/src/index.ts) + +## `@deepseek-ai/dsh-repeat-tool-guard` + +```ts config-catalog +/** + * Plugin config, validated by the same-named schemastery schema plus the + * load-time checks in `apply` (misconfiguration fails loud: an empty + * `thresholds` list, a non-integer, a value below 2, or a duplicate throws at + * plugin load, never a silent fall-back). `include`/`exclude` entries are + * `*`-wildcard predicates over tool names at call time, not references to + * registry entries — a pattern matching no currently registered tool is valid + * (`exclude: [mcp_*]` must stay legal in a deployment that loads no MCP tools). + */ +export interface Config { + /** Consecutive-repeat counts that trigger a reminder (default `[3, 5, 8]`). */ + thresholds?: number[] + /** Tool-name patterns to track; empty means every tool is tracked. */ + include?: string[] + /** Tool-name patterns transparent to the chain (neither count nor reset). */ + exclude?: string[] + /** + * Maximum characters of canonical arguments quoted in the DETAILED reminder + * (default 500). Large payloads (a `write` body, a long command) would + * otherwise ride into the next request unbounded — precisely in a loop + * scenario; the cap bounds the reminder, never the detection (the chain key + * always compares the FULL canonical string). + */ + argumentsPreviewChars?: number +} +``` + +Source: [`packages/guard/repeat-tool-guard/src/index.ts:55`](../packages/guard/repeat-tool-guard/src/index.ts) ## `@deepseek-ai/dsh-session-persistence-jsonl` Requires: `sessions` ```ts config-catalog +/** Plugin config: where the JSONL backend keeps its session logs (`root` is required — no default). */ export interface Config { /** * Root directory for all session files. Required (no default): a default of @@ -346,7 +437,7 @@ export interface Config { } ``` -Source: [`packages/session-persistence/session-persistence-jsonl/src/index.ts:34`](../packages/session-persistence/session-persistence-jsonl/src/index.ts) +Source: [`packages/session-persistence/session-persistence-jsonl/src/index.ts:35`](../packages/session-persistence/session-persistence-jsonl/src/index.ts) ## `@deepseek-ai/dsh-session-persistence-sqlite` @@ -390,7 +481,8 @@ Source: [`packages/session-persistence/session-persistence-sqlite/src/index.ts:5 * App config: the swappable per-demo values, each routed to where the app wires * it. `model`/`resumeSessionId` configure the pre-created `main` agent (through * {@link @deepseek-ai/dsh-agent-core}'s forwarded `agents` list); `persona` is - * the deployment persona (forwarded to the system-prompt plugin); + * the deployment persona (forwarded to the system-prompt plugin); `toolOrder` + * is the explicit model-facing tool order (forwarded to the system-prompt plugin); * `persistenceRoot` is the JSONL backend's directory; `welcome` is the UI banner. */ export interface Config { @@ -398,6 +490,8 @@ export interface Config { model: string /** Deployment persona (the system-prompt plugin's `persona` config). */ persona?: string + /** Explicit model-facing tool order (the system-prompt plugin's `toolOrder` config; see dsh-system-prompt). */ + toolOrder?: string[] /** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */ persistenceRoot?: string /** stdin-chat banner printed once on start. Defaults to `'ready.'`. */ @@ -411,7 +505,7 @@ export interface Config { } ``` -Source: [`packages/ui/stdio-agent/src/index.ts:59`](../packages/ui/stdio-agent/src/index.ts) +Source: [`packages/ui/stdio-agent/src/index.ts:60`](../packages/ui/stdio-agent/src/index.ts) ## `@deepseek-ai/dsh-subagent-acp` @@ -481,7 +575,7 @@ export interface Config { } ``` -Source: [`packages/subagent/subagent-fork/src/index.ts:34`](../packages/subagent/subagent-fork/src/index.ts) +Source: [`packages/subagent/subagent-fork/src/index.ts:38`](../packages/subagent/subagent-fork/src/index.ts) ## `@deepseek-ai/dsh-subagent-mock` @@ -528,11 +622,12 @@ export interface Config { } ``` -Source: [`packages/subagent/subagent-spawn/src/index.ts:26`](../packages/subagent/subagent-spawn/src/index.ts) +Source: [`packages/subagent/subagent-spawn/src/index.ts:36`](../packages/subagent/subagent-spawn/src/index.ts) ## `@deepseek-ai/dsh-system-prompt` ```ts config-catalog +/** Plugin config: the deployment-authored fragment of the system prompt (see {@link Config.persona} for its contract). */ export interface Config { /** * The deployment's persona — the ONE deployment-authored fragment of the @@ -547,10 +642,33 @@ export interface Config { * deployment opens with the harness identity alone. */ persona?: string + /** + * Explicit model-facing tool order, as a list of `ToolSchema.name`s: listed + * tools take their listed position, and tools absent from the list are + * inserted at the {@link TOOL_ORDER_REST} (`''`) entry in + * lexicographic name order. A configured list must contain the rest entry + * exactly once, no duplicate names, and no name without a registered tool — + * a misconfigured order blocks work instead of silently reaching a model + * request: shape violations throw at load, and an unregistered name rejects + * every assembly. `TOOL_ORDER_REST` is reserved for the list marker and may + * not be a collected tool name; such a provider output also rejects the + * assembly. The single assembly-time validation rejects either failure + * before any model request — the earliest moment the registered tool set + * exists to check against, since tool plugins register after this service + * constructs. When omitted, tools are ordered lexicographically by name. + * Applied to the tools + * {@link SystemPrompt.assemble} collects, BEFORE the + * `system-prompt/assemble` waterfall — like the sections' `order` sort, it + * canonicalizes what the registry contributed (registration order is a + * plugin-load artifact); a waterfall listener that mutates the tool list + * owns the determinism of what it emits. Rationale (and why not per-plugin + * weights): docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md. + */ + toolOrder?: string[] } ``` -Source: [`packages/core/system-prompt/src/index.ts:113`](../packages/core/system-prompt/src/index.ts) +Source: [`packages/core/system-prompt/src/index.ts:179`](../packages/core/system-prompt/src/index.ts) ## `@deepseek-ai/dsh-tool-fs` @@ -608,6 +726,7 @@ Source: [`packages/subagent/tool-subagent/src/index.ts:44`](../packages/subagent Requires: `tools` · `web` · `systemPrompt` ```ts config-catalog +/** Plugin config: which web tools to register, the source cap, and per-tool budgets. */ export interface Config { /** Register `web_search`. Defaults to true. */ search?: boolean @@ -615,10 +734,14 @@ export interface Config { fetch?: boolean /** Upper bound on sources returned by one `web_search` call. */ searchMaxResults?: number + /** Cooperative timeout budget (ms) for `web_fetch`. Defaults to 30000. */ + fetchTimeoutMs?: number + /** Cooperative timeout budget (ms) for `web_search`. Defaults to 30000. */ + searchTimeoutMs?: number } ``` -Source: [`packages/web/tool-web/src/index.ts:36`](../packages/web/tool-web/src/index.ts) +Source: [`packages/web/tool-web/src/index.ts:40`](../packages/web/tool-web/src/index.ts) ## `@deepseek-ai/dsh-web` @@ -644,6 +767,7 @@ Source: [`packages/web/web/src/index.ts:68`](../packages/web/web/src/index.ts) Requires: `web` ```ts config-catalog +/** Plugin config: the provider's transport and size limits plus its `User-Agent` (all defaulted). */ export interface Config { /** Maximum accepted request URL length. */ maxUrlLength?: number @@ -662,13 +786,14 @@ export interface Config { } ``` -Source: [`packages/web/web-fetch-local/src/index.ts:33`](../packages/web/web-fetch-local/src/index.ts) +Source: [`packages/web/web-fetch-local/src/index.ts:34`](../packages/web/web-fetch-local/src/index.ts) ## `@deepseek-ai/dsh-web-search-deepseek` Requires: `web` ```ts config-catalog +/** Plugin config (all optional — `apply` fills env-var and constant defaults). */ export interface Config { /** DeepSeek API key. Falls back to `$DEEPSEEK_API_KEY`. Empty → unavailable. */ apiKey?: string @@ -685,13 +810,14 @@ export interface Config { } ``` -Source: [`packages/web/web-search-deepseek/src/index.ts:47`](../packages/web/web-search-deepseek/src/index.ts) +Source: [`packages/web/web-search-deepseek/src/index.ts:48`](../packages/web/web-search-deepseek/src/index.ts) ## `@deepseek-ai/dsh-web-search-exa` Requires: `web` ```ts config-catalog +/** Plugin config (all optional — `apply` fills env-var and constant defaults). */ export interface Config { /** Exa API key. Falls back to `$EXA_API_KEY`. Empty → provider unavailable. */ apiKey?: string @@ -706,13 +832,14 @@ export interface Config { } ``` -Source: [`packages/web/web-search-exa/src/index.ts:38`](../packages/web/web-search-exa/src/index.ts) +Source: [`packages/web/web-search-exa/src/index.ts:39`](../packages/web/web-search-exa/src/index.ts) ## `@deepseek-ai/dsh-web-search-perplexity` Requires: `web` ```ts config-catalog +/** Plugin config (all optional — `apply` fills env-var and constant defaults). */ export interface Config { /** Perplexity API key. Falls back to `$PERPLEXITY_API_KEY`. Empty → unavailable. */ apiKey?: string @@ -727,7 +854,7 @@ export interface Config { } ``` -Source: [`packages/web/web-search-perplexity/src/index.ts:32`](../packages/web/web-search-perplexity/src/index.ts) +Source: [`packages/web/web-search-perplexity/src/index.ts:33`](../packages/web/web-search-perplexity/src/index.ts) ## Loadable plugins with no config @@ -738,6 +865,7 @@ These load from a `cordis.yml` entry with no `config:` block; they declare no co - `@deepseek-ai/dsh-llm` ([`packages/llm/llm/src/index.ts`](../packages/llm/llm/src/index.ts)) - `@deepseek-ai/dsh-session` ([`packages/core/session/src/index.ts`](../packages/core/session/src/index.ts)) - `@deepseek-ai/dsh-subagent` ([`packages/subagent/subagent/src/index.ts`](../packages/subagent/subagent/src/index.ts)) +- `@deepseek-ai/dsh-timeout-policy` — requires `tools` ([`packages/timeout/timeout-policy/src/index.ts`](../packages/timeout/timeout-policy/src/index.ts)) - `@deepseek-ai/dsh-tool-bash` — requires `tools` · `bash` · `systemPrompt` ([`packages/bash/tool-bash/src/index.ts`](../packages/bash/tool-bash/src/index.ts)) - `@deepseek-ai/dsh-tool-todo` — requires `tools` ([`packages/todo/tool-todo/src/index.ts`](../packages/todo/tool-todo/src/index.ts)) - `@deepseek-ai/dsh-tools` — requires `systemPrompt` ([`packages/core/tools/src/index.ts`](../packages/core/tools/src/index.ts)) @@ -747,6 +875,7 @@ These load from a `cordis.yml` entry with no `config:` block; they declare no co Abstract service classes — a deployment loads a concrete implementation package instead ([capability seams](rfc/implemented/architecture/2026-06-13-capability-seams.md)). - `@deepseek-ai/dsh-bash` — abstract `BashExecutor` ([`packages/bash/bash/src/index.ts`](../packages/bash/bash/src/index.ts)) +- `@deepseek-ai/dsh-code-runtime` — abstract `CodeRuntime` ([`packages/code-runtime/code-runtime/src/index.ts`](../packages/code-runtime/code-runtime/src/index.ts)) - `@deepseek-ai/dsh-compact` — abstract `CompactService` ([`packages/compact/compact/src/index.ts`](../packages/compact/compact/src/index.ts)) - `@deepseek-ai/dsh-fs` — abstract `FileSystem` ([`packages/fs/fs/src/index.ts`](../packages/fs/fs/src/index.ts)) - `@deepseek-ai/dsh-session-persistence` — abstract `SessionPersistence` ([`packages/session-persistence/session-persistence/src/index.ts`](../packages/session-persistence/session-persistence/src/index.ts)) @@ -755,7 +884,9 @@ Abstract service classes — a deployment loads a concrete implementation packag Imported as libraries by other packages; a `cordis.yml` cannot load them. +- `@deepseek-ai/dsh-acp-snapshot` ([`packages/support/acp-snapshot/src/index.ts`](../packages/support/acp-snapshot/src/index.ts)) - `@deepseek-ai/dsh-app-boot` ([`packages/ui/app-boot/src/index.ts`](../packages/ui/app-boot/src/index.ts)) - `@deepseek-ai/dsh-brand` ([`packages/util/brand/src/index.ts`](../packages/util/brand/src/index.ts)) - `@deepseek-ai/dsh-hook-protocol` ([`packages/hooks/hook-protocol/src/index.ts`](../packages/hooks/hook-protocol/src/index.ts)) - `@deepseek-ai/dsh-subagent-inprocess` ([`packages/subagent/subagent-inprocess/src/index.ts`](../packages/subagent/subagent-inprocess/src/index.ts)) +- `@deepseek-ai/dsh-timeout` ([`packages/util/timeout/src/index.ts`](../packages/util/timeout/src/index.ts)) diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 753cf8f4d0..7c21e59489 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -23,7 +23,7 @@ An agent was registered in the AgentRegistry and is ready to receive messages. Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:248`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:264`](../../packages/core/agent/src/types.ts) ### `agent/disposed` — emit @@ -35,7 +35,7 @@ An agent was disposed and removed from the registry; its fiber and any in-flight Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:255`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:271`](../../packages/core/agent/src/types.ts) ### `agent/error` — emit @@ -47,7 +47,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:404`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:420`](../../packages/core/agent/src/types.ts) ### `agent/pre-step` — serial @@ -61,7 +61,7 @@ Serial (awaited in registration order), not a waterfall: a listener mutates the Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:333`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:349`](../../packages/core/agent/src/types.ts) ### `agent/prompt-submit` — waterfall @@ -73,7 +73,7 @@ Waterfall: decide what happens to ONE drained queued message before it becomes a Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:346`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:362`](../../packages/core/agent/src/types.ts) ### `agent/queued` — emit @@ -85,7 +85,7 @@ A message entered the agent's inbox (queued or steering). `source` is the resolv Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:273`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:289`](../../packages/core/agent/src/types.ts) ### `agent/request` — waterfall @@ -97,7 +97,7 @@ Waterfall: shape the step's call configuration — model switching, sampling ove Types: [Agent](../core-data-structures/core.md) · [LlmCallConfig](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:369`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:385`](../../packages/core/agent/src/types.ts) ### `agent/session-start` — emit @@ -109,7 +109,7 @@ The agent's session lifecycle began, fired once before its first turn. `source` Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:288`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:304`](../../packages/core/agent/src/types.ts) ### `agent/status` — emit @@ -121,7 +121,7 @@ Agent status changed (`idle` ⇄ `running`, or → `disposed`). Drive lifecycle Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:264`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:280`](../../packages/core/agent/src/types.ts) ### `agent/step-result` — waterfall @@ -133,7 +133,7 @@ Waterfall: post-process the assembled assistant Message before tool dispatch (va Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:379`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:395`](../../packages/core/agent/src/types.ts) ### `agent/turn-continuation` — waterfall @@ -145,7 +145,7 @@ Waterfall: override the turn-continuation decision via a typed ContinuationDecis Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:392`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:408`](../../packages/core/agent/src/types.ts) ## `fs/*` @@ -307,11 +307,23 @@ A tool was registered or unregistered (the available tool set changed). 'tools/change'(): void ``` -Source: [`packages/core/tools/src/index.ts:87`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:118`](../../packages/core/tools/src/index.ts) + +### `tools/execute` — waterfall + +Around-dispatch waterfall wrapping the registry's core tool dispatch, between the `tools/pre-execute` gate and the `tools/post-execute` seam. A listener receives `(exec, next)`: call `next()` to delegate to dispatch (returning its ToolExecutionResult, optionally wrapped), or return a replacement result without calling `next()` to short-circuit dispatch. The base `next()` IS the dispatch-with-normalization thunk — a thrown tool (or unknown tool) is already normalized to an `isError` result by the time a listener's `await next()` returns, so a wrapper never sees a raw throw from the tool body. This is the seam a timeout/retry/metrics plugin wraps: it can mutate `exec` (e.g. replace `exec.signal` with a per-call deadline) BEFORE `next()` and inspect the result AFTER. (Cordis `next()` ignores any passed arguments and re-invokes downstream with the shared payload, so a wrapper mutates `exec` in place rather than passing a new object to `next()`.) Multiple listeners compose by registration order — an outer one wraps the inner ones plus dispatch. + +```ts cordis-catalog +'tools/execute'(this: ToolRegistry, exec: ToolExecution, next: () => Promise): Promise +``` + +Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) + +Source: [`packages/core/tools/src/index.ts:97`](../../packages/core/tools/src/index.ts) ### `tools/post-execute` — waterfall -Waterfall AFTER a tool runs — where hook plugins inspect the result and accept it (optionally REPLACING the model-facing content, and/or attaching `additionalContext` for the next request) or block it with corrective `feedback` (Claude Code's `PostToolUse`). Listeners receive `(exec, result, next)`: call `next()` to delegate to the default (accept unchanged), or return a PostToolDecision to override. The core tool dispatch sits between the two waterfalls as plain code, all inside `execute`'s outer try/catch (and the tool body keeps its own inner try/catch, so a thrown tool still reaches `post-execute` as an `isError` result). +Waterfall AFTER a tool runs — where hook plugins inspect the result and accept it (optionally REPLACING the model-facing content, and/or attaching `additionalContext` for the next request) or block it with corrective `feedback` (Claude Code's `PostToolUse`). Listeners receive `(exec, result, next)`: call `next()` to delegate to the default (accept unchanged), or return a PostToolDecision to override. Core tool dispatch runs earlier as the base `next()` of the `tools/execute` waterfall, all inside `execute`'s outer try/catch (and the tool body keeps its own inner try/catch, so a thrown tool still reaches `post-execute` as an `isError` result). ```ts cordis-catalog 'tools/post-execute'(this: ToolRegistry, exec: ToolExecution, result: ToolExecutionResult, next: () => Promise): Promise @@ -319,7 +331,7 @@ Waterfall AFTER a tool runs — where hook plugins inspect the result and accept Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:82`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:113`](../../packages/core/tools/src/index.ts) ### `tools/pre-execute` — waterfall @@ -331,7 +343,7 @@ Waterfall BEFORE a tool runs — the gate where sandbox, permission, and hook pl Types: [ToolExecution](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:66`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:77`](../../packages/core/tools/src/index.ts) ## Inherited events (cordis core + loader/hmr/timer) diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index f0ae4ea77e..5f4fa4ff78 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -21,7 +21,7 @@ createAgent(options: CreateAgentOptions): AgentHandle async resume(options: ResumeAgentOptions): Promise ``` -Source: [`packages/core/agent-loop/src/index.ts:64`](../../packages/core/agent-loop/src/index.ts) +Source: [`packages/core/agent-loop/src/index.ts:68`](../../packages/core/agent-loop/src/index.ts) ## `ctx.agents` — `AgentRegistry` @@ -67,6 +67,25 @@ Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../c Source: [`packages/bash/bash/src/index.ts:59`](../../packages/bash/bash/src/index.ts) +## `ctx.codeRuntime` — `CodeRuntime` (abstract seam) + +Abstract code-execution service. Subclass, implement run and the two descriptors, and load the subclass as a plugin — it registers as `ctx.codeRuntime` (one implementation per context; loading a second throws, cordis' standard duplicate-service behavior). + +Semantics every implementation must honor: + +- run resolves with an error FIELD for every program outcome — parse/transform failures, thrown exceptions, budget expiry, abort, substrate death (CodeRunFailure's taxonomy). It REJECTS only for caller misuse of the seam itself (e.g. a run submitted after disposal). +- Binding calls bridge to the caller's CodeBindingFunctions verbatim; arguments and resolutions must be structured-cloneable, and the runtime treats the program as a hostile peer (arbitrary binding names are own properties, malformed traffic is rejected or ignored, never crashes the host). +- Runs are isolated from each other: no state survives from one run to the next through the runtime. +- Disposal reaches quiescence: in-flight runs are terminated AND awaited before the service's own teardown completes (no orphan substrate survives `fiber.dispose()`). + +```ts cordis-catalog +abstract run(request: CodeRunRequest): Promise +``` + +Types: [CodeRunRequest](../core-data-structures/code-runtime.md) · [CodeRunResult](../core-data-structures/code-runtime.md) + +Source: [`packages/code-runtime/code-runtime/src/index.ts:59`](../../packages/code-runtime/code-runtime/src/index.ts) + ## `ctx.compact` — `CompactService` (abstract seam) Abstract compaction service. Subclass implement the two abstract methods, and load the subclass as a plugin — it registers as `ctx.compact` (one implementation per context; loading a second throws, which is cordis' standard duplicate-service behavior). @@ -124,7 +143,7 @@ stream(options: GenerateOptions): AsyncIterable Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:84`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:88`](../../packages/llm/llm/src/index.ts) ## `ctx.sessionPersistence` — `SessionPersistence` (abstract seam) @@ -146,7 +165,7 @@ abstract list(): Promise Types: [SessionEvent](../core-data-structures/core.md) -Source: [`packages/session-persistence/session-persistence/src/index.ts:98`](../../packages/session-persistence/session-persistence/src/index.ts) +Source: [`packages/session-persistence/session-persistence/src/index.ts:102`](../../packages/session-persistence/session-persistence/src/index.ts) ## `ctx.sessions` — `SessionStore` @@ -161,9 +180,10 @@ enter(session: Session): () => void announce(session: Session): void get(id: SessionId): Session | undefined list(): Session[] +fork(source: SessionForkSource, boundary?: number, childSessionId?: SessionId): Session ``` -Source: [`packages/core/session/src/index.ts:371`](../../packages/core/session/src/index.ts) +Source: [`packages/core/session/src/index.ts:405`](../../packages/core/session/src/index.ts) ## `ctx.subagents` — `SubagentService` @@ -186,14 +206,14 @@ Registry service (`ctx.systemPrompt`): plugins contribute ordered text sections, section(section: PromptSection): () => void tools(provider: () => ToolSchema[]): () => void variable(name: string, provider: (context: AssembleContext) => string | undefined): () => void -assemble(context: AssembleContext = {}): Promise +async assemble(context: AssembleContext = {}): Promise ``` -Source: [`packages/core/system-prompt/src/index.ts:198`](../../packages/core/system-prompt/src/index.ts) +Source: [`packages/core/system-prompt/src/index.ts:291`](../../packages/core/system-prompt/src/index.ts) ## `ctx.tools` — `ToolRegistry` -Tool registry (`ctx.tools`): tool plugins register definitions; the agent loop executes calls through the `tools/pre-execute` → dispatch → `tools/post-execute` pipeline. The registry contributes its schemas into the system-prompt assembly. +Tool registry (`ctx.tools`): tool plugins register definitions; the agent loop executes calls through the `tools/pre-execute` → `tools/execute` → `tools/post-execute` pipeline. The registry contributes its schemas into the system-prompt assembly. ```ts cordis-catalog register(definition: ToolDefinition): () => void @@ -204,7 +224,7 @@ async execute(exec: ToolExecution): Promise Types: [ToolDefinition](../core-data-structures/tools.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:268`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:307`](../../packages/core/tools/src/index.ts) ## `ctx.web` — `WebService` diff --git a/docs/core-data-structures/code-runtime.md b/docs/core-data-structures/code-runtime.md new file mode 100644 index 0000000000..1f87e8e8a4 --- /dev/null +++ b/docs/core-data-structures/code-runtime.md @@ -0,0 +1,94 @@ +# Code Runtime + +The code-execution seam — a [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) whose interface ([dsh-code-runtime](../../packages/code-runtime/code-runtime), `ctx.codeRuntime`) runs one model-written program against host-provided async bindings and reports what it printed and returned. Code execution is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). Backends differ by execution substrate and source language, both readonly descriptors on the service; the worker-thread backend and the tool-registry consumer (Code Mode) are specified in the [Code Mode RFC](../rfc/proposed/feature/2026-06-15-code-mode.md). + +Source: [`packages/code-runtime/code-runtime/src/types.ts`](../../packages/code-runtime/code-runtime/src/types.ts) + +## The run: request in, result out + +A `CodeRunRequest` carries **everything the runtime acts on** — per the "explicit > implicit at package seams" rule, defaulting (time budgets, output caps) is the implementation's validated config, never a hidden `??` inside `run()`: + +```ts type-equiv +interface CodeRunRequest { + /** + * The program source, in the runtime's {@link ../index.ts | language}. It + * runs as the body of an async function: top-level `await` and `return` + * are available, and the completion value becomes + * {@link CodeRunResult.value}. + */ + program: string + /** Host functions exposed to the program, one global object per namespace. */ + bindings: CodeBindingNamespace[] + /** + * Abort the run: the runtime stops the program (hard, even mid-loop) and + * resolves with a {@link CodeRunFailure} of kind `'abort'`. In-flight + * binding calls are the CALLER's to settle — the runtime only stops asking. + */ + signal?: AbortSignal +} +``` + +The result reports an error as a **field**, never a rejection of `run()` — reporting a failed program is the caller's job, not an exception path (mirroring `BashExecutor.run`'s resolve-on-failure contract): + +```ts type-equiv +interface CodeRunResult { + /** + * The program's completion value (its top-level `return`), when it ran to + * completion and the value survived the runtime's serialization boundary; + * a non-transferable value is replaced by a string rendering, and a failed + * or value-less run leaves this absent. + */ + value?: unknown + /** Everything the program emitted, in order (capped by the implementation). */ + logs: CodeLogEntry[] + /** Present iff the run failed; see {@link CodeRunFailure} for the taxonomy. */ + error?: CodeRunFailure +} +``` + +## Bindings: host functions as program globals + +Each `CodeBindingNamespace` becomes one global object of async callables inside the program (the Code Mode consumer passes one: `tools`). Arguments and resolutions must be structured-cloneable — a runtime may bridge calls across a serialization boundary — and a runtime treats binding names as hostile input (`__proto__` is an ordinary own property, never a prototype collision): + +```ts type-equiv +interface CodeBindingNamespace { + /** The global identifier the program sees (must be a valid JS identifier). */ + global: string + /** The callable members, keyed by the exact name the program calls. */ + functions: Record +} +``` + +```ts type-equiv +type CodeBindingFunction = (args: unknown) => Promise +``` + +## Captured output and the failure taxonomy + +Logs arrive in emission order, attributed to their channel (the runtime's `console` shim, or stray writes to the underlying streams): + +```ts type-equiv +interface CodeLogEntry { + /** Which channel produced the text. */ + source: 'console' | 'stdout' | 'stderr' + /** The console method used; present only when `source` is `'console'`. */ + level?: 'log' | 'info' | 'warn' | 'error' | 'debug' + /** The captured text (possibly truncated by the implementation's caps, marked in-band). */ + text: string +} +``` + +Failure kinds are **orthogonal outcomes reported independently** (per [defensive-patterns](../defensive-patterns.md)): a budget expiry is not an exception, an abort is not a timeout, and a substrate death (e.g. OOM) is neither: + +```ts type-equiv +interface CodeRunFailure { + /** The failure class (see the interface doc for each kind's meaning). */ + kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' + /** Human-readable detail, suitable for feeding back to a model to self-correct. */ + message: string +} +``` + +## The service + +`CodeRuntime` (`ctx.codeRuntime`, abstract — defined in [`packages/code-runtime/code-runtime/src/index.ts`](../../packages/code-runtime/code-runtime/src/index.ts)) is `run(request)` plus two readonly descriptors: `language` (what the program must be written in — `'typescript'` is the well-known value; a consumer generating language-specific presentation switches on it and fails loud on one it cannot present) and `isolation` (the execution substrate — `'worker-thread'`, `'process'`, `'container'`; a diagnostic label, **not a security claim**). Implementations must keep runs isolated from each other (no cross-run state) and dispose to quiescence: in-flight runs are terminated and awaited before teardown completes. diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 8cd9f1c30b..615c222d94 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -20,6 +20,7 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t | [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` | | [tools.md](tools.md) | `ToolDefinition` full fields, the schema DSL, `ToolExecution`/`ToolResult`, tool-presentation UI types, the `tools/pre-execute`/`tools/post-execute` pipeline | | [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, background `BashTask`s | +| [code-runtime.md](code-runtime.md) | the code-execution seam: `CodeRunRequest`/`Result`, binding namespaces, captured logs, the `CodeRunFailure` taxonomy | | [filesystem.md](filesystem.md) | the filesystem seam: `FsTarget`, read/write/edit outcomes, observed-file state, `FsErrorCode` | | [compaction.md](compaction.md) | the compaction seam: the `compact/*` session events, `CompactionResult`, the `CompactService` interface | | [subagent.md](subagent.md) | the subagent seam: the named-provider registry, `SubagentStartRequest`/`Result`/`Run`, the start-time-vs-runtime capability split | @@ -187,7 +188,7 @@ The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition` ### The request envelope: `LlmCallConfig` and the logged header -Requests are built by the loop, not shaped per call: the non-content half of a request — the `EpochHeader`: this call configuration plus the rendered system prompt and assembled tool schemas — is logged session state (`request/header` snapshot and delta events, [session.md](session.md#the-request-header-events-requestheader-and-requestheader-delta)), so every conversation request is a pure function of the session log ([reconstructability RFC](../rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). The `agent/request` waterfall receives a frozen `LlmCallConfig` seed and a listener returns a replacement to switch model or sampling — the loop logs whatever the request actually uses. Loop-built requests arrive at `llm/stream` deep-frozen; mutation throws. +Requests are built by the loop, not shaped per call: the non-content half of a request — the `EpochHeader`: this call configuration plus the rendered system prompt and the tool schemas in the assembly's canonical order (dsh-system-prompt's `toolOrder` config, lexicographic when unset) — is logged session state (`request/header` snapshot and delta events, [session.md](session.md#the-request-header-events-requestheader-and-requestheader-delta)), so every conversation request is a pure function of the session log ([reconstructability RFC](../rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). The `agent/request` waterfall receives a frozen `LlmCallConfig` seed and a listener returns a replacement to switch model or sampling — the loop logs whatever the request actually uses. Loop-built requests arrive at `llm/stream` deep-frozen; mutation throws. FIXME(call-config-shape): revisit the exact definition of this type — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit here out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them. diff --git a/docs/core-data-structures/session.md b/docs/core-data-structures/session.md index 641bbcd0df..6c4712114f 100644 --- a/docs/core-data-structures/session.md +++ b/docs/core-data-structures/session.md @@ -200,6 +200,14 @@ export interface SurfaceNode { Everything else (`turn/*`, `step/*`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`. +## Live-session fork API + +`ctx.sessions.create(id, { seed, meta })` is the low-level replay/fork primitive. For ordinary live-session forks, `SessionStore` exposes one policy API: + +- `fork(source, boundary?, childSessionId?)` accepts a live `Session` object or live `SessionId`, selects source events through the inclusive `boundary` seq (default: current last event), requires the boundary event to be `turn/end`, then creates a live child session with deep-cloned seed events plus child metadata (`parentSession`, `seedLength`, and inherited `cwd`). + +An explicit `boundary` lets callers fork from a previous completed turn even if the source has newer events or an open current turn. The API rejects non-`turn/end` boundaries instead of clipping silently. Broader turn-enclosure sanity stays in the existing `dsh-invariants` plugin and persistence repair path rather than being duplicated in `fork()`. `dsh-subagent-fork` keeps its completed-prefix clipping because tool-time delegation usually starts while the parent turn is open; ordinary session branching should make the requested boundary explicit. + ## What started a turn: `TurnTriggerMap` ```ts type-equiv diff --git a/docs/core-data-structures/subagent.md b/docs/core-data-structures/subagent.md index 926e6ce9b8..f21ef15669 100644 --- a/docs/core-data-structures/subagent.md +++ b/docs/core-data-structures/subagent.md @@ -20,7 +20,7 @@ interface SubagentCapabilities { ## The start request -What a caller asks for when starting a subagent. The tool layer builds this from the model's `{ description, prompt }` plus its own config; the service validates the start-time capabilities against the named provider, then passes it to `provider.start`. `parent` is REQUIRED — in-process backends read `parent.session.header` for the working directory, the `parentSession` lineage, and the delegation depth. The three optional fields (`outputSchema`, `maxDepth`, `toolFilter`) each gate on the matching `SubagentCapabilities` flag. +What a caller asks for when starting a subagent. The tool layer builds this from the model's `{ description, prompt }` plus its own config; the service validates the start-time capabilities against the named provider, then passes it to `provider.start`. `parent` is REQUIRED — in-process backends read `parent.session.header` for the working directory, the `parentSession` lineage, and the delegation depth. The three optional fields (`outputSchema`, `maxDepth`, `toolFilter`) each gate on the matching `SubagentCapabilities` flag. `outputSchema` is an object-rooted JSON Schema within the subset `assertSupportedOutputSchema` (dsh-tools) enforces — a schema outside it is rejected loud at start; the in-process backends realize it with a forced `structured_output` capture tool (see the [driver README](../../packages/subagent/subagent-inprocess/README.md)). ```ts type-equiv interface SubagentStartRequest { @@ -28,7 +28,7 @@ interface SubagentStartRequest { parent: Agent signal?: AbortSignal agentOptions?: AgentOptions - outputSchema?: SchemaSpec + outputSchema?: StructuredOutputSchema maxDepth?: number toolFilter?: { allow?: string[]; deny?: string[] } } diff --git a/docs/core-data-structures/tools.md b/docs/core-data-structures/tools.md index a05ffb3966..96d3e79bdc 100644 --- a/docs/core-data-structures/tools.md +++ b/docs/core-data-structures/tools.md @@ -11,6 +11,14 @@ A `ToolSchema` (the model-facing fields) plus the `execute` function and optiona ```ts type-equiv interface ToolDefinition extends ToolSchema { execute(args: unknown, exec: ToolExecution): Promise + /** + * Cooperative tool-call timeout budget in milliseconds. Omit for no deadline. + * Enforced by `@deepseek-ai/dsh-timeout-policy` (a `tools/execute` wrapper); it + * is NEVER sent to the model — `schemas()` whitelists only name/description/ + * parameters. Declaring it asserts this tool forwards `exec.signal` to a + * cooperative implementation that can reach quiescence when the signal aborts. + */ + timeoutMs?: number /** * Optional: how to present the PENDING state of one call in a UI, derived from * the call's `args` (parsed arguments, `unknown` — the tool validates/narrows @@ -138,6 +146,40 @@ type PostToolDecision = Call `next()` to delegate to the default (allow / accept-unchanged), or return a decision to short-circuit. A `pre-execute` `deny` (or `ask`, which degrades to deny until the permission system lands) skips dispatch and yields an `isError` result; input rewrite is deliberately NOT offered on `PreToolDecision` (it would desync the pre-execution audit/history/UI from what ran — its own proposed RFC). A `post-execute` `accept` may replace the model-facing `content` (clean, because `tool/result` is logged after `execute()` returns); a `block` turns the call into an `isError` whose content is the corrective `feedback`. Core dispatch sits between the waterfalls as plain code; the tool body keeps its own try/catch so a thrown tool still reaches `post-execute` as an `isError`. An unregistered tool routes through the same catch as a tool-thrown error, so both failure classes get a structured `{ name, code }` (`ToolNotFoundError` → `UNKNOWN_TOOL`) — the loop records a failed tool call instead of failing the whole turn. +## The structured-output schema subset + +The vocabulary a caller uses to demand a machine-readable result from a subagent (`SubagentStartRequest.outputSchema`, [subagent.md](subagent.md#the-start-request)) or a workflow `agent()` call. It is deliberately NOT full JSON Schema: the schema travels verbatim to the model as a forced tool's `parameters`, and the produced value is validated client-side by `validateStructuredValue` — so every accepted keyword must be one the validator actually enforces, and `assertSupportedOutputSchema` rejects anything else loud (`OutputSchemaError`, listing every violation). Both walkers reason over own enumerable properties only (JSON carries nothing else) and reject non-plain objects (`Date`, `Map`) that would serialize lossily. + +```ts type-equiv +type StructuredScalar = string | number | boolean | null +``` + +```ts type-equiv +type StructuredSchemaType = 'object' | 'array' | 'string' | 'number' | 'integer' | 'boolean' | 'null' +``` + +```ts type-equiv +interface StructuredSchemaNode { + type: StructuredSchemaType + properties?: Record + required?: string[] + additionalProperties?: boolean + items?: StructuredSchemaNode + enum?: StructuredScalar[] + const?: StructuredScalar + description?: string + title?: string + default?: unknown + examples?: unknown +} +``` + +A schema is an object-rooted node (`enum`/`const` are scalar-only; `description`/`title`/`default`/`examples` are annotations, allowed and ignored but still required to be JSON data — they ride the wire): + +```ts type-equiv +type StructuredOutputSchema = StructuredSchemaNode & { type: 'object' } +``` + ## Tool-presentation UI vocabulary How a tool wants its call shown in a UI (an editor tool-call card, a CLI log line), provider-neutral so a tool describes itself without depending on any client protocol. `presentCall`/`presentResult` return a **`card`-tagged render intent** — a discriminated union a UI bridge switches on: diff --git a/docs/core-data-structures/web.md b/docs/core-data-structures/web.md index 05c3f648c0..f6bd6406ab 100644 --- a/docs/core-data-structures/web.md +++ b/docs/core-data-structures/web.md @@ -91,4 +91,4 @@ Selection never depends on registration, config, or HMR order: a capability has ## The service -`WebService` (`ctx.web`, defined in [`packages/web/web/src/index.ts`](../../packages/web/web/src/index.ts)) is a provider registry plus a provider-selecting execution surface, close to `LlmService`'s shape: `registerSearchProvider`/`registerFetchProvider` (duplicate ids throw `WEB_DUPLICATE_PROVIDER`, return disposers) and `search`/`fetch` (resolve the provider at call time, throw a structured `WebError` when the capability cannot run). Providers issue requests with the platform-native `fetch` (Node 24), mirroring `dsh-llm-deepseek`; the `dsh-web-fetch-local` provider owns safe retrieval (http/https-only, credential rejection, byte/char/timeout/redirect caps, same-origin-only redirects with per-hop re-validation, charset decoding) while `dsh-tool-web` owns presentation (HTML→markdown). SSRF / private-network blocking is deferred (see the RFC) — until it lands, `web_fetch` must not be enabled where it can reach sensitive internal targets. +`WebService` (`ctx.web`, defined in [`packages/web/web/src/index.ts`](../../packages/web/web/src/index.ts)) is a provider registry plus a provider-selecting execution surface, close to `LlmService`'s shape: `registerSearchProvider`/`registerFetchProvider` (duplicate ids throw `WEB_DUPLICATE_PROVIDER`, return disposers) and `search`/`fetch` (resolve the provider at call time, throw a structured `WebError` when the capability cannot run). Providers issue requests with platform-native `fetch` at the repo's Node floor, mirroring `dsh-llm-deepseek`; the `dsh-web-fetch-local` provider owns safe retrieval (http/https-only, credential rejection, byte/char/timeout/redirect caps, same-origin-only redirects with per-hop re-validation, charset decoding) while `dsh-tool-web` owns presentation (HTML→markdown). SSRF / private-network blocking is deferred (see the RFC) — until it lands, `web_fetch` must not be enabled where it can reach sensitive internal targets. diff --git a/docs/development.i18n.yaml b/docs/development.i18n.yaml index 2ab3478a61..c1e557170d 100644 --- a/docs/development.i18n.yaml +++ b/docs/development.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -development.md: 97ca3f6b9fc9653ab658e480e6155fc1e121854f -development.zh.md: e837afb6a01ed4d0c4801886bd6ca6a7602ac573 +development.md: bd6f6b561480419abea7a42a44b4078e2c59b1cb +development.zh.md: 54bf19765d2b4dc419e6b71684dbcfcd28230541 diff --git a/docs/development.md b/docs/development.md index 97ca3f6b9f..bd6f6b5614 100644 --- a/docs/development.md +++ b/docs/development.md @@ -2,11 +2,11 @@ English | [中文](development.zh.md) -This guide covers the local setup needed to work on DeepSeek Harness and understand the local hooks, daily checks, and CI gates. +This onboarding guide helps project contributors get started with the local environment, daily workflow, and CI flow; see the RFCs for design rationale and technical trade-offs. ## Prerequisites -- Node.js 24 or newer. The repo declares `node >=24`; CI runs the matrix on Node 24 and 26. +- Node.js supports 22.19+ and 24+. CI covers 22.19, 24, and 26; see the [Node engine floor RFC](rfc/implemented/process/2026-07-06-node-engine-floor.md). - Corepack-enabled pnpm. The repo pins `pnpm@11.7.0` in `package.json`; run `corepack enable` if `pnpm --version` does not resolve through Corepack. - Git. - Optional: a DeepSeek API key for the REPL/ACP agent demos and real-API e2e tests. @@ -63,11 +63,11 @@ lefthook is configured in `lefthook.yml` as an early local checkpoint before rev The vendor manifest guard checks that changes under `vendor/*/src` are staged with the matching `vendor/README.md` manifest update. See `vendor/README.md` before editing vendored code. -These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests without coverage, while CI runs `pnpm run test:coverage`; CI also runs echo-agent and built-bin smoke tests and exercises the matrix on Node 24 and 26. +These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests without coverage, while CI runs `pnpm run test:coverage`; CI also runs echo-agent and built-bin smoke tests and exercises the compatibility matrix on Node 22.19, 24, and 26. ## CI gates -The keyless GitHub workflow has six jobs: five Node 24 lanes run static gates, lint, coverage, snapshot replay, and artifact gates separately, and the Node 26 compatibility job runs `pnpm run check:node-compat`. The lane schedulers fan out independent gates from `package.json`: constraints, typecheck, lint, coverage, snapshot replay, `doc-sync` members, module-graph freshness, `knip`, and the echo-agent smoke test. +The keyless GitHub workflow has eight jobs: five Node 24 lanes run static gates, lint, coverage, snapshot replay, and artifact gates separately, and three compatibility jobs run `pnpm run check:node-compat` on Node 22.19, 24, and 26. The lane schedulers fan out independent gates from `package.json`: constraints, typecheck, lint, coverage, snapshot replay, `doc-sync` members, module-graph freshness, `knip`, and the echo-agent smoke test. `pnpm run build` feeds the artifact lane, and `publint`, `verify-node-next-types`, and built-bin smoke tests wait for build output. The separate real-API workflow runs `pnpm run test:e2e` with a secret and `DSH_E2E_MAX_WORKERS=14`. @@ -85,6 +85,7 @@ pnpm run lint:fix # eslint . --fix pnpm run doc-typecheck # compile checked TypeScript snippets in Markdown docs pnpm run gen-cordis-catalog # regenerate docs/cordis-catalog/events.md + services.md from source pnpm run verify-cordis-catalog # fail if either cordis catalog is stale +pnpm run verify-export-jsdoc # fail if a module-level package export lacks complete JSDoc pnpm run gen-doc-graphs # regenerate generated relationship docs from source and curated graph definitions pnpm run verify-doc-graphs # fail if generated relationship docs are stale pnpm run gen-rfc-index # regenerate the docs/rfc/README.md index tables from the RFC tree diff --git a/docs/development.zh.md b/docs/development.zh.md index e837afb6a0..54bf19765d 100644 --- a/docs/development.zh.md +++ b/docs/development.zh.md @@ -2,11 +2,11 @@ [English](development.md) | 中文 -本指南覆盖参与 DeepSeek Harness 开发所需的本地环境搭建,并帮助你理解本地钩子、日常检查与 CI 门禁。 +本文面向参与项目开发的贡献者,帮助你上手本地环境、日常工作流和 CI 流程。相关设计考量和技术取舍参见 RFC,不在这里展开。 ## 前置条件 -- Node.js 24 或更新版本。仓库声明 `node >=24`;CI 在 Node 24 和 26 上跑矩阵。 +- Node.js 支持 22.19+ 和 24+。CI 覆盖 22.19、24、26;见 [Node engine floor RFC](rfc/implemented/process/2026-07-06-node-engine-floor.md)。 - 启用了 Corepack 的 pnpm。仓库在 `package.json` 中钉住 `pnpm@11.7.0`;如果 `pnpm --version` 无法通过 Corepack 解析,先运行 `corepack enable`。 - Git。 - 可选:一个 DeepSeek API key,用于 REPL/ACP agent(智能体)演示和真实 API 的 e2e 测试。 @@ -63,11 +63,11 @@ lefthook 在 `lefthook.yml` 中配置,作为评审前的本地早期检查点 vendor manifest 守卫检查 `vendor/*/src` 下的改动是否连同对应的 `vendor/README.md` manifest 更新一起暂存。编辑 vendor 代码前先看 `vendor/README.md`。 -这些钩子并不与 CI 完全一致。特别是:`pre-push` 跑不带覆盖率的单元测试,而 CI 跑 `pnpm run test:coverage`;CI 还会跑 echo-agent 和 built-bin 冒烟测试,并在 Node 24 和 26 上跑矩阵。 +这些钩子并不与 CI 完全一致。特别是:`pre-push` 跑不带覆盖率的单元测试,而 CI 跑 `pnpm run test:coverage`;CI 还会跑 echo-agent 和 built-bin 冒烟测试,并在 Node 22.19、24 和 26 上跑兼容性矩阵。 ## CI 门禁 -keyless GitHub 工作流有六个 job:五个 Node 24 lane 分别运行 static gates、lint、coverage、snapshot replay 和 artifact gates,Node 26 兼容性 job 运行 `pnpm run check:node-compat`。各 lane 调度器并发运行来自 `package.json` 的独立门禁:constraints、typecheck、lint、coverage、snapshot replay、`doc-sync` 成员、module graph 新鲜度、`knip` 和 echo-agent 冒烟测试。 +keyless GitHub 工作流有八个 job:五个 Node 24 lane 分别运行 static gates、lint、coverage、snapshot replay 和 artifact gates,三个兼容性 job 在 Node 22.19、24 和 26 上运行 `pnpm run check:node-compat`。各 lane 调度器并发运行来自 `package.json` 的独立门禁:constraints、typecheck、lint、coverage、snapshot replay、`doc-sync` 成员、module graph 新鲜度、`knip` 和 echo-agent 冒烟测试。 `pnpm run build` 供给 artifact lane,`publint`、`verify-node-next-types` 和 built-bin 冒烟测试等待 build 输出。单独的真实 API 工作流带密钥运行 `pnpm run test:e2e`,并设置 `DSH_E2E_MAX_WORKERS=14`。 @@ -85,6 +85,7 @@ pnpm run lint:fix # eslint . --fix pnpm run doc-typecheck # compile checked TypeScript snippets in Markdown docs pnpm run gen-cordis-catalog # regenerate docs/cordis-catalog/events.md + services.md from source pnpm run verify-cordis-catalog # fail if either cordis catalog is stale +pnpm run verify-export-jsdoc # fail if a module-level package export lacks complete JSDoc pnpm run gen-doc-graphs # regenerate generated relationship docs from source and curated graph definitions pnpm run verify-doc-graphs # fail if generated relationship docs are stale pnpm run gen-rfc-index # regenerate the docs/rfc/README.md index tables from the RFC tree diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 24a2793826..6174f7f77f 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -7,17 +7,17 @@ This matrix shows which packages dispatch each harness-owned event and which pac | Event | Mode | Declared in | Dispatchers | Listeners | | --- | --- | --- | --- | --- | -| `agent/created` | `emit` | [`packages/core/agent/src/types.ts:248`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) | -| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:255`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) | -| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:404`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:333`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | -| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:346`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:273`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:369`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:288`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:264`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`stdio-agent`](../packages/ui/stdio-agent) | -| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:379`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:392`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/created` | `emit` | [`packages/core/agent/src/types.ts:264`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) | +| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:271`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) | +| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:420`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:349`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | +| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:362`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | +| `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:289`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:385`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:304`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:280`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`stdio-agent`](../packages/ui/stdio-agent) | +| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:395`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:408`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:123`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:138`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:109`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | @@ -31,8 +31,9 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:91`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) | | `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:38`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | - | | `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:44`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - | -| `tools/change` | `emit` | [`packages/core/tools/src/index.ts:87`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - | -| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:82`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `tools/pre-execute` | `waterfall` | [`packages/core/tools/src/index.ts:66`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `tools/change` | `emit` | [`packages/core/tools/src/index.ts:118`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - | +| `tools/execute` | `waterfall` | [`packages/core/tools/src/index.ts:97`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`timeout-policy`](../packages/timeout/timeout-policy) | +| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:113`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | +| `tools/pre-execute` | `waterfall` | [`packages/core/tools/src/index.ts:77`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | Maintenance mode: hybrid generated: Cordis event declarations and most producer/listener edges are AST-scanned; dynamic dispatch sites are classified in `scripts/gen-doc-graphs.ts`. diff --git a/docs/module-graph.md b/docs/module-graph.md index ed1cc592d4..1e4e70151f 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -9,6 +9,7 @@ Inter-package dependencies among the `@deepseek-ai/dsh-*` harness packages, deri flowchart TD subgraph group_util["packages/util"] pkg_brand["brand"] + pkg_timeout["timeout"] end subgraph group_llm["packages/llm"] pkg_llm["llm"] @@ -54,6 +55,9 @@ flowchart TD pkg_web_search_exa["web-search-exa"] pkg_web_search_perplexity["web-search-perplexity"] end + subgraph group_timeout["packages/timeout"] + pkg_timeout_policy["timeout-policy"] + end subgraph group_todo["packages/todo"] pkg_tool_todo["tool-todo"] end @@ -68,6 +72,7 @@ flowchart TD pkg_session_persistence_sqlite["session-persistence-sqlite"] end subgraph group_support["packages/support"] + pkg_acp_snapshot["acp-snapshot"] pkg_invariants["invariants"] pkg_llm_replay["llm-replay"] pkg_subagent_mock["subagent-mock"] @@ -78,14 +83,23 @@ flowchart TD pkg_app_boot["app-boot"] pkg_stdio_agent["stdio-agent"] end + subgraph group_code_runtime["packages/code-runtime"] + pkg_code_runtime["code-runtime"] + pkg_code_runtime_worker["code-runtime-worker"] + end + subgraph group_guard["packages/guard"] + pkg_repeat_tool_guard["repeat-tool-guard"] + end pkg_llm --> pkg_brand pkg_bash --> pkg_brand + pkg_code_runtime_worker --> pkg_code_runtime pkg_llm_deepseek --> pkg_llm pkg_llm_pi_ai --> pkg_llm pkg_session --> pkg_brand pkg_session --> pkg_llm pkg_system_prompt --> pkg_llm pkg_bash_local --> pkg_bash + pkg_bash_local --> pkg_timeout pkg_fs --> pkg_brand pkg_fs --> pkg_llm pkg_web --> pkg_llm @@ -97,6 +111,7 @@ flowchart TD pkg_fs_policy --> pkg_fs pkg_compact --> pkg_llm pkg_compact --> pkg_session + pkg_web_fetch_local --> pkg_timeout pkg_web_fetch_local --> pkg_web pkg_web_search_deepseek --> pkg_web pkg_web_search_exa --> pkg_web @@ -143,6 +158,9 @@ flowchart TD pkg_tool_web --> pkg_system_prompt pkg_tool_web --> pkg_tools pkg_tool_web --> pkg_web + pkg_timeout_policy --> pkg_llm + pkg_timeout_policy --> pkg_timeout + pkg_timeout_policy --> pkg_tools pkg_tool_todo --> pkg_agent pkg_tool_todo --> pkg_session pkg_tool_todo --> pkg_tools @@ -156,6 +174,8 @@ flowchart TD pkg_acp --> pkg_session pkg_acp --> pkg_session_persistence pkg_acp --> pkg_tools + pkg_repeat_tool_guard --> pkg_agent + pkg_repeat_tool_guard --> pkg_tools pkg_agent_core --> pkg_agent pkg_agent_core --> pkg_agent_loop pkg_agent_core --> pkg_invariants @@ -171,6 +191,8 @@ flowchart TD pkg_subagent_inprocess --> pkg_llm pkg_subagent_inprocess --> pkg_session pkg_subagent_inprocess --> pkg_subagent + pkg_subagent_inprocess --> pkg_system_prompt + pkg_subagent_inprocess --> pkg_tools pkg_tool_subagent --> pkg_agent pkg_tool_subagent --> pkg_llm pkg_tool_subagent --> pkg_subagent @@ -205,21 +227,25 @@ flowchart TD | Package | Group | Depends on | | --- | --- | --- | | [`brand`](../packages/util/brand) | `util` | — | +| [`timeout`](../packages/util/timeout) | `util` | — | +| [`acp-snapshot`](../packages/support/acp-snapshot) | `support` | — | | [`app-boot`](../packages/ui/app-boot) | `ui` | — | +| [`code-runtime`](../packages/code-runtime/code-runtime) | `code-runtime` | — | | [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand) | | [`bash`](../packages/bash/bash) | `bash` | [`brand`](../packages/util/brand) | +| [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime) | | [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) | | [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) | | [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm) | | [`system-prompt`](../packages/core/system-prompt) | `core` | [`llm`](../packages/llm/llm) | -| [`bash-local`](../packages/bash/bash-local) | `bash` | [`bash`](../packages/bash/bash) | +| [`bash-local`](../packages/bash/bash-local) | `bash` | [`bash`](../packages/bash/bash), [`timeout`](../packages/util/timeout) | | [`fs`](../packages/fs/fs) | `fs` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm) | | [`web`](../packages/web/web) | `web` | [`llm`](../packages/llm/llm) | | [`agent`](../packages/core/agent) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt) | | [`fs-local`](../packages/fs/fs-local) | `fs` | [`fs`](../packages/fs/fs) | | [`fs-policy`](../packages/fs/fs-policy) | `fs` | [`fs`](../packages/fs/fs) | | [`compact`](../packages/compact/compact) | `compact` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | -| [`web-fetch-local`](../packages/web/web-fetch-local) | `web` | [`web`](../packages/web/web) | +| [`web-fetch-local`](../packages/web/web-fetch-local) | `web` | [`timeout`](../packages/util/timeout), [`web`](../packages/web/web) | | [`web-search-deepseek`](../packages/web/web-search-deepseek) | `web` | [`web`](../packages/web/web) | | [`web-search-exa`](../packages/web/web-search-exa) | `web` | [`web`](../packages/web/web) | | [`web-search-perplexity`](../packages/web/web-search-perplexity) | `web` | [`web`](../packages/web/web) | @@ -236,12 +262,14 @@ flowchart TD | [`tool-fs`](../packages/fs/tool-fs) | `fs` | [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`subagent`](../packages/subagent/subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`tools`](../packages/core/tools) | | [`tool-web`](../packages/web/tool-web) | `web` | [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`web`](../packages/web/web) | +| [`timeout-policy`](../packages/timeout/timeout-policy) | `timeout` | [`llm`](../packages/llm/llm), [`timeout`](../packages/util/timeout), [`tools`](../packages/core/tools) | | [`tool-todo`](../packages/todo/tool-todo) | `todo` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | | [`hooks-codex`](../packages/hooks/hooks-codex) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | | [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`tools`](../packages/core/tools) | +| [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) | | [`agent-core`](../packages/core/agent-core) | `core` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tool-bash`](../packages/bash/tool-bash), [`tools`](../packages/core/tools) | | [`subagent-acp`](../packages/subagent/subagent-acp) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent) | -| [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent) | +| [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`tool-subagent`](../packages/subagent/tool-subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools) | | [`hooks-claude`](../packages/hooks/hooks-claude) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools) | | [`subagent-mock`](../packages/support/subagent-mock) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent) | diff --git a/docs/persistence-catalog.md b/docs/persistence-catalog.md index c29cbc0d99..58fd548665 100644 --- a/docs/persistence-catalog.md +++ b/docs/persistence-catalog.md @@ -23,7 +23,7 @@ Raw stream chunk — token-level replay fidelity. Types: [StreamChunk](core-data-structures/llm-streaming.md) -Source: [`packages/core/session/src/types.ts:298`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:304`](../packages/core/session/src/types.ts) #### `assistant/message` — surface @@ -35,7 +35,7 @@ Assembled assistant message for one step (derived history uses this). Carries th Types: [ContentBlock](core-data-structures/core.md) · [TokenUsage](core-data-structures/llm-streaming.md) -Source: [`packages/core/session/src/types.ts:305`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:311`](../packages/core/session/src/types.ts) ### `compact/*` @@ -83,7 +83,7 @@ In-session context injection (file-change notices, subdir AGENTS.md, skill conte Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:296`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:302`](../packages/core/session/src/types.ts) ### `hook/*` @@ -119,7 +119,7 @@ A queued prompt an `agent/prompt-submit` listener VETOED — the durable record Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:290`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:296`](../packages/core/session/src/types.ts) ### `request/*` @@ -131,7 +131,7 @@ Full snapshot of the EpochHeader the NEXT request is built under, with the Reque 'request/header': { header: EpochHeader; reason: RequestHeaderReason } ``` -Source: [`packages/core/session/src/types.ts:350`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:356`](../packages/core/session/src/types.ts) #### `request/header-delta` — log-only @@ -141,7 +141,7 @@ Amendment to the folded EpochHeader: at least one of a SystemDelta, a ToolsDelta 'request/header-delta': { system?: SystemDelta; tools?: ToolsDelta; config?: LlmCallConfig } ``` -Source: [`packages/core/session/src/types.ts:361`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:367`](../packages/core/session/src/types.ts) ### `steering/*` @@ -155,7 +155,7 @@ Steering content injected between steps of a running turn. Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:323`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:329`](../packages/core/session/src/types.ts) ### `step/*` @@ -167,7 +167,7 @@ Closes step `step` of turn `turn`. 'step/end': { turn: number; step: number } ``` -Source: [`packages/core/session/src/types.ts:277`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:283`](../packages/core/session/src/types.ts) #### `step/start` — log-only @@ -177,7 +177,7 @@ Opens step `step` of turn `turn` — one model call plus the tool executions it 'step/start': { turn: number; step: number } ``` -Source: [`packages/core/session/src/types.ts:275`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:281`](../packages/core/session/src/types.ts) ### `todo/*` @@ -193,7 +193,7 @@ NOT a SurfaceEventType: it produces no LLM message and never reaches `deriveMess Types: [TodoItem](core-data-structures/session.md) -Source: [`packages/core/session/src/types.ts:337`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:343`](../packages/core/session/src/types.ts) ### `tool/*` @@ -207,7 +207,7 @@ The model requested one tool invocation: `name` with the raw `arguments` JSON st Types: [CallId](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:311`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:317`](../packages/core/session/src/types.ts) #### `tool/result` — surface @@ -219,7 +219,7 @@ A completed tool call's model-facing result, plus an optional tool-private `meta Types: [CallId](core-data-structures/core.md) · [ContentBlock](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:321`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:327`](../packages/core/session/src/types.ts) ### `turn/*` @@ -233,7 +233,7 @@ Closes turn `turn` with the TurnEndReason that ended it. The loop fires the awai Types: [TurnEndReason](core-data-structures/session.md) -Source: [`packages/core/session/src/types.ts:273`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:279`](../packages/core/session/src/types.ts) #### `turn/start` — log-only @@ -245,7 +245,7 @@ Opens turn `turn`. `trigger` records what started it — a drained message batch Types: [TurnTrigger](core-data-structures/session.md) -Source: [`packages/core/session/src/types.ts:267`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:273`](../packages/core/session/src/types.ts) ### `user/*` @@ -259,4 +259,4 @@ A user-visible prompt (queued message drained at turn start). Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:279`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:285`](../packages/core/session/src/types.ts) diff --git a/docs/rfc/INDEX.md b/docs/rfc/INDEX.md index 7a47ffe816..45b8f46183 100644 --- a/docs/rfc/INDEX.md +++ b/docs/rfc/INDEX.md @@ -10,8 +10,9 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; |---|---| | [Agent Client Protocol (ACP) support — drive the coding agent from external editors](proposed/feature/2026-06-14-acp-agent-client-protocol.md) | 2026-06-14 | | [Multiplex concurrent ACP sessions over one connection](proposed/feature/2026-06-14-acp-multi-session.md) | 2026-06-14 | -| [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/feature/2026-06-15-optional-code-mode.md) | 2026-06-15 | +| [Code Mode — the model writes TypeScript against the tool registry](proposed/feature/2026-06-15-code-mode.md) | 2026-06-15 | | [Pre-tool input rewrite — a consistent design](proposed/feature/2026-06-30-pre-tool-input-rewrite.md) | 2026-06-30 | +| [Claude Code and Codex subagent backends (out-of-process delegation to external coding agents)](proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md) | 2026-07-07 | | [Plan mode — a logged per-agent session mode](proposed/feature/2026-07-07-plan-mode.md) | 2026-07-07 | ### Simplification @@ -59,7 +60,10 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [dsh-hooks-claude + dsh-hooks-codex — the Claude Code / Codex hook bridges](implemented/feature/2026-06-30-hook-bridges.md) | 2026-06-30 | | [dsh-hook-protocol — the shared Claude Code / Codex hook wire-protocol core](implemented/feature/2026-06-30-hook-protocol-lib.md) | 2026-06-30 | | [Interception seams — the typed-Decision surface a hook programs against](implemented/feature/2026-06-30-interception-seams.md) | 2026-06-30 | +| [SessionStore fork API](implemented/feature/2026-06-30-session-store-fork-api.md) | 2026-06-30 | | [Subagent lifecycle enrichment — lastAssistantMessage (observe-only)](implemented/feature/2026-06-30-subagent-observe-enrich.md) | 2026-06-30 | +| [Explicit model-facing tool order](implemented/feature/2026-07-06-explicit-tool-order.md) | 2026-07-06 | +| [Repeat-tool-call guard plugin](implemented/feature/2026-07-08-repeat-tool-guard.md) | 2026-07-08 | ### Simplification @@ -120,6 +124,8 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Prompt variables and tool-guidance ownership](implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md) | 2026-07-05 | | [Every LLM request is reconstructable from the session log](implemented/architecture/2026-07-05-reconstructable-requests.md) | 2026-07-05 | | [Subagent provider-lifecycle events — `subagent/provider-added` / `subagent/provider-removed`](implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.md) | 2026-07-05 | +| [A shared timeout/deadline primitive, with hard-kill left to each capability](implemented/architecture/2026-07-06-timeout-deadline-library.md) | 2026-07-06 | +| [Tool-call timeout policy as a plugin](implemented/architecture/2026-07-07-tool-call-timeout-policy.md) | 2026-07-07 | ### Process @@ -143,7 +149,9 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Generate the RFC index tables](implemented/process/2026-07-04-generate-rfc-index-tables.md) | 2026-07-04 | | [Generated persistence log event catalog](implemented/process/2026-07-04-persistence-log-catalog.md) | 2026-07-04 | | [One gated in-file format for RFCs](implemented/process/2026-07-05-uniform-rfc-format.md) | 2026-07-05 | +| [Export-surface JSDoc gate](implemented/process/2026-07-06-export-surface-jsdoc-gate.md) | 2026-07-06 | | [Generated plugin config catalog](implemented/process/2026-07-06-generated-config-catalog.md) | 2026-07-06 | +| [Raise the Node LTS engine floor to 22.19](implemented/process/2026-07-06-node-engine-floor.md) | 2026-07-06 | | [Parallel GitHub CI gates](implemented/process/2026-07-06-parallel-github-ci-gates.md) | 2026-07-06 | | [Parallel pre-push gates](implemented/process/2026-07-06-parallel-pre-push-gates.md) | 2026-07-06 | @@ -161,6 +169,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Hook snapshot matrix — end-to-end goldens for both bridges](implemented/testing/2026-07-04-hook-snapshot-matrix.md) | 2026-07-04 | | [Single-source the acp-agent replay config](implemented/testing/2026-07-04-single-source-acp-replay-config.md) | 2026-07-04 | | [Pin request-header content in one snapshot scenario](implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md) | 2026-07-06 | +| [Extract the ACP snapshot suite into a support package](implemented/testing/2026-07-08-shared-acp-snapshot-package.md) | 2026-07-08 | ## Rejected diff --git a/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md b/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md index a03df16c8c..660b9821ff 100644 --- a/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md +++ b/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md @@ -66,7 +66,7 @@ flowchart LR `@deepseek-ai/dsh-web` depends only on Cordis and low-level harness support. It declares `ctx.web`, provider interfaces, request/result types, the provider status type, and error codes. It does not import tool, agent, session, LLM, or provider packages. -Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credentials, endpoint config, provider-specific request mapping, provider-specific response parsing, and provider-specific error translation into `WebError`. They issue network requests with the platform-native `fetch` (Node 24), mirroring `@deepseek-ai/dsh-llm-deepseek`'s adapter, NOT a cordis HTTP-client service (`ctx.http`/`@cordisjs/plugin-http`) — even where a Perplexity provider's request is shaped like an OpenAI-compatible chat completion, that wire shape is a provider-private detail and does not make the provider depend on `ctx.llm`. A provider does NOT own the `ctx.web` key (two search providers cannot both own it): like `dsh-llm-deepseek`, each provider package is a function/namespace plugin (`inject: ['web']`) whose `apply` constructs the backend and calls `ctx.web.registerSearchProvider` / `registerFetchProvider`. `@deepseek-ai/dsh-web` is the `export default` service that owns the key. +Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credentials, endpoint config, provider-specific request mapping, provider-specific response parsing, and provider-specific error translation into `WebError`. They issue network requests with platform-native `fetch` at the repo's Node floor, mirroring `@deepseek-ai/dsh-llm-deepseek`'s adapter, NOT a cordis HTTP-client service (`ctx.http`/`@cordisjs/plugin-http`) — even where a Perplexity provider's request is shaped like an OpenAI-compatible chat completion, that wire shape is a provider-private detail and does not make the provider depend on `ctx.llm`. A provider does NOT own the `ctx.web` key (two search providers cannot both own it): like `dsh-llm-deepseek`, each provider package is a function/namespace plugin (`inject: ['web']`) whose `apply` constructs the backend and calls `ctx.web.registerSearchProvider` / `registerFetchProvider`. `@deepseek-ai/dsh-web` is the `export default` service that owns the key. `@deepseek-ai/dsh-tool-web` depends on `@deepseek-ai/dsh-web`, `@deepseek-ai/dsh-tools`, `@deepseek-ai/dsh-system-prompt`, and Cordis. It never imports concrete provider packages. diff --git a/docs/rfc/implemented/architecture/2026-07-06-timeout-deadline-library.md b/docs/rfc/implemented/architecture/2026-07-06-timeout-deadline-library.md new file mode 100644 index 0000000000..4902a0e833 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-07-06-timeout-deadline-library.md @@ -0,0 +1,98 @@ +# RFC: A shared timeout/deadline primitive, with hard-kill left to each capability + +Status: implemented + +## Problem + +Timeout handling was drifting apart across the tool-bearing capabilities, and the divergence was not superficial — it was the same logic re-implemented three ways, each with its own subtle correctness burden. + +- **bash** ([packages/bash/bash-local/src/run.ts](../../../../packages/bash/bash-local/src/run.ts)) had a full, correct timeout inside the process plumbing: a config-clamped `timeoutMs`, two independent triggers — a `killTimer` for the timeout and an `onAbort` listener for upstream cancellation — each calling one `kill()` closure that escalates SIGTERM→grace→SIGKILL on the process group, and two orthogonal outcome booleans (`timedOut`, `aborted`) latched independently. +- **web_fetch** ([packages/web/web-fetch-local/src/provider.ts](../../../../packages/web/web-fetch-local/src/provider.ts)) had a correct but *hand-rolled* timeout: it constructed an `AbortController`, wired `setTimeout(() => controller.abort(new WebError(…, 'WEB_FETCH_TIMEOUT')))`, manually added and removed the upstream-signal listener, cleared the timer in a `finally`, and recovered the timeout reason from `signal.reason` in a `translateAbortOrNetwork` helper because the reader surfaces a bare `AbortError`. +- **web_search** ([packages/web/tool-web/src/search.ts](../../../../packages/web/tool-web/src/search.ts)) had **no timeout at all**: `WebSearchRequest` ([packages/web/web/src/types.ts](../../../../packages/web/web/src/types.ts)) carries no `timeoutMs` field, and each provider's `search()` only forwards `exec.signal`. (web_search stays untimed here — see Consequences.) + +Each new external-process or network tool re-derived the same four things — clamp the requested value, start a timer, fuse the timeout with upstream cancellation, and distinguish "timed out" from "cancelled" on the way out — and the fusion and reason-recovery are exactly the parts that are easy to get subtly wrong (web_fetch's `signal.reason` dance is evidence). At the same time, the *termination* each performs is irreducibly different: bash kills an OS process group (work runs in a child process, outside this runtime, reachable only by signal), while web aborts an in-process `fetch` (undici tears down the socket). There is no single mechanism that can stop all of them. + +The two reference agents surveyed converged on the same split. Codex models "what will end this exec early" as one value (`ExecExpiration`, an enum fusing timeout and a cancellation token) whose `wait_with_outcome()` returns `TimedOut | Cancelled`, while the actual `kill_process_group` lives outside it — and that abstraction is reused *only* across the exec family, with MCP, model-stream, and guardian each keeping their own bespoke `tokio::time::timeout`. Claude Code shares nothing: bash and ripgrep each own a private SIGTERM→SIGKILL kill and distinguish timeout from cancellation by throwing distinct error types, while file I/O has no timeout. Both confirm the boundary drawn here: the timing-and-classification half is worth sharing within a family of like-terminated operations; the termination half is not shareable and stays in each capability. + +## Decision + +`@deepseek-ai/dsh-timeout` lives under `packages/util/` (peer to `dsh-brand`) and owns the *timing and classification* half of timeout; the *termination* half — the hard kill — stays in each capability's implementation. It is a library of pure functions, **not** a cordis service or plugin: it takes no `ctx`, registers nothing, holds no cross-call state, and emits no events. There is deliberately no central "timeout service" that would have to know how to stop every capability's work — that knowledge is exactly what a microkernel keeps out of shared layers, and what Codex's exec-only `ExecExpiration` scope demonstrates. + +### The library surface + +Three functions plus one reason type: + +```ts ignore-check +/** The internal reason attached to a timeout abort, so consumers can classify it after the fact. */ +export class TimeoutReason extends Error { + override name = 'TimeoutReason' + + constructor(readonly code: string, readonly timeoutMs: number) { + super(`${code} after ${timeoutMs}ms`) + } +} + +/** Validate/fill a caller's optional positive hint from the backend's default, then cap at its max. */ +export function clampTimeout( + requested: number | undefined, + def: number, + max: number, + name = 'timeoutMs', +): number + +/** + * Build a deadline signal that aborts on upstream cancellation OR on timeout, + * with the timeout carrying a `TimeoutReason`. `timeoutMs <= 0` means "no + * timeout" (background tasks): forward only the upstream signal, arm no timer. + * The returned object's `[Symbol.dispose]` clears the timer — `using` for a + * scope-lifetime consumer, a manual call for an event-lifetime one. + */ +export function deadline( + upstream: AbortSignal | undefined, + timeoutMs: number, + code: string, +): { signal: AbortSignal; [Symbol.dispose](): void } + +/** Recover the TimeoutReason from an aborted signal (or error); `code` scopes the match to this deadline's timer. */ +export function timeoutOf(x: AbortSignal | { reason?: unknown }, code?: string): TimeoutReason | undefined +``` + +`deadline` is `AbortSignal.any([upstream, ])` with three things the standard library does not give: a typed, identifiable `TimeoutReason` on the timeout abort (native `AbortSignal.timeout()` yields a fixed `TimeoutError`, indistinguishable across timeout kinds), an internal `timeoutMs <= 0` "no timeout" sentinel for backend-owned background work, and a `Symbol.dispose` cleanup that works with both `using` and manual disposal. `AbortSignal.any` is a Node ≥ 20 primitive; it is the single mechanism that fuses two abort sources into one, adopting the reason of whichever fires first. External request hints validate as positive finite numbers via `clampTimeout` before they reach `deadline`; `0` is not a model-/plugin-facing "disable timeout" value. When `timeoutMs <= 0` and no upstream signal is present, `deadline()` returns a never-aborting signal plus a no-op disposer so callers keep one call shape. `TimeoutReason` is an internal classification reason: providers translate it into seam-specific public errors or result fields before returning to callers. `timeoutOf`'s optional `code` scopes classification to the caller's own deadline: when the `upstream` is itself a deadline (a future `tools/execute` middleware arming a per-call deadline), `AbortSignal.any` preserves the outer `TimeoutReason` if it fires first, and an unscoped match would misreport the outer timeout as the inner capability's own; scoping to `code` reads a foreign timeout as an ordinary upstream cancel. + +### The division of labor + +| Concern | Owner | +|---|---| +| Validate request hint and clamp default/max | `dsh-timeout` (`clampTimeout`) — pure arithmetic plus the shared positive-finite request contract | +| Arm timer, abort on deadline, carry reason, fuse with upstream cancel | `dsh-timeout` (`deadline`) | +| Clear the timer | `dsh-timeout` (`[Symbol.dispose]`) | +| Classify the first abort reason after abort | `dsh-timeout` (`timeoutOf`) | +| **Actually terminate the work** | the capability's implementation | +| The default/max *values* | the capability's config | +| The timeout `code` string | the capability (`WEB_FETCH_TIMEOUT` ≠ `BASH_TIMEOUT`) | + +The signal only *notifies*; termination is always the listener's job, and the listener differs by capability. bash writes its own `addEventListener('abort', kill)` because the OS process lives outside this runtime and nothing else will kill it; web hands `d.signal` to `fetch` and undici tears down the socket. This is why file read/write/edit take **no** `timeoutMs`: a local syscall is best-effort-abortable at most, a timeout could not force `fsync`/`rename` to stop, and adding one would be an implicit default that violates explicit-over-implicit. Both reference agents leave file I/O untimed for the same reason. + +### How each capability consumes it + +- **web_fetch** — the tool stays validate-and-forward; the provider's hand-rolled controller + `setTimeout` + manual listener + `finally` + `signal.reason` recovery is replaced by provider-owned `deadline`/`timeoutOf`. A pre-aborted upstream signal still throws `WEB_ABORTED` up front; otherwise `fetch` runs against the fused `d.signal`, and `translateAbortOrNetwork` classifies a thrown error by the signal (`timeoutOf` → `WEB_FETCH_TIMEOUT`, else aborted → `WEB_ABORTED`, else network → `WEB_PROVIDER_ERROR`). The public error-code contract is unchanged, and `TimeoutReason` never crosses the web seam as the public error. +- **bash** — `resolve()` stays a pure request-to-spec step: it clamps with `clampTimeout(request.timeoutMs, config.timeoutMs, config.maxTimeoutMs, 'bash-local: request.timeoutMs')` and carries `request.signal` through unchanged. Foreground `run()` owns the timeout: `using d = deadline(spec.signal, spec.timeoutMs, 'BASH_TIMEOUT')`, then `runBash` receives only `d.signal`. `runBash` no longer owns any timer — it listens for abort and runs its existing SIGTERM→grace→SIGKILL process-group kill, and its `SpawnSpec`/`SpawnOutcome` no longer carry `timeoutMs`/`timedOut`/`aborted` (the executor classifies from the deadline signal instead). `run()` computes `timedOut = timeoutOf(d.signal, 'BASH_TIMEOUT') !== undefined` and `aborted = d.signal.aborted && !timedOut`, so the public seam booleans (`BashRunResult.timedOut`/`aborted`) are mutually exclusive — the shared deadline reports the cause that first cut the command short, and the `code` scope keeps a nested outer deadline from being misread as bash's own timeout. Background `start()` creates no deadline and forwards only the upstream signal, so background tasks stay timeout-free; a task's killed-vs-completed status reads its own `spec.signal.aborted`. + +## Consequences + +- `runBash`'s outcome no longer independently latches `timedOut` and `aborted`; a timeout and a user abort racing before process close now report a single first-abort cause instead of both being true. The uniform SIGTERM→grace→SIGKILL kill is unchanged, and the seam type `BashRunResult` keeps both booleans (now mutually exclusive), so `dsh-tool-bash`'s result rendering is untouched. +- `SpawnSpec.timeoutMs` and `SpawnOutcome.timedOut`/`aborted` were removed rather than kept as always-zero/always-false vestiges: with `runBash` owning no timer and the executor owning classification, they were read nowhere. This is the one deviation from the literal proposal shape (which passed `timeoutMs: 0` into `runBash`); an always-0 field read by nothing is dead weight under the per-file coverage gate. +- web_fetch shed its bespoke controller/timer/listener/reason-recovery; the classifier now keys off the deadline signal (`timeoutOf` + `aborted`) rather than the thrown error's shape, which is robust across both the request-phase reject-with-reason and the read-phase bare-`AbortError`. +- `AbortSignal.any` and `using`/`Symbol.dispose` enter the repo for the first time here (Node ≥ 24 baseline, already met). + +Out of scope, named to mark the boundary: `web_search` can gain an optional model-facing `timeout_ms` once its tool-schema/snapshot coverage is planned; future ripgrep-backed fs discovery tools can consume the same provider-owned deadline shape once they exist; a `tools/execute` waterfall middleware could arm a default deadline for every tool call by driving `exec.signal` — that would be a plugin that *consumes* this library and still only notifies, the hard kill remaining each capability's job. + +## Alternatives considered + +**A unified timeout *plugin* / `ctx.timeout` service.** Rejected on microkernel grounds. A service that could stop any tool's work would have to understand every capability's termination mechanism (process-group SIGKILL, socket teardown, syscall-boundary checks) — the "kernel knows too much" the architecture forbids. Codex's `ExecExpiration` is scoped to the exec family precisely because the kill it drives (`killpg`) is process-family-specific; MCP and model-stream keep their own. There is no coherent middle layer that owns termination for everything, so the shared piece can only be the pure timing/classification half — a library, not a service. + +**Per-tool ad-hoc timeout, no shared code (the prior status quo, and Claude Code's choice).** Rejected because it was already producing divergence and duplicated correctness burden: web_fetch hand-rolled the exact controller/reason logic that future network/process-backed tools would each have to re-derive, and the fusion + `signal.reason` recovery are the error-prone parts. Claude Code tolerates full duplication; this repo has a single shared abort channel (`exec.signal` on every `execute`) that makes a small shared primitive strictly cleaner, so the cost/benefit differs. + +**A `withTimeout(promise, ms)` wrapper instead of a signal factory.** Rejected because racing a promise against a timer resolves the *tool-call* promise on deadline without stopping the underlying work — the child process or fetch socket leaks on. Handing out a signal and requiring the capability to listen is what forces a real termination path to exist. This mirrors the "dispose must reach quiescence, not just request it" defensive rule. + +**Keep bash's two independent triggers (`killTimer` + `onAbort`) rather than fusing.** Rejected for the convergence goal: fusing into one `deadline` signal removes bash's bespoke timer and gives every capability one shape. The trade-off is that bash's `timedOut`/`aborted` booleans become first-abort classifications rather than independent facts that can both be true when timeout and user abort race before process close. That is acceptable because the result reports the cause that first cut the command short; the termination action stays the same uniform SIGTERM→grace→SIGKILL kill. Note the deliberate non-alignment with Codex: Codex forks its kill by outcome (timeout → immediate SIGKILL; cancel → SIGTERM + 50 ms grace → SIGKILL), whereas the fused signal drives one uniform `kill()` for both, matching Claude Code's unified bash kill. Splitting the kill by `timeoutOf` is possible later if a need appears; there is none now. diff --git a/docs/rfc/implemented/architecture/2026-07-07-tool-call-timeout-policy.md b/docs/rfc/implemented/architecture/2026-07-07-tool-call-timeout-policy.md new file mode 100644 index 0000000000..362f5cb5e8 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-07-07-tool-call-timeout-policy.md @@ -0,0 +1,110 @@ +# RFC: Tool-call timeout policy as a plugin + +Status: implemented + +## Problem + +The [timeout/deadline RFC](2026-07-06-timeout-deadline-library.md) extracted the timing-and-classification primitive into `@deepseek-ai/dsh-timeout`, but timeout policy was still attached to individual capabilities and model-facing schemas. `bash` exposed `timeoutMs`; `web_fetch` exposed `timeout_ms`; `web_search` had no model-facing timeout even though providers already honor `exec.signal`; a future grep/glob tool would either import the timeout library directly or invent its own timeout policy. That is the wrong authoring shape for a plugin SDK: a tool author should normally forward `exec.signal` to the implementation it calls, and deployment policy should decide the budget. + +At the same time, not every timeout in the repo is a model-facing tool-call budget. Hooks execute command hooks by calling `ctx.bash` directly, not through `ctx.tools.execute()`, and the `bash` model tool multiplexes foreground execution, background start, background polling, and hook reuse through the same backend. Moving every timeout into a tool plugin in one step would conflate those paths and risk breaking hook timeout semantics. + +## Decision + +Tool-call timeout is a policy that applies only to model-facing tool execution, in three parts: + +- `@deepseek-ai/dsh-timeout` remains the shared library that owns `deadline()` and `timeoutOf()`. +- `@deepseek-ai/dsh-tools` has an around-dispatch waterfall, `tools/execute`, between `tools/pre-execute` and `tools/post-execute`. +- `@deepseek-ai/dsh-timeout-policy` reads each tool's declared `timeoutMs` from the registry and wraps a call that has one by deriving a new `exec.signal`. + +The execution pipeline is: + +```text +ctx.tools.execute(exec) + -> tools/pre-execute + -> tools/execute + -> registry dispatch (the base next()) + -> tool.execute(args, exec) + -> thrown tool errors normalize to ToolExecutionResult + -> tools/post-execute +``` + +The default behavior is conservative: a tool that declares no `timeoutMs` receives no `TOOL_TIMEOUT` deadline from the plugin. + +### The `tools/execute` around seam + +`@deepseek-ai/dsh-tools` declares a `tools/execute` waterfall whose base `next()` is the dispatch-with-normalization thunk — the same inner `try`/`catch` that turns a thrown tool (or unknown tool) into an `isError` `ToolExecutionResult`. A listener receives `(exec, next)`: it calls `next()` to delegate to dispatch (returning its result, optionally wrapped) or returns a replacement result to short-circuit dispatch. The whole pipeline still sits inside `execute`'s outer try/catch, so a throwing listener becomes an `isError` result, never a turn failure. + +That the catch is the base `next` — not something outside the waterfall — is load-bearing: when a provider sees the timeout signal and throws its own upstream-abort error, registry dispatch first converts it to a normal error result, and only then can `timeout-policy` replace the final result with `TOOL_TIMEOUT`. + +### The `timeout-policy` plugin + +The plugin is `@deepseek-ai/dsh-timeout-policy`, a zero-config function/namespace plugin (`name` / `inject` / `apply`) in the `packages/timeout/` group. The per-tool budget is DECLARED on the tool, not on this plugin: a `ToolDefinition` carries an optional `timeoutMs`, which the owning tool plugin sets from its own config. `dsh-tool-web`, for example, resolves `fetchTimeoutMs` / `searchTimeoutMs` (default 30000) onto the `web_fetch` / `web_search` definitions: + +```yaml +- id: timeout-policy + name: '@deepseek-ai/dsh-timeout-policy' +- id: tool-web + name: '@deepseek-ai/dsh-tool-web' + config: + fetchTimeoutMs: 30000 + searchTimeoutMs: 30000 +``` + +Keeping the tool name out of this plugin's config is deliberate: a budget keyed by a free-text tool name could be mistyped (`web_fech`) and then silently apply to nothing. Declaring `timeoutMs` on the tool makes that failure class structurally impossible — the enforcer reads `ctx.tools.get(exec.name)?.timeoutMs`, and `exec.name` is the tool being dispatched, so the lookup always resolves and there is no unknown-name path to warn or throw about. `timeoutMs` is validated positive-finite by `defineTool` at definition time. For a tool that declares a budget the listener arms `deadline(exec.signal, timeoutMs, 'TOOL_TIMEOUT')`, swaps the derived signal onto `exec` for the downstream dispatch, restores the caller's own signal afterward, and returns a structured `TOOL_TIMEOUT` result when `timeoutOf(d.signal, 'TOOL_TIMEOUT')` matches. A tool with no declared budget delegates unchanged. + +Signal replacement is by **in-place mutation of `exec.signal`**, not by passing a new object to `next()`. Cordis's waterfall `next()` ignores any arguments handed to it and re-invokes downstream listeners with the shared payload array (`vendor/cordis/src/events.ts`), so the documented cordis idiom — mutate the shared object, then delegate — is the only mechanism that reaches dispatch. The plugin restores `exec.signal` to the caller's original in a `finally` so `tools/post-execute` never sees this plugin's (possibly already-aborted) deadline signal. + +`timeout-policy` owns both uses of the `TOOL_TIMEOUT` code: the internal deadline code passed to `deadline()`/`timeoutOf()` (scoped so a nested outer deadline reads as an ordinary cancel) and the structured tool-result error code. Its replacement result is: + +```ts ignore-check +function toolTimeoutResult(callId: CallId, timeoutMs: number): ToolExecutionResult { + return { + callId, + content: [{ type: 'text', text: `Error: tool call timed out after ${timeoutMs}ms` }], + isError: true, + error: { name: 'ToolTimeoutError', code: 'TOOL_TIMEOUT' }, + } +} +``` + +This is a cooperative deadline. It does not kill arbitrary work by racing the tool promise; the tool or the capability it calls must honor `exec.signal` and reach quiescence. Declaring `timeoutMs` therefore MEANS "this tool is cooperative with `exec.signal`", which the plugin README states as its contract. + +No new session event is needed for reconstructability: `TOOL_TIMEOUT` is the final model-facing `tool/result` for that call, so the existing session log already records the content and structured `{ name, code }` error the next model request sees. + +### Existing tool adaptation + +`web_fetch` and `web_search` are migrated. `dsh-tool-web` keeps ownership of their model-facing schemas, and those schemas expose no timeout knob: `web_fetch` dropped its `timeout_ms` parameter to match the reference-agent shape, and `web_search` stays query-only. The tool bodies do not import `@deepseek-ai/dsh-timeout`; they forward `exec.signal` to `ctx.web`. + +`dsh-web-fetch-local` keeps a provider-level timeout (`timeoutMs`/`maxTimeoutMs`) as a large resource backstop for direct `ctx.web.fetch()` callers and misconfigured deployments; it owns no model-facing timeout. When a `TOOL_TIMEOUT` signal reaches the fetch provider first, provider-scoped classification treats it as upstream `WEB_ABORTED`, and the outer `tools/execute` wrapper replaces the final tool result with `TOOL_TIMEOUT`. A shipped web-tool deployment configures the provider backstop above the `timeout-policy` budget so the tool-call policy normally wins for model calls. + +`bash` stays on the current backend timeout path. `dsh-tool-bash` continues to expose `timeoutMs` and `run_in_background`; `dsh-bash-local` continues to use `@deepseek-ai/dsh-timeout` for `BASH_TIMEOUT`; hook bridges continue to call `runHook()` and pass `timeoutMs` through `ctx.bash`. This keeps foreground/background/hook behavior stable. + +`read`, `write`, `edit`, `todo_write`, `bash_output`, and `bash_kill` do not opt into tool-call timeout: they are local filesystem or short registry/session operations where a deadline would be best-effort only or unnecessary. + +A future model-facing grep/glob tool can be implemented on top of `ctx.bash` without importing `@deepseek-ai/dsh-timeout`: it forwards `exec.signal` to `ctx.bash`, and declares its own `timeoutMs` (from its plugin's config) for the enforcer to apply. If bash-local's backend timeout becomes a problem for such a tool, the bash seam can later add a caller-owned-deadline mode; that is outside this cut. + +## Alternatives considered + +**Name the plugin `tool-timeout`.** The literal RFC name matched the `gen-tool-catalog` completeness guard's `packages/*/tool-*` glob, which requires every match to register a model-facing tool. This plugin registers none — it is a `tools/execute` wrapper — so a `tool-*` name would either fail `verify-tool-catalog` or force a misleading boot entry. The package is `@deepseek-ai/dsh-timeout-policy` in a new `packages/timeout/` group; the cordis.yml `id` can still be `timeout-policy`. + +**Keep per-tool timeout handling only.** This was the shape for `bash` and `web_fetch`, and it matches Claude Code and Codex for shell commands. It loses for web-style tools because every new timeout-capable tool must choose validation, cap semantics, docs, snapshots, and classification. The plugin centralizes policy and classification while leaving each tool's schema focused on business input. + +**Move all timeout policy out of bash-local immediately.** Cleaner long-term — bash-local would become a pure subprocess executor and all callers would own their deadlines. It loses as the first step because hooks call `ctx.bash` directly and the bash model tool has foreground/background semantics that are not the same tool-call lifetime. Keeping `BASH_TIMEOUT` preserves those paths while tool-call timeout proves itself on simpler tools. + +**Use a global default budget for every tool.** Convenient, but it surprises tool authors: any tool that accidentally runs longer than the global budget would start failing once the plugin loads. A per-tool declared budget makes adoption deliberate. + +**Expose a model-facing `timeout_ms` override.** Claude Code's `WebFetch`/`WebSearch` and Codex's web tools keep timeout out of the model-call shape. A model override would make timeout part of prompt semantics and force schema/argument-stripping rules into `timeout-policy`. Web timeout stays deployment policy only. + +**Let `timeout-policy` match tool arguments itself.** A rule engine such as "disable timeout when `bash.run_in_background` is true" would make the policy plugin know tool-specific argument semantics. Avoided by not migrating bash to tool-call timeout. + +**Use `tools/pre-execute` plus `tools/post-execute` instead of a new around seam.** A pre listener could arm a deadline and mutate `exec.signal`; a post listener could classify and replace. That loses because the deadline lifetime would cross two independent waterfalls: a call-id map, cleanup on every pre-deny/tool-throw/post-throw/dispose path, and ordering rules with every other listener. `tools/pre-execute` is also the allow/deny gate, not an execution wrapper. `tools/execute` gives the timeout one lexical scope: arm, delegate, classify, dispose. + +**Use `Promise.race` to enforce timeouts for non-cooperative tools.** Rejected for the same reason as the timeout-library RFC: it returns control to the caller while the underlying process, fetch, or provider operation may still be running. The plugin only sends a signal; termination remains the implementation's responsibility. + +## Consequences + +- `@deepseek-ai/dsh-tools` gains an around-dispatch surface after the interception seams deliberately split pre/post tool hooks. Its contract is narrow — wrap registry dispatch, not replace the pre-gate or post-result policy — and the base `next()` is dispatch-with-normalization so a wrapper never sees a raw tool throw. +- Multiple `tools/execute` listeners compose by ordinary Cordis waterfall order: a listener that calls `next()` wraps downstream listeners plus dispatch; one that returns without `next()` short-circuits them. A deployment combining timeout with a future retry/sandbox/metrics wrapper chooses semantics by registration order ("timeout covers the whole retry" vs "timeout covers each attempt"). +- Opt-in by declaration is a deliberate misconfiguration risk: a tool can declare a `timeoutMs` without honoring `exec.signal`, and that tool will not stop on timeout. The plugin contract states that declaring a budget means cooperative; the web tools prove the pattern on tools that already forward the signal. +- During the transition `bash` and the migrated web tools use different timeout paths on purpose: `TOOL_TIMEOUT` is the model-facing tool-call budget, while `BASH_TIMEOUT` remains the bash backend timeout used by bash and hooks. +- Deviation from the literal proposal, recorded per the implemented-RFC rule: the plugin package is `@deepseek-ai/dsh-timeout-policy` (not `tool-timeout`), signal replacement is in-place `exec.signal` mutation before `next()` (not `next({ ...exec, signal })`, which cordis ignores), and the per-tool budget is declared on the `ToolDefinition` (`timeoutMs`, set by the owning tool plugin from its config) rather than mapped by tool name in this plugin's config — so the enforcer is zero-config and a mistyped tool name is impossible. All three are described in `## Decision` above. diff --git a/docs/rfc/implemented/feature/2026-06-30-session-store-fork-api.md b/docs/rfc/implemented/feature/2026-06-30-session-store-fork-api.md new file mode 100644 index 0000000000..bf366afc02 --- /dev/null +++ b/docs/rfc/implemented/feature/2026-06-30-session-store-fork-api.md @@ -0,0 +1,41 @@ +# RFC: SessionStore fork API + +Status: implemented + +## Problem + +The event-sourced session log already has the primitive a fork needs: create a new session with a seed event prefix, then derive model history from that seeded log exactly as replay does. That primitive is intentionally low-level: `ctx.sessions.create(id, { seed, meta })` accepts any valid seed, but ordinary live-session branching needs policy around which prefix can be copied, which metadata is stamped on the child, and how errors are classified. + +The semantic hazard is the fork boundary. A valid user-visible fork seed must be contiguous and turn-enclosed. Forking inside an active turn would copy an open `turn/start`, possibly an open `step/start`, and possibly dangling tool calls. That violates turn-enclosure and provider-transcript invariants, and it creates a misleading child history that appears to have participated in an unfinished parent turn. The existing [subagent seam](../../implemented/feature/2026-06-21-subagent-capability-seam.md) deliberately solves a different problem: tool-triggered subagent forks usually happen while the parent turn is open, so `dsh-subagent-fork` clips the seed to the parent's last completed-turn prefix. A general session fork should not silently clip; it should either fork the requested boundary or reject it. + +## Decision + +`dsh-session` owns ordinary live-session forking directly on `ctx.sessions`. There is no separate `dsh-session-fork` package or `ctx.sessionFork` service: the API has no independent backend, event vocabulary, lifecycle, or persistence behavior, and all durable work delegates to the existing session store and persistence backends. + +The store exposes one operation: + +```ts ignore-check +type SessionForkSource = Session | SessionId + +class SessionStore extends Service { + fork(source: SessionForkSource, boundary?: number, childSessionId?: SessionId): Session +} +``` + +`boundary` is the inclusive source event `seq` to copy through. When omitted, it defaults to the source session's current last event; on an empty source, omitted `boundary` creates an empty child. Fork-specific validation only checks that the requested boundary exists and is a `turn/end`. The selected prefix is then deep-cloned into the child seed. The child inherits the source session's `cwd`, stamps `parentSession` to the source id, and sets `seedLength` to the copied prefix length. When `childSessionId` is omitted, `SessionStore` generates one using its existing id policy. + +The boundary rule is structural: an empty selected prefix is forkable, and any non-empty selected prefix must end at `turn/end`, regardless of the turn-end reason (`completed`, `aborted`, `error`, `disposed`, `max-tokens`, `interrupted`, or a future merge-extensible reason). A boundary that is not an existing event seq, is not a safe integer, or does not point at `turn/end` is rejected with a typed `SessionForkError` code. Broader session-log sanity remains in the existing invariant/repair layers: `dsh-invariants` checks turn enclosure and richer event ordering in dev, while persistence repair handles the valid crash-tail case of a final interrupted turn. The API also classifies non-live source ids (`SESSION_NOT_FOUND`), stale `Session` object references whose id is live on a different instance (`SESSION_NOT_LIVE`), duplicate requested child ids (`SESSION_ALREADY_EXISTS`), and invalid boundary values (`INVALID_BOUNDARY`). + +## Alternatives considered + +**Separate `ctx.sessionFork` service.** This was the first implementation, but review showed it overfit the capability-seam pattern. The code had no swappable backend, no extra event surface, no independent ownership lifecycle, and no durable behavior beyond `ctx.sessions.create({ seed, meta })`. Keeping a separate package would make callers discover and install a second service just to perform policy around a session-store primitive. + +**Two functions: `snapshot()` plus `fork()`.** This preserved a reusable seed/metadata computation, but the only supported consumer created a session immediately. It also made the surface feel more abstract than the concrete operation users need. A single `fork()` with an explicit `boundary` keeps the API direct while still supporting previous-point forks. + +**Silently clip open turns to the last completed boundary.** That is correct for `dsh-subagent-fork`, where delegation often starts while the parent turn is open and the child should inherit only the completed prefix. It is wrong for ordinary user/session branching because it hides that the requested fork point was not actually a valid boundary and silently drops the parent turn tail. + +## Consequences + +The public surface stays small and discoverable: live session branching is part of `ctx.sessions`, next to `create({ seed })`, rather than a standalone service or a two-step helper pair. Persistence continues to work through existing `session/created` and `session/flush` behavior: a forked child starts life with seeded events, so existing backends persist that seed once and preserve `parentSession` / `seedLength` in the header. + +The v1 scope still excludes ACP `session/fork`, unloaded persisted-session forking, model-facing tools, and subagent refactors. If a future ACP method is added, it should advertise the capability only after it has transcript/snapshot coverage; this RFC adds no editor-facing updates, so no ACP snapshot is required now. Fork-child replay remains covered by the existing [seed-boundary testing RFC](../../implemented/testing/2026-06-22-fork-child-replay-seed-boundary.md), while this API gets focused `dsh-session` unit tests plus JSONL persistence coverage. diff --git a/docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md b/docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md new file mode 100644 index 0000000000..3604e70972 --- /dev/null +++ b/docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md @@ -0,0 +1,49 @@ +# RFC: Explicit model-facing tool order + +Status: implemented + +## Problem + +The order of the tool list a model call carries — `request/header.tools` on the session log and `GenerateOptions.tools` on the wire — was an emergent artifact: the tool registry returns schemas in registration order, the system-prompt assembly concatenates providers in registration order, and the loop logged and dispatched the result verbatim. Registration order is plugin load order, and plugin load order is a race: the cordis loader imports every `cordis.yml` entry concurrently, so which tool plugin registers first depends on module-import completion timing. The plugin dependency relation cannot rescue this — it is a partial order under which independent tool plugins (e.g. `tool-subagent` vs `tool-todo`) are incomparable, so both interleavings are legal linearizations. This stopped being theoretical when a CI runner resolved the race differently from every recording machine: snapshot goldens pinned one permutation of `request/header.tools`, the `node 22.18` CI leg produced the other, and 5/5 snapshot tests failed on a diff that was pure array reordering. Tool order is part of the request bytes (prompt-cache stability, potentially model behavior) and, since the reconstructability contract, part of the durable session log — it must be a decision, not a residue. + +## Decision + +The system-prompt assembly owns the canonical model-facing tool order, exactly where it already owns section order. `toolOrder?: string[]` on `dsh-system-prompt` is the optional explicit policy: + +- A listed tool that is registered takes its listed position. +- A listed name with no registered tool is a configuration error. Shape errors (rest entry missing or duplicate names) fail from the service constructor; an unregistered name rejects every `assemble()` — the earliest moment the registered tool set exists to check against (tool plugins register after the service constructs), and the only universal one (registrations can change at any time; cordis has no "all plugins loaded" event). Under the shipped loop the first turn fails before any model request — see the consequences below for the exact blast radius. +- A registered tool absent from the list is inserted at the `''` rest entry (`TOOL_ORDER_REST`), in lexicographic name order among the other unlisted tools. +- No collected tool may use `TOOL_ORDER_REST` as its `ToolSchema.name`; the assembly rejects that reserved name before ordering. +- The list must contain the rest entry exactly once and no duplicate names. +- When `toolOrder` is unset, the canonical order is plain lexicographic name order (code-unit comparison, locale-independent), so determinism requires no configuration. + +The policy is applied where the list is born: `assemble()`, before the `system-prompt/assemble` waterfall. The assembly canonicalizes the tools it collects from providers the same way it sorts sections by their `order` field — on the initial assembly, killing the registration-order entropy at its source. Everything downstream inherits the order untouched: the waterfall, the loop's `EpochHeader`, the `request/header` event, the deep-frozen request, and the dev invariant's cross-check all see one deterministic list, with no new loop change. + +Scope is deliberately narrow: this fixes the REGISTRATION-ORDER race, not plugin behavior. A `system-prompt/assemble` listener may still add, remove, or rearrange tools — same as it may edit sections after their sort — and owns the determinism of what it emits; the waterfall contract already demands deterministic listeners (the reconstructability invariant would catch a listener that diverges between build and replay). + +Config plumbing follows the `persona` precedent, and `toolOrder` sits beside it: the app configs (`dsh-stdio-agent`, `dsh-acp-agent`) accept the key and forward it through `dsh-agent-core` (whose schema is the intersection of the owners' schemas) to the `SystemPrompt` child. One schemastery footnote is load-bearing: a schemastery array defaults to `[]`, but an omitted `toolOrder` must stay ABSENT (= lexicographic) rather than become an explicitly-configured empty list (invalid — it lacks the rest entry), so every schema on the chain forces the default to `undefined`. + +## Alternatives considered + +- **Registration order (the status quo)** — a concurrent-import race, host-dependent (the CI flake above), invisible in review. +- **A linearization of the plugin dependency graph** — the relation is partial and independent tool plugins are incomparable; the flake happened with the partial order fully satisfied. +- **Per-plugin `weight` on each tool contribution** — scatters the order across plugins yet still needs a global numbering convention nobody owns (the section `order` bands show that coordination cost being paid by hand). +- **Sorting in `ToolRegistry.schemas()` (the registry layer)** — equally deterministic, but the registry is a membership store consumed by more than the assembly; ordering is a prompt-composition concern, and the assembly already owns the composition policy for sections. +- **A `LlmService` config + `orderTools()` method the loop calls before logging the header** — works, but adds a public service method and a loop edit solely to apply a policy at a distance; every future request composer must remember the call. Canonicalizing where the list is born makes an unordered list unrepresentable, with zero new surface. +- **Normalizing inside `llm.stream()`** — runs after the header event is logged (the flake survives) and rebuilds the deep-frozen envelope, silently disarming the reconstruction invariant. +- **An exhaustive list (no rest entry)** — every newly loaded tool plugin would break boot; the mandatory rest entry keeps unlisted tools deterministic and their position explicit. +- **A boot-time validation pass (a `SystemPrompt.assertToolOrderSatisfied()` called by `dsh-app-boot` after `loader.await()`)** — would turn the misconfiguration into a startup death instead of a first-turn failure, but costs a public service method plus a structural coupling from the generic boot glue to one service, and cannot replace the assembly-time check anyway (embedded callers never run app boot; registrations change after boot). No existing event can host the check either: cordis v4 has no ready-like event, `loader/entry-init`/`internal/status` fire mid-load (racy against tool registration, the very entropy this RFC kills), and the agent lifecycle events are no earlier than the assembly. One enforcement point at `assemble()` was judged worth the later failure moment. + +## Consequences + +- Every assembly — and therefore every `request/header` event and model request — has a deterministic tool order on every host; the CI-vs-local golden flip is structurally gone. The default order is lexicographic, no longer registration order. +- `PromptAssembly.tools` itself is canonical, so every assembly consumer (the loop, waterfall listeners, any future prompt inspector) sees the model-facing order; provider registration order is observable nowhere downstream of the registry. +- The snapshot suite's single pinned request-header fixture (`text-turn`) carries the new canonical tool order; every other ACP snapshot keeps the header bulk scrubbed as `{{system}}`/`{{tools}}`, per the pinned-header design. +- A pure tool reordering between steps is representable only as a `request/header` `'fallback'` snapshot (the name-keyed `ToolsDelta` cannot express it); with a stable canonical order such reorders no longer occur in practice, so the fallback path stays a safety valve. +- The `toolOrder` key rides the app → `agent-core` → `SystemPrompt` forwarding chain, so deployments set it next to `persona` in the app config; `dsh-llm` and the agent loop are untouched. +- A misspelled or unloaded tool name in `toolOrder` fails the turn at prompt assembly, not the boot: the loop assembles inside the turn (after `turn/start`, before `step/start`), so the rejection reaches the turn's outer catch — the turn closes balanced with an `error` reason carrying the message, `agent/error` mirrors it, no step opens, no `request/header` is logged, no request reaches the adapter, and the agent returns to idle. Every turn fails identically until the config is fixed; the process itself stays up (matching the repo rule that explicit config references must not be silently ignored — the enforcement point is the assembly because no earlier universal moment exists). +- A tool provider that returns the reserved rest-entry name has the same prompt-assembly failure shape as an unknown listed name. This keeps the sentinel from becoming an ambiguous real tool and preserves the "never drops a tool" ordering contract. + +## Testing + +Unit tests on `dsh-system-prompt` pin the ordering semantics (lexicographic default, listed/rest placement, unknown-name rejection at assembly, reserved tool-name rejection, stable handling of shared names, provider-order independence), the pre-waterfall contract (listeners observe the canonical list; a listener-appended tool is not re-sorted), and each invalid-list rejection at load. Loop-level tests assert the `request/header` fold carries the canonical order for scrambled registration orders (identical across permutations), that a configured `toolOrder` reaches both the logged header and the dispatched deep-frozen request, that the frozen loop-built envelope survives to the adapter, and that an unregistered `toolOrder` name fails the turn with a balanced `error` `turn/end`, an `agent/error`, no step, no logged header, and no dispatched request. Forwarding is asserted at every level that exposes the key (`dsh-agent-core`, `dsh-stdio-agent`, `dsh-acp-agent`). The snapshot tier replays all scenarios while only the pinned `text-turn` header carries the full canonical tool list; non-pinning fixtures continue to compare through `{{tools}}`. diff --git a/docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md b/docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md new file mode 100644 index 0000000000..9d0446dcad --- /dev/null +++ b/docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md @@ -0,0 +1,73 @@ +# RFC: Repeat-tool-call guard plugin + +Status: implemented + +## Problem + +A model stuck in a loop re-issues the same tool call with byte-identical arguments — re-running a failing grep, re-reading an unchanged file, polling a command that already gave its answer — and each round trip burns tokens, wall-clock, and (for paid APIs) money without adding information. The harness has nothing that notices: the loop has no step budget, no plugin tracks call repetition, and the model only escapes when it happens to vary its own behavior. The failure mode is real and cheap to detect — [pi-repeat-tool-guard](https://github.com/Kingwl/pi-repeat-tool-guard) ships exactly this as a pi coding-agent extension: count consecutive identical calls and, past a threshold, append a `` telling the model to stop repeating itself and change course. + +The harness already has every seam the pi extension uses, and better ones: [the interception-seams RFC](2026-06-30-interception-seams.md) gives `tools/post-execute` a sanctioned way to attach model-facing context to a finished call, the loop buffers and injects that context with call/result adjacency preserved, and injected context is a logged `context/message` — so a native guard satisfies the model-visible ⟺ logged rule with no new session event. What was missing was only the plugin itself. + +## Decision + +The guard is a loop-hygiene plugin, not a model-facing tool: it never appears in the tool list, never vetoes or rewrites a call, and adds exactly one behavior — it watches each agent's stream of tool calls, counts runs of consecutive calls to the same tool with identical canonicalized arguments, and at configured run lengths injects an escalating advisory reminder telling the model to stop repeating itself, re-read the last result, and either change approach or conclude. The purpose is to break unproductive loops within a few wasted steps instead of letting them run to the turn's natural end — while leaving the decision (retry differently, gather more evidence, or finish) entirely with the model, so a legitimately repeated call is delayed by nothing and blocked by nothing. + +The plugin is `@deepseek-ai/dsh-repeat-tool-guard` at `packages/guard/repeat-tool-guard/`, opening the `guard/` group for loop-hygiene plugins (single-package groups have precedent: [the todo-write RFC](2026-06-29-todo-write-tool.md) shipped `todo/tool-todo`). It registers three listeners and holds all state in plugin-local maps keyed by `AgentId` — the tool registry is a context-level singleton whose waterfalls interleave every agent's calls (subagents run on the same context), so per-agent keying is correctness, not polish. + +- **`tools/post-execute` (waterfall)** — the one detection point. The listener receives `(exec, result)` together, so counting and reminder delivery need no cross-event pending map (the pi extension needs one only because its `tool_call`/`tool_result` hooks are separate events). It always delegates via `next()` and, when a threshold is hit, folds a reminder onto the downstream decision's `additionalContext` — the observe-and-enrich posture [the hooks bridges](2026-06-30-hook-bridges.md) already use, honoring the waterfall contract. Counting happens here rather than in `tools/pre-execute` because post-execute also runs for denied calls (`ToolRegistry.execute` routes a deny through the same pipeline), and a model hammering a denied call is exactly the loop worth breaking. +- **`agent/prompt-submit` (waterfall)** — pure reset hook: delegate via `next()`, clear the submitting agent's chain. A user interjection changes the context; repetition across it is not a loop. +- **`agent/status` (emit)** — on `disposed`, drop the agent's state, bounding the maps over harness lifetime. + +### Detection semantics + +The chain key is `(tool name, canonical arguments)`; a call identical to the previous tracked call increments the agent's consecutive counter, a different tracked call resets it to 1. Canonicalization is a deep key-sort plus `JSON.stringify`: `ToolExecution.arguments` is by construction the loop's `JSON.parse` output (or the raw string fallback for malformed argument JSON, which is itself a comparable value), so the pi original's bigint/circular/`undefined` handling has no inputs here and is deliberately dropped. + +Two deliberate rules, both documented in [the package README](../../../../packages/guard/repeat-tool-guard/README.md) because they are behavior a reader would otherwise guess at: + +- **Untracked calls are transparent to the chain.** A call excluded by `include`/`exclude` neither increments nor resets the counter, so `grep X → todo_write → grep X` still counts as two consecutive `grep X` when `todo_write` is excluded. This is what makes exclusion useful — bookkeeping tools interleaved into a loop must not launder it — and it is the pi extension's (undocumented) semantics, kept on purpose and written down. +- **Calls without an agent are ignored.** A direct `ctx.tools.execute()` caller (tests, non-loop consumers) has no model to remind and no `AgentId` to key on. + +### Reminder delivery + +Reminders ride `additionalContext` (source `{kind: 'plugin', plugin: 'repeat-tool-guard'}` — the label is load-bearing per `HookContext`), never a `content` replacement: the `tool/result` event stays the tool's own output for audit, and the loop appends buffered context as `context/message`(s) after the step's results, which the session renders as the tagged synthetic-user envelope and derived history replays. Thresholds escalate: the first configured threshold gets a short "you are repeating yourself, analyze the previous result" nudge; each later threshold gets the detailed form naming the tool, the repeat count, and the canonical arguments (head-truncated at `argumentsPreviewChars`, default 500 — a looping `write`-sized payload must not ride into the next request unbounded; the chain key always compares the full canonical string), and stating that the calls made no progress. The pi original hardcodes the gentle text to the literal count 3; the guard keys it to `thresholds[0]`, fixing that bug in the port. When the downstream decision already carries `additionalContext` (a hook bridge on the same call), the guard concatenates content under its own `source` — a `HookContext` holds one `MessageSource`, and `source.kind` is what framing depends on. + +### Config + +```yaml +- id: repeat-tool-guard + name: '@deepseek-ai/dsh-repeat-tool-guard' + config: + thresholds: [3, 5, 8] # default; consecutive counts that trigger a reminder + include: [] # tool-name patterns to track; empty ⇒ all tools + exclude: [todo_write] # tool-name patterns transparent to the chain + argumentsPreviewChars: 500 # default; cap on arguments quoted in the detailed reminder +``` + +`thresholds` is validated at load and throws on an empty list, a non-integer, a value below 2, or a duplicate — misconfiguration fails loud, replacing the pi original's silent fall-back to defaults. `include`/`exclude` entries support `*` wildcards. Patterns are predicates over whatever tools exist at call time, not references to a registry entry, so an entry matching no currently registered tool is NOT an error — unlike `toolOrder`'s referent check, `exclude: [mcp_*]` must stay valid in a deployment that loads no MCP tools. + +## Testing + +**Unit** — the suite drives a real agent loop against a scripted mock adapter (no network) and covers, at per-file 100%: counting/reset semantics (identical, different-tracked, untracked-transparent, prompt-submit reset, disposal cleanup, per-agent isolation), canonicalization (deep key-order insensitivity), threshold escalation including the `thresholds[0]` gentle-text rule, denied-call counting, no-agent transparency, wildcard escaping, config fail-loud cases, and both fold-onto-downstream paths (block and accept-with-replacement). **Snapshot** — the `repeat-tool-guard` scenario in the acp-agent example suite scripts five identical `todo_write` calls and pins both reminder tiers (gentle at the third, detailed at the fifth) as `context/message`s in the ACP transcript and the session log; the guard is loaded in the example's live tree (`cordis.yml`), inert for every other scenario (none repeats a call three times). The scenario is authored keyless (like `error-finish`/`cancel`): deterministically forcing a live model to repeat one call three times is not a stable recording. **e2e** — none: the plugin is provider-independent and deterministic, and the seam contracts it relies on are e2e-covered by their owners. + +## Alternatives considered + +- **Append the reminder into the tool result** (`accept` with replaced `content` — the pi extension's mechanism, which patches result content because that is the only channel its API offers) — rejected: it makes the logged `tool/result` lie about what the tool returned, and `additionalContext` exists precisely as the separate sanctioned channel for post-execute commentary, with loop-level buffering that preserves call/result adjacency. +- **Count in `tools/pre-execute` with a pending-reminder map** (the pi two-phase shape) — rejected: post-execute alone sees `(exec, result)` together and also fires for denied calls, so one listener with no cross-event state covers strictly more attempts with less machinery. +- **Escalate to `block` at the highest threshold** — rejected for the initial scope: a blocked call punishes legitimate identical repeats (polling a long-running terminal, re-checking a file the agent expects to change), and an advisory reminder keeps the model in control. Revisit with evidence; the decision shape (`PostToolDecision`) already supports it. +- **A per-deployment external hook via the CC/Codex bridges** (a `PostToolUse` script) — rejected as the answer: it works for one deployment, but a shipped, unit-tested, `cordis.yml`-configurable plugin is the harness-native form, without per-call subprocess cost. +- **A loop-level step or repetition budget in `agent-loop`** — rejected: "plugins, not loop changes"; a hard step budget is a blunter, orthogonal control that would need its own proposal. +- **Fuzzy/near-identical detection** (normalized paths, similar-but-not-equal arguments) — rejected: exact match after canonicalization is cheap, deterministic, and explainable to the model; similarity thresholds invite false positives and need evidence before they earn complexity. +- **Placing the package in `core/`** — rejected: core is the product spine; a behavioral guard is an optional leaf plugin, and the `todo/` precedent is a small dedicated group per plugin family. + +## Consequences + +- The reminder is advisory by design: idempotent polling patterns that repeat identical calls on purpose still receive nudges past the thresholds, and the pressure valves are config (`thresholds`, `exclude`) plus reminder text that explicitly allows finishing when enough evidence has been gathered. Each trigger costs reminder tokens on the next request; thresholds bound the frequency. +- Chain state is in-memory only: a session resumed from persistence starts with a fresh chain, so a loop spanning a resume draws its reminders later than a live one — accepted, the guard is a heuristic nudge, not a logged invariant, and persisting counter state would buy little for real complexity. +- When multiple post-execute producers attach context on one call, the fold concatenates under the guard's `source`; ordering between plugins follows listener registration order. The seam cannot represent mixed provenance — a limit inherited from `HookContext`, not owned by this plugin. +- Implementing the snapshot tier surfaced a hidden assumption in the suite kit: the fixture guard equated "authored model scenario" with "override-driven". The `Scenario` table now carries an explicit `overridden` flag, and the sidecar's presence is checked BOTH ways against it (an unregistered stray sidecar would silently replace the derived script) — the suite kit is stricter than it was before this plugin existed. + +## Deferred + +- Compaction does not reset chains: a compacted history changes what the model sees, but the repetition risk usually survives compaction. +- Escalating to `block` at a high threshold is not implemented; `PostToolDecision` already supports it if evidence arrives. +- Subagent chains stay isolated per agent; no sharing mechanism exists until a concrete case appears. diff --git a/docs/rfc/implemented/process/2026-06-11-quality-gates.md b/docs/rfc/implemented/process/2026-06-11-quality-gates.md index 69b1beb554..505cea90ff 100644 --- a/docs/rfc/implemented/process/2026-06-11-quality-gates.md +++ b/docs/rfc/implemented/process/2026-06-11-quality-gates.md @@ -14,7 +14,7 @@ Every AGENTS.md promise gets a command that exits non-zero, wired into git hooks - ESLint strict-type-checked + @stylistic (the house style, enforced); vendored code excluded. - Per-file 100% coverage on `packages/*/src` (v8); unreachable defensive guards carry `/* v8 ignore */ ` with stated reasons instead of deletion. - knip (dead code/deps), publint (package correctness), workspace constraints (workspace rules: private, cordis peer+dev, uniform version, ESM), and a NodeNext consumer typecheck for built package declarations. -- lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and pre-push (tests, hygiene); CI runs the full matrix on node 24/26 plus a demo smoke test driving the echo-agent end to end. +- lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and pre-push (tests, hygiene); CI runs the full matrix on node 22.19/24/26 plus a demo smoke test driving the echo-agent end to end. ## Consequences diff --git a/docs/rfc/implemented/process/2026-07-06-export-surface-jsdoc-gate.md b/docs/rfc/implemented/process/2026-07-06-export-surface-jsdoc-gate.md new file mode 100644 index 0000000000..2e98808426 --- /dev/null +++ b/docs/rfc/implemented/process/2026-07-06-export-surface-jsdoc-gate.md @@ -0,0 +1,43 @@ +# RFC: Export-surface JSDoc gate + +Status: implemented + +## Problem + +The [cordis JSDoc completeness gate](2026-07-04-cordis-jsdoc-completeness-gate.md) made undocumented parameters and results impossible on the cordis surface — `interface Events` members and `ctx.` service classes — but that surface is a fraction of what a plugin author imports. The AGENTS.md rule "every export (and non-obvious method) has a JSDoc explaining semantics" stayed prose-checkable only by review everywhere else, and nothing at all asked for `@param`/`@returns` on ordinary exported functions. A survey at adoption found 203 under-documented module-level exports across 34 packages: seam-adjacent helpers (`runBash`, `readForEdit`, `htmlToMarkdown`), format codecs, whole undocumented interfaces and type aliases — exactly the names an IDE consumer hovers. + +## Decision + +A new gate, `scripts/verify-export-jsdoc.ts` (`pnpm run verify-export-jsdoc`, wired into `doc-sync` beside `verify-cordis-catalog`), walks every module-level exported name under each `packages///src/` tree. The parsing and check helpers moved from `gen-cordis-catalog.ts` into a shared `scripts/jsdoc.ts`, so "documented" means the same thing on both surfaces: description prose ends at the first block tag, every checkable parameter needs a non-empty `@param`, a non-void ANNOTATED return needs a non-empty `@returns`, a stale `@param` errors, and violations aggregate into one report. + +The contract by declaration kind: + +- Every exported name needs JSDoc with non-empty description prose. +- Function-like exports (function declarations; consts with function initializers or an INLINE callable annotation; non-identifier function default exports) follow the full function contract, with wrapper expressions (parentheses, `as`/`satisfies` casts, non-null assertions) peeled before classifying. A const whose declarator is annotated with a NAMED type (`export const f: Handler = …`) defers the signature contract to that type's own declaration and `@returns` stays optional; an inline `(x: T) => U` annotation or single-call-signature literal is the surface signature itself and gets the full contract, and a literal mixing call/construct signatures with anything else is refused outright (no single signature to hold the tags against — extract a named type). +- Exported classes need class-level prose; public methods (statics included — reachable on the exported name) follow the function contract; public properties and accessors need prose (a get/set pair is covered by the getter). Overload implementations are exempt — the signatures carry the docs. +- Exported interfaces, type aliases, and enums need prose on the declaration; member-level enforcement is deliberately deferred (the highest-value member surface — seam service classes — is already under the cordis gate). +- Exported namespaces recurse (inside an ambient `declare` namespace every member exports implicitly); the namespace itself needs prose only when it does not merge with a documented same-name declaration (the Config-namespace idiom documents the plugin once). +- `declare module` / `declare global` bodies and `export … from` re-export statements are skipped: an augmentation is not an export of the package, and a re-exported definition is checked where it is defined. An `export import X = N.member` alias documents ITSELF — its target may be a non-exported namespace member no walk visits — and only prose-only target kinds are gate-supported: a callable, class, or namespace target carries signature/member contracts the alias prose cannot hold, so the gate refuses it and demands the declaration be exported directly. +- Everything else fails CLOSED: `export =` is refused outright, parameters the base never names keep their `@param` duty even as binding patterns, and an exported statement kind the dispatch does not recognize is itself a violation — no export form can pass unchecked by omission. + +Three exemption families keep the gate from demanding boilerplate, in the spirit of the cordis gate's `this`/`next` exemptions (documenting an exempt name anyway is allowed; only absence goes unchecked): + +- **Heritage members.** A class member whose name exists on an `extends`/`implements` heritage type is exempt: the seam declaration is the doc's one home, and the IDE inherits it on hover — re-documenting every `LocalBashExecutor.run` invites drift. The exemption stops where the override grows surface the base never documented: a protected-only base member does not exempt a public override, parameters the base never names keep their `@param` duty (an underscore-prefixed rename of a base parameter — the deliberately-unused marker — is the same parameter), and a concrete result above a void base return keeps its `@returns` duty (an unannotated override's inferred return is classified by the checker, so a faithful void override needs no boilerplate annotation). Heritage lookups and that one return classification are the walk's only TYPE CHECKER questions (heritage types live across package boundaries, resolved through the repo `paths` map); everything else stays pure AST, and the annotated-return requirement is kept for symmetry with the cordis gate (it bound nothing at adoption — every exported function was already annotated). +- **Plugin-protocol slots.** Top-level `name` / `inject` / `reusable` / `Config` consts and the `apply` entry, plus the same slots as statics on a plugin class, are framework protocol: their shape is fixed by cordis, and the module doc comment plus the `interface Config` carry the plugin's real semantics. +- **Constructors**, mirroring the cordis gate: plugin classes are framework-constructed, and the class doc owns the story. + +`collectExportJsdocViolations()` returns the violation list (the CLI exits 1 on non-empty) so the negative-path tests in `packages/core/agent/tests/verify-export-jsdoc.spec.ts` assert on findings directly, driving fixture packages through every rejection and every exemption. + +## Alternatives considered + +- **eslint-plugin-jsdoc** (`require-jsdoc`/`require-param`/`require-returns`) — covers the mechanical core but cannot express the repo's contract: the heritage-member exemption needs cross-package type resolution, the protocol-slot and namespace-merge idioms are cordis-specific, and the completeness semantics (prose-above-tags, stale-tag errors, aggregate reporting) already have one home in `scripts/jsdoc.ts` shared with the catalog generator. Two subtly different definitions of "documented" is the failure mode this repo's one-home rule exists to prevent. +- **Extending `gen-cordis-catalog.ts`** — the catalog generator renders a curated surface and gates its freshness; a repo-wide walk has no catalog to render. Sharing the helpers while keeping the walks separate keeps each gate's scope legible. +- **Enforcing interface/type-alias member docs** — deferred: it would multiply the checked surface for members that are largely self-describing fields, while the seam classes carrying the load-bearing member contracts are already gated. Revisit if member-doc drift shows up in review. + +## Consequences + +- A new export cannot land undocumented: `verify-export-jsdoc` fails `doc-sync`, which pre-push and CI already run. The 203 gaps found at adoption were filled in the same change, so the gate landed green. +- Exported functions must annotate return types (universal at adoption, now load-bearing) and use identifier parameters where `@param` must name them. +- Seam docs are canonical: an implementation inherits its heritage docs, and behavior notes worth keeping on the implementation are additions, not requirements. +- The gate builds a `ts.Program` (~6s) — the one doc gate that pays for type resolution; acceptable inside `doc-sync`, which already compiles doc snippets. +- The protocol-slot names are reserved by convention at module top level; a non-protocol export coincidentally named `apply` or `Config` would go unchecked — accepted, documented here. diff --git a/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md b/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md new file mode 100644 index 0000000000..72c9f09eda --- /dev/null +++ b/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md @@ -0,0 +1,37 @@ +# RFC: Raise the Node LTS engine floor to 22.19 + +Status: implemented + +## Problem + +The Node 22 branch of the root `engines.node` range is a contract for the installed workspace, not only for the runtime APIs the harness source calls directly. It must be no lower than package `engines.node` declarations for dependencies the workspace installs on that branch; otherwise `pnpm install --engine-strict` fails at an advertised LTS version, and non-strict installs run outside a dependency's supported runtime. + +## Decision + +Set `engines.node` to `^22.19.0 || >=24.0.0` and test the keyless CI compatibility matrix on `['22.19', 24, 26]`. The real-API e2e workflow stays on Node 24 because it exercises API integration rather than the runtime floor. + +Two Node features gate the source runtime: + +- **`node:sqlite`** — `packages/session-persistence/session-persistence-sqlite` does a top-level `import { DatabaseSync } from 'node:sqlite'`. The module dropped its `--experimental-sqlite` flag requirement at **22.13** (LTS) and **23.4** (Current); before those, importing it throws at load. +- **Native TypeScript type-stripping** — the `packages/ui/stdio-agent/tests/built-bin.e2e.ts` smoke boots the published `lib/bin.js` under plain `node` (no tsx) and loads the example's `.ts` plugins (`mock-llm.ts`, `echo-tool.ts`). Type-stripping is the default from **22.18** (LTS) and **23.6** (Current); before those it needs `--experimental-strip-types`. + +Those source features clear on the 22.x line at **22.18**, but the installed Pi adapter dependency raises the advertised LTS floor. `@deepseek-ai/dsh-llm-pi-ai` depends on `@earendil-works/pi-ai@0.79.3`, whose package declares `engines.node >=22.19.0`, so the LTS floor is **22.19**. The 24.x branch remains `>=24.0.0`. The disjoint range excludes Node 23 entirely: Node 23.0–23.5 still has at least one flagged source feature, and the 23 line is non-LTS/EOL, so advertising `>=23.6` would add a dead release line and a CI leg no deployment should use. + +`@types/node` remains pinned to the 22.x line (`^22.20.0`) to match the LTS support line: reaching for a Node 23+/24+/25+ API fails `tsc` on every machine and in the typecheck gate, rather than compiling clean and surviving to a runtime failure only a floor matrix leg could catch. The whole tree typechecks clean against the Node 22 type surface today, so the pin costs nothing. + +## Consequences + +- The advertised LTS branch no longer undercuts the Pi adapter dependency floor. +- CI proves the Node 22 LTS floor directly with Node 22.19, keeps the Node 24 branch on `node: 24`, and keeps Node 26 for the next even line. +- The built-bin smoke needs no version-conditional flag: at 22.19 type-stripping is already the default, so the test stays the plain `node lib/bin.js` path it documents. +- A future dependency or source API that raises the runtime floor must move `engines.node`, the compatibility matrix, and this RFC in the same change. + +## Alternatives considered + +- **Keep `^22.18.0 || >=24.0.0`.** Rejected: it advertises an LTS version lower than the Pi adapter dependency floor. `@earendil-works/pi-ai@0.79.3` requires `>=22.19.0`. +- **Downgrade or pin `@earendil-works/pi-ai` to preserve the 22.18 advertised range.** Rejected: the current Pi adapter dependency is part of the intended workspace, and 22.19 is still inside the Node 22 LTS line. +- **Floor `>=22.13` (the `node:sqlite` boundary) plus `--experimental-strip-types` in the built-bin smoke on 22.13–22.17.** Rejected: it adds a version-conditional test flag for one narrow range and dresses up an experimental-flag dependency as first-class support. The Pi adapter dependency already requires a higher LTS floor. +- **Open-ended `>=22.19`.** Rejected: it advertises support for Node 23.0–23.5, where `node:sqlite` (until 23.4) or type-stripping (until 23.6) is still flagged. +- **Include Node 23.6+ (`^22.19.0 || >=23.6.0`).** Rejected: 23.6+ does run both source features unflagged, but Node 23 is end-of-life; advertising a dead release line adds a range term and a CI leg for a runtime no deployment should use. +- **Matrix `[22, 24, 26]` instead of pinning `22.19`.** Rejected: floating major-version entries drift upward over time and silently stop exercising the declared LTS floor. +- **Keep `@types/node` ahead of the floor (`^25`).** Rejected: types ahead of the runtime floor let a Node 24/25-only API compile clean and fail only at runtime on 22.x. Pinning `@types/node` to the 22.x line turns that into a compile error everywhere. diff --git a/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md b/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md index 6e73d5d333..ae5ea96f83 100644 --- a/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md +++ b/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md @@ -68,7 +68,7 @@ The replay plugin lives in its own package, `@deepseek-ai/dsh-llm-replay` (`pack ### Two subcommands, replay in the default gate -`pnpm run test:snapshot` runs replay (keyless) and is composed into the default `pnpm run test` gate so every PR gets the regression check (the main `vitest.config.ts` include stays narrow; the gate is `test && test:snapshot`). `pnpm run test:snapshot:record` requires `DEEPSEEK_API_KEY` (loaded from repo `.env` first), hits the real API, harvests the produced `session.jsonl` (the replay source AND the expected-log artifact), and `--update`s the stdout golden in one pass. Both forward a scenario filter. A missing fixture in replay **fails loud** with a "record first" message rather than self-skipping (the e2e self-skip rule is a CI-secret accommodation, not appropriate here — a committed-fixture test that silently vanishes is a coverage hole). A no-model scenario's `session.jsonl` simply has no `assistant/chunk` events (empty derived script); fail-loud still applies if a model call happens with no entry. An orphan-fixture guard test fails on a golden/fixture not referenced by any scenario (Vitest does not prune orphaned raw goldens), and a per-kind required-fixture guard asserts each scenario ships exactly the files its kind needs (`input.json` + `stdout.golden.jsonl` + `session.jsonl` for ALL scenarios — the harness passes `/session.jsonl` to `llm-replay` unconditionally, so even a no-model scenario needs its header-only fixture or `loadReplayScript()` fails; `replay.override.json` additionally for authored model scenarios). +`pnpm run test:snapshot` runs replay (keyless) and is composed into the default `pnpm run test` gate so every PR gets the regression check (the main `vitest.config.ts` include stays narrow; the gate is `test && test:snapshot`). `pnpm run test:snapshot:record` requires `DEEPSEEK_API_KEY` (loaded from repo `.env` first), hits the real API, harvests the produced `session.jsonl` (the replay source AND the expected-log artifact), and `--update`s the stdout golden in one pass. Both forward a scenario filter. A missing fixture in replay **fails loud** with a "record first" message rather than self-skipping (the e2e self-skip rule is a CI-secret accommodation, not appropriate here — a committed-fixture test that silently vanishes is a coverage hole). A no-model scenario's `session.jsonl` simply has no `assistant/chunk` events (empty derived script); fail-loud still applies if a model call happens with no entry. An orphan-fixture guard test fails on a golden/fixture not referenced by any scenario (Vitest does not prune orphaned raw goldens), and a per-kind required-fixture guard asserts each scenario ships exactly the files its kind needs (`input.json` + `stdout.golden.jsonl` + `session.jsonl` for ALL scenarios — the harness passes `/session.jsonl` to `llm-replay` unconditionally, so even a no-model scenario needs its header-only fixture or `loadReplayScript()` fails; `replay.override.json` exactly for the scenarios whose table entry sets `overridden` — required with the flag, forbidden without it, because the harness forwards the sidecar purely on file existence and an unregistered stray would silently replace the derived script). ## Alternatives considered diff --git a/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md b/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md index 982c0fcb3c..e567c4ff24 100644 --- a/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md +++ b/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md @@ -54,7 +54,7 @@ The repo secret is named `DEEPSEEK_API_KEY_EXTERNAL`; it is mapped to the `DEEPS ### Scope, runtime shape -Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the `engines` floor): these tests exercise *API integration*, not node-version compat, which ci.yml's Node 24/26 jobs already own; a second Node version would double real-API calls for no added signal. `vitest.e2e.config.ts` runs files through a bounded worker pool (`DSH_E2E_MAX_WORKERS`, default `4`, CI value `14`) so CI and local with-key runs parallelize independent files while retaining a one-line serial escape hatch for quota investigations. `timeout-minutes: 45` bounds a wedged run given 120s/test and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled. +Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the primary line): these tests exercise API integration, not node-version compatibility, which ci.yml's Node 22.19/24/26 matrix owns. `vitest.e2e.config.ts` runs files through a bounded worker pool (`DSH_E2E_MAX_WORKERS`, default `4`, CI value `14`) so CI and local with-key runs parallelize independent files while retaining a one-line serial escape hatch for quota investigations. `timeout-minutes: 45` bounds a wedged run given 120s/test and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled. ## Security diff --git a/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md b/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md index 07a1cfd731..2a6f8b7ae3 100644 --- a/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md +++ b/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md @@ -32,4 +32,4 @@ Reviewers lose one artifact name that made the expected persisted log visually s ## Implementation note -The comparison normalizes BOTH sides, but each against its OWN volatile values, not a shared context. A raw harvested `session.jsonl` bakes in the recording run's session id, cwd, and timestamps; the replay run produces fresh ones. `normalizeSessionLog` scrubs cwd by exact string match, so normalizing the fixture against the *replay* run's cwd would leave the recorded cwd in the header unscrubbed and the compare would fail. The harness therefore derives the fixture's normalize context from its OWN header line (`{ type:'session', id, cwd }`) — `fixtureContext()` in `acp.snapshot.ts` — so both sides scrub to the same `{{sessionId}}`/`{{cwd}}` tokens. An authored fixture copied from the old golden already carries the normalized header (`id:'{{sessionId}}'`, `cwd:'{{cwd}}'`), which yields those tokens as the volatile values and scrubs idempotently. The session-log side uses a plain normalized-string `toEqual`, NOT `toMatchFileSnapshot`, so a run never overwrites the fixture. +The comparison normalizes BOTH sides, but each against its OWN volatile values, not a shared context. A raw harvested `session.jsonl` bakes in the recording run's session id, cwd, and timestamps; the replay run produces fresh ones. `normalizeSessionLog` scrubs cwd by exact string match, so normalizing the fixture against the *replay* run's cwd would leave the recorded cwd in the header unscrubbed and the compare would fail. The harness therefore derives the fixture's normalize context from its OWN header line (`{ type:'session', id, cwd }`) — `fixtureContext()` in `dsh-acp-snapshot`'s suite module — so both sides scrub to the same `{{sessionId}}`/`{{cwd}}` tokens. An authored fixture copied from the old golden already carries the normalized header (`id:'{{sessionId}}'`, `cwd:'{{cwd}}'`), which yields those tokens as the volatile values and scrubs idempotently. The session-log side uses a plain normalized-string `toEqual`, NOT `toMatchFileSnapshot`, so a run never overwrites the fixture. diff --git a/docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md b/docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md index 4e4b6a85d7..862dfd42fa 100644 --- a/docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md +++ b/docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md @@ -8,7 +8,7 @@ Every model-driving ACP snapshot fixture (`session.jsonl`) embedded the full com ## Decision -Exactly one scenario — `text-turn`, flagged `pinsHeader` in `acp.snapshot.ts` — commits and compares the full request-header content. Every other fixture stores and compares that content as stable tokens via the pure normalizer `scrubRequestHeaders` in `snapshot-normalize.ts`: a `request/header` event's `header.system` becomes `"{{system}}"` and `header.tools` becomes `"{{tools}}"`; a `request/header-delta` keeps its structural facts — the system delta's `keepStart`/`keepEnd` line positions with one `{{system}}` token per inserted line, the tools delta's added/removed/changed tool names — and tokenizes only the bulk (prompt text, schema bodies), so two different deltas still compare different. The scrub is composed in front of `normalizeSessionLog` on BOTH sides of a non-pinning scenario's log compare and applied to the harvested logs record mode writes, so a re-record cannot smuggle the content back. Absent fields stay absent — WHETHER a header carried a prompt or tools is behavior and stays visible — and `config`/`reason` stay verbatim: a model swap churns every fixture by design (it invalidates the recorded responses), while a prompt or schema edit churns none of them (replay derives model behavior exclusively from `assistant/chunk` events and never reads header content — see `dsh-llm-replay`). +Exactly one scenario — `text-turn`, flagged `pinsHeader` in the `acp.snapshot.ts` scenario table — commits and compares the full request-header content; the pin mechanics live in [`dsh-acp-snapshot`](../../../../packages/support/acp-snapshot/README.md), whose suite factory enforces one pin per consuming suite. Every other fixture stores and compares that content as stable tokens via the pure normalizer `scrubRequestHeaders` in that package's `normalize.ts`: a `request/header` event's `header.system` becomes `"{{system}}"` and `header.tools` becomes `"{{tools}}"`; a `request/header-delta` keeps its structural facts — the system delta's `keepStart`/`keepEnd` line positions with one `{{system}}` token per inserted line, the tools delta's added/removed/changed tool names — and tokenizes only the bulk (prompt text, schema bodies), so two different deltas still compare different. The scrub is composed in front of `normalizeSessionLog` on BOTH sides of a non-pinning scenario's log compare and applied to the harvested logs record mode writes, so a re-record cannot smuggle the content back. Absent fields stay absent — WHETHER a header carried a prompt or tools is behavior and stays visible — and `config`/`reason` stay verbatim: a model swap churns every fixture by design (it invalidates the recorded responses), while a prompt or schema edit churns none of them (replay derives model behavior exclusively from `assistant/chunk` events and never reads header content — see `dsh-llm-replay`). A system-prompt or tool-schema change therefore lands as exactly one committed-fixture diff — the pinned `text-turn` header line — updated by hand or by re-recording that one scenario (`pnpm run test:snapshot:record` with `-t text-turn`). diff --git a/docs/rfc/implemented/testing/2026-07-08-shared-acp-snapshot-package.md b/docs/rfc/implemented/testing/2026-07-08-shared-acp-snapshot-package.md new file mode 100644 index 0000000000..910f179313 --- /dev/null +++ b/docs/rfc/implemented/testing/2026-07-08-shared-acp-snapshot-package.md @@ -0,0 +1,36 @@ +# RFC: Extract the ACP snapshot suite into a support package + +Status: implemented + +## Problem + +The ACP snapshot tier ([snapshot RFC](2026-06-19-acp-snapshot-tests.md)) was built from three modules living inside one example's test directory: `snapshot-harness.ts` (boot the real bin subprocess, drive it over ACP JSON-RPC, harvest the persisted logs), `snapshot-normalize.ts` (the pure golden normalizers), and the ~150-line scenario body plus fixture guards in `acp.snapshot.ts` (record/replay modes, the stdout-golden and log compares, the pinned-header uniformity guard, the orphan/required-file/single-pin meta-tests). + +A second ACP example wanting snapshot coverage — the sandbox/approval composition is the immediate consumer — could only copy those modules, forking exactly the logic that must not drift: record write-back, header scrubbing, child-session harvest ordering. The spawn/client glue was already triplicated across `acp.e2e.ts`, `hooks.e2e.ts`, and the harness (`TODO(acp-test-harness)`). Location also decided test rigor: the per-file 100% coverage gate measures `packages/*/*/src` only, so none of this machinery was measured — the same gap that had moved `dsh-llm-replay` out of `examples/` into [packages/support](../../../../packages/support/README.md). And the harness's ACP client hardcoded `requestPermission → cancelled`, so an approval round-trip — the headline behavior of the sandbox composition — could not be expressed at the snapshot tier at all. + +## Decision + +The machinery lives in [`packages/support/acp-snapshot`](../../../../packages/support/acp-snapshot/README.md) (`@deepseek-ai/dsh-acp-snapshot`); an example's `*.snapshot.ts` is its scenario table, its agent paths, and one factory call, over its own `snapshots/` fixtures and `cordis.snapshot.yml` overlay ([single-source replay config](2026-07-04-single-source-acp-replay-config.md)). Reading `DSH_SNAPSHOT` stays at that edge — the library takes a resolved `mode`. + +**`src/harness.ts`** — `runScenario` and the input-script/result types, parameterized by an `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath`; absolute paths the consuming suite resolves from its own `import.meta.url`). The client's `session/request_permission` handler consumes an optional `InputScript.permissionAnswers` FIFO queue, each entry selecting by option **kind** (ids are agent-issued randoms a committed script cannot know; kinds are the ACP-stable vocabulary, mapped to the offered `optionId` at answer time); an absent or exhausted queue answers `cancelled`, and a kind the request never offered rejects the run — the agent itself is answered `cancelled`, so the scenario bug fails the harness rather than being absorbed as an agent-side denial. This is what lets an approval suite drive allow/reject round-trips deterministically from `input.json`. + +**`src/normalize.ts`** — the pure normalizers, hook-free by policy: when a future event carries a new volatile field (an approval duration, say), the shared normalizer learns it in the same change, keeping one home for what "normalized" means rather than per-suite scrub extensions. + +**`src/suite.ts`** — the `Scenario` type and `defineAcpSnapshotSuite(options)`, registering the per-scenario compares, record-mode fixture write-back, the header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin, non-pinning fixtures are `scrubRequestHeaders` fixed points). The pinned-header contract ([pinned-header RFC](2026-07-06-pin-request-header-content-in-one-scenario.md)) is per-suite: each suite flags exactly one `pinsHeader` scenario (the factory throws on zero, a meta-test rejects more than one; WHICH scenario pins is the table's reviewable choice), and the uniformity guard compares only that suite's sessions. The pure helpers (`childFixturePaths`, `fixtureContext`, `normalizedHeaders`, `headerDeltaCount`) are exported for direct unit coverage. + +## Alternatives considered + +- **Copy the modules into each example** — the fork this RFC exists to prevent: the record/guard logic is exactly the code that must stay byte-identical across suites, and examples are outside the coverage gate, so each copy is also unmeasured. +- **A shared module directory under `examples/`** — keeps the code outside the coverage gate and forces relative imports across example boundaries, against the package-name import convention; `examples/` leaves stay thin by design. +- **A `/testing` subpath export of `dsh-acp-agent`** — couples test infrastructure into a product package's surface and dependency set; `packages/support/` exists precisely for real-but-lower-compatibility dev/test packages, with `dsh-llm-replay` as the precedent this package completes. +- **Export raw test-body functions instead of a suite factory** — each example would re-own the `describe`/`it` skeleton (~80 lines of registration boilerplate per suite) for no flexibility gain; the factory keeps consumers to a scenario table plus one call, and the exported pure helpers preserve unit-testability inside the factory design. +- **An injectable ACP `Client` factory instead of declarative `permissionAnswers`** — maximally flexible, but it leaks SDK client construction to every consumer and reopens per-example drift in exactly the layer being unified; a declarative queue keeps `input.json` the single scripting surface and stays golden-normalizable. +- **Generalize beyond ACP (a transport-agnostic snapshot harness)** — no second transport exists; the harness is ACP-shaped end to end (SDK client, JSON-RPC frames, `session/update` waiters), and a speculative abstraction would be a seam split ahead of any consumer. + +## Testing + +Extraction parity was proven mechanically: after the move, `pnpm run test:snapshot` matched the base commit's result with zero byte changes under `examples/acp-agent/tests/snapshots/`. The package's `src/` holds per-file 100% statements/branches/functions/lines under the gating unit run, driven through the REAL spawn path by a scripted fake ACP bin (`tests/fixtures/fake-acp-agent.ts`, behavior scripted per scenario via a `behavior.json` beside the fixture): `harness.spec.ts` covers every step op, both expect-error arms, the permission queue (selection, fallback, impossible-click), env forwarding, workspace seeding, and the harvest ordering/noise/fallback branches; `suite.spec.ts` runs the factory for real at collection time — a replay suite over committed synthetic fixtures and a record suite over a temp copy (write-back never touches the committed tree; `ACP_SNAPSHOT_SPEC_BOOTSTRAP=1` re-bootstraps it) — plus direct cases for the pure helpers. Two structurally unreachable guards carry reasoned `v8 ignore` comments. The fake bin substitutes the `session/new` cwd, not `process.cwd()`, into scripted logs, matching what the real bin's header carries (darwin realpaths `/var/folders/…` to `/private/var/folders/…`). + +## Consequences + +A new example gets the whole snapshot tier from a scenario table plus fixtures — the sandbox branch merges master down and adds its own suite (own pin scenario, own overlay, fixtures via `test:snapshot:record`, approvals via `permissionAnswers`). The costs: `suite.ts` imports vitest, so the package is importable only inside a vitest run — a shape no other package has, stated in its README; each suite pins its own ~8 KB header fixture (a genuinely distinct composition deserves its own pin; an identical one would be caught by that suite's uniformity guard); and the e2e launcher duplication remains (`TODO(acp-test-harness)`) — the harness is the extraction target when that migration lands. diff --git a/docs/rfc/proposed/feature/2026-06-15-code-mode.md b/docs/rfc/proposed/feature/2026-06-15-code-mode.md new file mode 100644 index 0000000000..b9a408241e --- /dev/null +++ b/docs/rfc/proposed/feature/2026-06-15-code-mode.md @@ -0,0 +1,139 @@ +# RFC: Code Mode — the model writes TypeScript against the tool registry + +Status: proposed + +## Problem + +Today the agent loop advertises every registered tool to the model as a native JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../architecture.md)), with **every** intermediate `tool-result` re-entering the model's context on the next request. + +For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each round-trip drags the entire intermediate result back into context whether the model needs it or not. + +Cloudflare's [Code Mode](https://blog.cloudflare.com/code-mode/) proposes an alternative grounded in a simple observation: LLMs are better at writing code than at emitting tool calls, because they have seen millions of lines of real code and comparatively few contrived tool-calling traces. Instead of one tool call per step, the model writes a TypeScript program against a generated API over the tools, the program executes in a sandboxed runtime, and the model curates what comes back — only what it prints or returns — instead of every intermediate result. + +An earlier draft of this RFC designed Code Mode as an add-on consumer plugin with zero core changes, deferring the execution substrate to a follow-up. Both constraints are dropped here, deliberately. First, the harness is pre-release and optimizes for the correct foundation over blast radius: tool presentation is the registry's own concern, and bolting a second presentation onto it from outside means transforming the registry's contribution after the fact — a waterfall listener whose correctness depends on listener ordering, which fights the [reconstructable-requests](../../implemented/architecture/2026-07-05-reconstructable-requests.md) design instead of riding it (that refactor removed request mutation from `agent/request`, the seam the old draft relied on). Second, the substrate question is answerable now: a Node `worker_threads` runtime gives real containment — separate isolate, empty environment, heap caps, and a `terminate()` that reliably stops a hot synchronous loop — where the old draft's `node:vm` stub had none of those, and it fits the harness's existing trust model (§Trust posture) without a hardening follow-up. + +## Proposal + +Three decisions, each elaborated in its own section below: + +1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (today's behavior, the default), `'code'` (the wire carries exactly one tool, `run_code`, plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas *and* `run_code` + SDK). The registry's existing tool-schema provider contributes whatever the mode dictates, so the wire tool list is shaped at its source — no interception, no listener-ordering caveats — and the logged request header records it for free. +2. **Code execution is a capability seam** — a new group `packages/code-runtime/` with the interface package `@deepseek-ai/dsh-code-runtime` owning `ctx.codeRuntime` ([capability seams](../../implemented/architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is a new implementation package, not a redesign. +3. **The shipped implementation is `@deepseek-ai/dsh-code-runtime-worker`**: one fresh Node worker thread per run, executing the model's TypeScript after type-strip, with bindings bridged over the message port, an empty environment, configurable heap/output/time caps, and hard termination. Its trust posture is bash-equivalent by design — no unsafe-acknowledgement flags — because the harness already ships `dsh-bash-local`, which executes arbitrary model-written shell commands with strictly *more* ambient authority. + +### The registry owns the mode + +`ToolRegistry` gains a schemastery-validated config (`static Config`), its first: `mode: 'native' | 'code' | 'both'`, default `'native'`. A deployment flips it from `cordis.yml` (`tools: { mode: code }`) — no code edit, per the no-hardcoded-tunables convention. + +**Wire tool list = the registry's contribution.** The registry already feeds the assembly through `ctx.systemPrompt.tools(() => this.schemas())`; the provider becomes mode-aware: `'native'` contributes all schemas (unchanged), `'code'` contributes only `run_code`'s schema, `'both'` contributes all schemas plus `run_code`. Because [`PromptAssembly.tools` is the single source the loop's request header snapshots](../../../../packages/core/system-prompt/src/index.ts), the collapse is automatically logged and reconstructable — model-visible ⟺ logged holds with zero new mechanism. Scope of the guarantee, stated honestly: the mode governs the **registry's** contribution, and the registry is the only shipped schema source — but `systemPrompt.tools()` is a public multi-provider API and the `system-prompt/assemble` waterfall may transform the assembly, so a deployment that wires a second direct provider (or a mutating listener) owns what it adds, exactly as in native mode. Those are deliberate acts; what the design eliminates is the *accidental* leak the old draft worried about — a listener-ordering race around an after-the-fact collapse — and the shipped-configuration invariant (`'code'` ⇒ assembled tools exactly `[run_code]`) is pinned by tests and, like every request, by the logged header. + +**Interaction with `toolOrder`, stated up front:** a configured `systemPrompt.toolOrder` naming native tools rejects every assembly under `mode: 'code'` (those names are no longer contributed), by the existing fail-loud rule for unlisted names. This is correct behavior, not a bug: a deployment switching modes updates its order config or drops it. + +**The SDK prompt section.** Under `'code'` and `'both'` the registry registers one lazy prompt section (`tools:sdk`, in the 100–199 tool-guidance order band) whose thunk regenerates, at each assembly, a TypeScript declaration of every registered tool except `run_code` itself, plus fixed usage instructions. The thunk reads the live store and emits tools in lexicographic name order, so its output is deterministic and stable across steps — an unchanged tool set produces byte-identical text (prefix-cache-friendly; a mid-session registration surfaces as one logged header delta, exactly like a native-mode tool change). + +**Codegen.** A pure `jsonSchemaToTs(schema)` module inside `dsh-tools` (sibling of `json-schema.ts` — `schemas()` and the SDK are two projections of the same store) maps the JSON-Schema subset the `defineTool` DSL emits (object/string/number/boolean/array, `properties`, `required`, string `enum` → literal union, nested objects, array `items`, `description` → JSDoc) to a TS type literal. It is **total**: any construct outside that subset (`$ref`, `oneOf`/`anyOf`, `integer`, future MCP shapes, …) degrades to `unknown` without throwing. Because `ToolSchema.name` is an arbitrary string, the SDK is declared as one object constant — `declare const tools: { "some-mcp-tool"(args: …): Promise; bash(args: …): Promise; … }` — quoted keys make every name reachable with no sanitization or alias-collision logic. Typing is advisory (the runtime executes type-stripped JS); the instructions say so. + +### The run_code tool and the dispatch bridge + +Under `'code'` and `'both'` the registry registers `run_code` in itself as an ordinary tool — one required parameter `{ code: string }` — so the unchanged loop dispatches it through the normal pipeline and `tools/pre-execute` / `tools/post-execute` gate it like any other call (a permission plugin can inspect the program text before it runs). Its `execute(args, exec)`: + +1. **Builds the bindings**: the bridge owns a **run-scoped `AbortController`** whose signal follows `exec.signal` (an outer cancel propagates in) and which the bridge itself fires the moment the run settles for any reason — completion, program exception, `computeMs`/`maxWallMs` expiry, worker exit. For every registered tool except `run_code`, the binding is an async function that (a) checks the run signal before and after (throwing stops the program — necessary because `ctx.tools.execute()` converts errors to `isError` data), (b) **JSON-normalizes the argument** — a `JSON.parse(JSON.stringify(args))` round-trip, rejecting that one call with a descriptive `Error` when the value does not survive (`BigInt`, circular structures) — because the seam's structured-clone boundary is wider than JSON while the session log accepts only JSON: normalizing BEFORE dispatch makes the dispatched form and the logged form the same JSON value by construction, so an executed sub-call can never fail at logging time, (c) awaits its turn on the **per-run serialization queue** (below), (d) calls `this.execute({ callId, name, arguments, agent: exec.agent, signal: runSignal })` with a deterministic sub-id `` CallId(`${exec.callId}:code:${n}`) `` — the run signal, not the bare outer one, so a budget expiry aborts an in-flight sub-tool (`bash-local` kills on its spec signal) instead of orphaning it, (e) appends a `tool/code-dispatch` session event, and (f) maps the result: success → the text-block contents joined as a `string` (non-text blocks become placeholders, an MVP limitation), `isError` → **the binding rejects** with an `Error` carrying the result text. Rejection is the deliberate model-facing contract — real code signals failure by throwing, `try/catch` and `Promise.all` short-circuiting behave as every model has seen them behave — where the old draft's `{ output, isError }` envelope made error handling a bespoke convention. +2. **Runs the program**: `ctx.codeRuntime.run({ program: args.code, bindings: [{ global: 'tools', functions }], signal: exec.signal })`. +3. **Surfaces the outcome — after reaching quiescence.** When `ctx.codeRuntime.run()` resolves, the bridge fires the run-scoped abort (cancelling any in-flight sub-dispatch and abandoning queued-unstarted ones), then **awaits the dispatch queue's drain before returning**, per the dispose-to-quiescence rule in [defensive patterns](../../../defensive-patterns.md): an aborted in-flight sub-call still settles and logs its `isError` `tool/code-dispatch` event *inside* the open turn, and nothing can append after `run_code` returns. A successful run then returns one text block — the captured console/stdout output followed by the rendered return value (if any) — plus a `meta` payload (capped logs, dispatch count) for presentation. A run with `result.error` throws a `CodeRunFailedError extends HarnessError` (`code: 'CODE_RUN_FAILED'`, message = the error kind and text plus captured logs so the model can self-correct); the registry's existing catch turns it into a structured `isError` result. + +**Sub-call `additionalContext` is suppressed, deliberately.** A `tools/post-execute` hook may attach `additionalContext` to a call; for loop-dispatched calls the loop buffers those and appends each as a `context/message` only after the step's `tool/result`s, preserving call/result adjacency. A sub-dispatch result's `additionalContext` has no such safe outlet from inside a running `run_code`: injecting immediately would land a `context/message` between the parent's `tool/call` and its `tool/result` (breaking the adjacency the buffering exists to protect), and `PostToolDecision.additionalContext` is singular where a program may produce many. The MVP therefore drops sub-call `additionalContext`, pinned by a test and stated in the hooks bridge's docs; the follow-up (a plural context channel or loop-level sub-dispatch buffering) is deferred until a real hook needs it through Code Mode. + +**Concurrency: serialized, enforced by the binding.** The bindings are async, so a model writing `Promise.all([tools.a(…), tools.b(…)])` starts both immediately — concurrent dispatch would be the *default*, while the tool contract still carries no concurrency-safety metadata (the open parallel-execution TODO). Each `run_code` invocation therefore owns a dispatch queue and every binding call chains onto it, so even `Promise.all` executes the underlying `ctx.tools.execute()` calls one at a time in submission order; when the run settles, queued-but-unstarted dispatches are abandoned. Lifting this per-tool once tools can declare themselves concurrency-safe is deferred work, same as before. + +**Presentation.** `run_code`'s render intent is decided here per the [render-intent RFC](../../implemented/architecture/2026-07-02-tool-render-intent-union.md): `presentCall` → a `generic` card, `kind: 'execute'`, title `Run code`, `rawInput` = the program text; `presentResult` → a `generic` card whose content is the captured output (from `meta`). Not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not. + +### Observability: `tool/code-dispatch` + +Each sub-dispatch appends one session event, declared by `dsh-tools` via `SessionEventMap` declaration merging (the map is merge-extensible for exactly this; `todo/write` is the log-only precedent): `tool/code-dispatch` with `{ parentCallId, subCallId, name, arguments, isError, resultSummary }` — `arguments` being the bridge's JSON-normalized value, the very one dispatched, so the append cannot fail on payload shape. It is log-only — `deriveEventMessage()` ignores unknown event types by design, so sub-calls never re-enter model context — but persistence and UIs get every call. As a log event it carries JSDoc prose but **no `@mode` tag** (that vocabulary belongs to cordis bus events; the persistence-catalog generator hard-errors on one) and lands in the regenerated `docs/persistence-catalog.md`; appends happen inside `run_code`'s execution, so the turn-enclosure invariant is satisfied by construction. A `run_code` execution arriving without `exec.agent` (the loop always supplies it; direct programmatic calls may not) still runs and simply skips event logging, exactly as the `ToolExecution` contract allows. + +### The code-runtime seam + +`packages/code-runtime/code-runtime/` — `@deepseek-ai/dsh-code-runtime`, depending only on `cordis`. An abstract `CodeRuntime extends Service` (`super(ctx, 'codeRuntime')`) plus the vocabulary: + +- `CodeRunRequest = { program: string; bindings: CodeBindingNamespace[]; signal?: AbortSignal }` +- `CodeBindingNamespace = { global: string; functions: Record Promise> }` — the runtime exposes each namespace as a global object of async functions inside the program; binding arguments and resolutions must be structured-cloneable (a runtime may cross a serialization boundary; ours does). +- `CodeRunResult = { value?: unknown; logs: CodeLogEntry[]; error?: CodeRunFailure }` — an error is a field on a resolved result, never a rejection of `run()`. +- `CodeLogEntry = { source: 'console' | 'stdout' | 'stderr'; level?: 'log' | 'info' | 'warn' | 'error' | 'debug'; text: string }` +- `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout. +- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the shipped backend; a Python backend would say so, and pair with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). + +Per explicit-over-implicit at seams, the request spells out everything the runtime acts on; defaulting (timeouts, caps) is the implementation's validated config, never a hidden `??` inside `run()`. Consumption uses the loop's established optional-backend idiom: cordis has no optional injection — every `inject` entry gates activation — so a static `inject` on the registry would hold `ctx.tools` (and every tool plugin behind it) hostage to a code runtime existing even under `mode: 'native'`; instead the registry reads `ctx.get('codeRuntime')` at use time, exactly as `agent-loop` consumes `sessionPersistence`, with absence failing loud in the provider thunk as above. The seam split is justified by real planned divergence on both axes — substrate (worker now; container/microVM later) and language (the Python/AssemblyScript direction sketched in the earlier draft survives as future work) — not by speculation: `dsh-tools` consumes the interface today and tests against a trivial in-repo fake, exactly the interface/implementation/consumer shape of the bash template. + +### The worker-thread runtime + +`@deepseek-ai/dsh-code-runtime-worker`, the second package of the `packages/code-runtime/` group. Per `run()`: + +1. **Type-strip host-side** with Node's built-in `stripTypeScriptTypes` (`node:module`; present across the repo's whole engines range, `^22.19.0 || >=24.0.0`, and position-preserving, so runtime error line numbers match the model's source). Strip-only mode rejects non-erasable syntax (`enum`, namespaces) — that rejection returns as `error.kind: 'exception'` with Node's message, the SDK instructions say "erasable TypeScript only", and the model self-corrects like any other program error. A syntax-level failure never spawns a worker. +2. **Spawn one fresh `Worker` per run** from the package's own bootstrap module: `env: {}` (truly empty — stronger than the scrubbed-env rule for spawned commands), `resourceLimits` from config, `stdout`/`stderr` captured into `logs` rather than inherited. No pooling and no cross-run state: a program's world dies with its worker, which keeps runs reconstructable from the log alone and makes state bleed unrepresentable. +3. **Execute** in the bootstrap: the stripped program becomes the body of an `AsyncFunction` whose parameters are the binding globals and a capturing `console` shim, so top-level `await` and `return` work and the program's completion value is the run's `value` (structured-cloneable values cross as-is; anything else is replaced by its `util.inspect` rendering, documented). +4. **Bridge bindings over the message port**: each binding function in the worker posts `{ id, global, name, args }` and awaits the reply; the host validates the name against the request's bindings, invokes, and replies `{ id, ok, value }` or `{ id, ok: false, message }` (a host-side binding rejection becomes a program-side rejection). The worker-side namespace objects are built null-prototype via `defineProperty`, so a binding named `__proto__`, `constructor`, or `toString` is an ordinary own property, not a prototype collision. Unknown names, duplicate ids, and post-settlement messages are rejected or ignored — the port protocol assumes a hostile peer, because the peer runs model code. +5. **Enforce caps — two independent budgets, because the peer is hostile.** The compute budget (`computeMs`) meters the worker's *measured busy time* via `worker.performance.eventLoopUtilization()` polling — not host-side "is an RPC pending" bookkeeping, which a program defeats by firing an un-awaited call at a slow tool and then spinning hot while the host thinks it is waiting. Measured busy time cannot be gamed: a hot loop accrues it whether or not a dispatch is in flight, and a program genuinely awaiting a slow tool accrues none, so a long-running `bash` sub-call still does not kill an innocent run. The wall ceiling (`maxWallMs`) never pauses for anything and backstops what busy-time cannot see (a program awaiting a promise nobody will resolve). Budget expiry, `signal` abort, and run completion all funnel into `worker.terminate()`, which ends hot synchronous loops too (measured; this was `node:vm`'s unfixable gap); the failure reports which budget fired. Heap overflow surfaces as the worker's OOM exit → `error.kind: 'worker-exit'`. Log and value sizes are capped by config, truncation marked in-band. All caps are validated config fields with defaults (`computeMs: 60_000`, `maxWallMs: 600_000`, `maxLogBytes: 65_536`, `maxValueBytes: 32_768`, `maxOldGenerationSizeMb: 512`), changeable from `cordis.yml`. +6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../defensive-patterns.md). + +### Trust posture + +The worker runtime is **containment, not a security boundary**, and the RFC says so without ceremony. Model code in the worker can reach Node globals — `fetch`, `process` (with an empty env), dynamic `import()` of built-ins — so a deliberately adversarial program has ambient authority comparable to what the harness's own `bash` tool already grants every model turn: `dsh-bash-local` runs arbitrary model-written commands with the host filesystem, network, and a scrubbed-but-populated environment. One asymmetry runs the other way and is stated plainly: `worker.terminate()` ends the thread, not OS processes a program may have spawned via `node:child_process` — weaker than `bash-local`'s process-group kill for direct children (equivalent for double-forked daemons, which survive both); the wall-clock ceiling bounds the worker itself, and orphan cleanup is the same deployment-level concern it already is for bash. Code Mode is gated where bash is gated — `tools/pre-execute`, where permission/sandbox plugins veto or approve the program before it runs — and adds containment bash does not have: empty env, heap caps, hard termination of the program itself, a separate isolate. The earlier draft's two-flag unsafe ceremony (`{ unsafe: true }` constructor + `allowUnsafeRuntime`) existed for a `node:vm` stub with *no* containment and is dropped with it; demanding scarier flags for the better-contained executor than for bash would be posture theater. A deployment that needs a hard boundary (untrusted multi-tenant input) needs it for bash too; that is a future `isolation: 'container'` backend, and the `isolation` descriptor exists so such a deployment can tell backends apart. + +### What the model sees + +The `tools:sdk` section carries the `.d.ts` plus fixed instructions: the program is the body of an async TypeScript function (erasable syntax only — no `enum`/namespaces; type annotations are advisory); call tools as `await tools.name(args)` (quoted access for exotic names); a failed tool call **rejects** with an `Error` carrying the tool's error text — catch it to handle and continue; calls run **sequentially** even under `Promise.all`; emit results via `return` and/or `console.log`, and only that curated output returns to the context — intermediate tool results never do. That last line is the payoff the whole design serves: output-side context cost becomes the model's own editorial decision. On the input side the `.d.ts` is not free — for a large tool surface it can rival the native JSON schemas it replaces (and `'both'` pays for the two side by side) — but it is prefix-stable, so provider prefix caching amortizes it; the win is workload-dependent and the RFC claims no more. + +## Plan + +Four stacked PRs, each gates-green (`typecheck`, `lint`, `test:coverage`, `test:snapshot`, `doc-sync`, `verify-module-graph`, `build`, `hygiene`) with docs updated in the same change: + +1. **This RFC revision** (docs-only): the file rewritten as above (renamed `2026-06-15-code-mode.md`, same first-proposed date), regenerated RFC index. +2. **`dsh-code-runtime`** (interface package): the group `packages/code-runtime/`, abstract `CodeRuntime`, vocabulary types, ctx-key declaration; docs in the same change — group README + package README, the `packages/README.md` group table row, the `ctx.codeRuntime` row in [docs/architecture.md](../../../architecture.md)'s service map, and the regenerated cordis catalog (the new service class). Unit tier: HMR safety (dispose removes `ctx.codeRuntime`), contract docs. This package has no behavior to snapshot or e2e; its coverage story is unit-only by design. +3. **`dsh-code-runtime-worker`**: the implementation above, plus the regenerated config catalog (its `Config`). Unit tier (real workers, no mocks — they are cheap and local): output/value capture, log source attribution, error kinds (exception incl. non-erasable syntax, abort, worker-exit under OOM), the two budgets from both sides (a hot loop with an un-awaited pending dispatch still dies at `computeMs` busy time; a program idling on a slow binding outlives `computeMs` and dies only at `maxWallMs`), binding bridge hostility cases (unknown name, duplicate id, post-settlement message, `__proto__`/`constructor`/`toString` binding names), structured-clone fallback, cap truncation, `env` emptiness verified from inside the program, dispose-awaits-exit. A real-load-path test runs the built package (`lib/`) so the worker entry resolves both unbuilt (tsx) and built — the published-bin guard from [docs/testing.md](../../../testing.md). +4. **Native code mode in `dsh-tools`** + the end-to-end surface: mode config, provider switch, `tools:sdk` section, `jsonSchemaToTs`, `run_code` + dispatch bridge + serialization queue, `tool/code-dispatch` event; regenerated tool, config, and persistence catalogs; [docs/architecture.md](../../../architecture.md) (tool-pipeline prose) and the [adding-a-tool cookbook](../../../cookbook/adding-a-tool.md) cross-reference updated in the same change; an `examples/` leaf + `demo:code` script wiring the worker runtime with `mode: 'code'`; move this RFC to `implemented/`. Coverage named per tier now, per the plan-time rule: **unit** — codegen table (DSL subset, quoted names, `unknown` degradation, determinism), provider contribution per mode, `toolOrder × mode` rejection, missing-runtime/wrong-language loud failures, serialization non-overlap (a probe tool records enter/exit under `Promise.all`), abort stops the queue, binding rejection on `isError`, `CodeRunFailedError` → structured `isError`, event payloads, `deriveMessages()` ignores the event, HMR safety (mode flip via config reload removes tool + section); **e2e (with-key, self-skips)** — a real model, `mode: 'code'`, a task requiring two tool calls and curation, asserting the wire tool list was exactly `run_code` and the transcript's dispatch events; **snapshot (keyless replay)** — goldens for a `run_code` turn in `'code'` and `'both'`, pinning the SDK section text, the collapsed header tools, dispatch events, and the result card. + +The four PRs land in order (each on the previous); per stacked-review practice, review fixes land on the PR that introduced them and merge down. + +## Alternatives considered + +**An add-on consumer plugin, zero core changes (the previous draft of this RFC).** Rejected on both halves. The wire-collapse half aged out from under it: it targeted the `agent/request` waterfall, which [reconstructable requests](../../implemented/architecture/2026-07-05-reconstructable-requests.md) has since re-typed to call-config-only, and the surviving alternative — transforming the assembly a waterfall listener receives — is strictly worse than contributing the right list in the first place (transformation must undo `toolOrder` canonicalization it cannot see the config for, and its correctness depends on where it sits in a listener chain). The deeper reason is ownership: which tools the model is offered, in which representation, is the registry's single concern — `schemas()` for function calling and the SDK for Code Mode are two projections of one store, and splitting the second projection into a satellite package would preserve a boundary the domain does not have. + +**`node:vm` as the reference runtime, hardening deferred (also the previous draft).** Rejected: `node:vm` is not isolation (prototype-chain escapes reach the host realm), cannot interrupt a hot loop, and forced the draft into a two-flag unsafe ceremony plus a mandatory follow-up RFC. The worker thread delivers the missing properties now — separate isolate, empty env, `resourceLimits`, reliable `terminate()` (all verified by probe before this revision) — at bash-equivalent trust, so the reference implementation and the production one are the same package and the ceremony dissolves. + +**Result elision / summarization over native tool-calling.** Addresses only the context-bloat half of the problem: trimming old `tool-result`s (now cheap to add as a logged surface replace, per the reconstructable-requests consequences) still pays one model round-trip per call and cannot express loops, branches, or joins. Complementary, not competing; it can layer under Code Mode for residual native calls. + +**Parallel native dispatch in the loop.** The other answer to round-trip cost; still valid future work (the open TODO), still blocked on concurrency-safety metadata, and still no composition — it parallelizes calls the model already decided on in one step. Code Mode's serialized-queue decision keeps the two compatible: when the metadata lands, both native parallel dispatch and per-tool binding parallelism unlock together. + +**Always-exclusive (Cloudflare-faithful, no mode).** Rejected for this SDK's primary consumer: a coding agent's bread-and-butter single calls (`bash`, `read`, `edit`) are already ideal as native calls, and forcing every edit through a program taxes the common case. The mode config keeps the faithful form (`'code'`) one line away without imposing it. + +**Per-tool visibility tiers (this tool native, that tool code-only).** Deferred again, knowingly: it needs per-tool metadata and a presentation split that `'native' | 'code' | 'both'` does not, and every learning it depends on (how models actually split usage under `'both'`) arrives only after this ships. + +**Sanitized identifier aliases in the SDK** (`my-tool` → `my_tool`, Cloudflare's approach). Rejected: quoted keys on a `declare const` make every name reachable with zero alias-collision logic; models handle `tools["my-tool"](…)` fine. + +**A REPL-style persistent kernel** (state survives across `run_code` calls). Rejected for the MVP: cross-call state would be invisible to the session log, breaking the reconstructability guarantee that every request is a pure function of the log; fresh-per-run keeps it. A kernel-style backend remains expressible behind the seam later, with its own logging story. + +## Acceptance criteria + +- `mode: 'native'` (and unset) is byte-for-byte today's behavior: same assemblies, same headers, same snapshots. +- Under `mode: 'code'`, the assembled tool list (and thus the logged `request/header`) is exactly `[run_code]`; under `'both'`, every native schema plus `run_code`; the `tools:sdk` section is present in both, absent under `'native'`, and its text is deterministic for a fixed tool set (byte-identical across consecutive assemblies). +- The generated `.d.ts` covers every registered tool except `run_code`, non-identifier names via quoted keys, unsupported schema constructs as `unknown`, without codegen ever throwing. +- A program calling two tools returns only its curated output; each sub-call appears as a `tool/code-dispatch` event ordered by log `seq`, flows through `tools/pre-execute`/`post-execute` (a deny reaches the program as a binding rejection), and never enters derived messages; a binding argument that does not survive JSON normalization (`BigInt`, a circular structure) rejects before dispatch — nothing executes unlogged. +- `Promise.all` over three bindings produces non-overlapping `ctx.tools.execute()` intervals (probe-tool assertion); aborting mid-program stops the worker and dispatches nothing further; a budget expiry during a slow sub-dispatch aborts that dispatch (the probe tool observes its signal fire), `run_code` returns only after the queue drains, and no `tool/code-dispatch` event lands after `run_code`'s own `tool/result` in the log. +- Worker runtime: a hot `for(;;){}` run ends at the `computeMs` busy-time budget with `error.kind: 'timeout'` — including when the program fired an un-awaited binding call first (the pending-RPC decoy); a program idling on a slow binding does not consume `computeMs` and is bounded only by `maxWallMs`; OOM under `resourceLimits` yields `kind: 'worker-exit'` with the host process healthy; `process.env` inside a program is empty; non-erasable syntax yields `kind: 'exception'` without a worker spawn; disposal awaits worker exit. +- Misconfiguration is loud before any model request: non-native mode with no `ctx.codeRuntime`, a runtime whose `language ≠ 'typescript'`, and `toolOrder` naming a non-contributed tool all reject the assembly with actionable messages. +- The demo runs against the real API via `demo:code`; the snapshot goldens replay keylessly; all repo gates pass on every PR of the stack. + +## Risks + +**The worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool, containment exceeds it, gating uses the same seams. Deployments needing more need a future `isolation: 'container'` backend — tracked as the seam's designed extension, not a TODO on this design. + +**`stripTypeScriptTypes` is marked experimental.** It is the same engine (amaro/swc) behind Node's own native `.ts` execution, exposed as an API across this repo's whole engines range. Mitigations: the runtime's unit suite pins the behaviors relied on (position preservation, erasable-only rejection message shape loosely), the call sits behind one private function, and `amaro`/`sucrase` are drop-in replacements if the API shifts. The erasable-only subset is a model-facing contract line, and the error path is a working feedback loop, not a dead end. + +**Prompt cost of the SDK, especially under `'both'`.** The `.d.ts` can rival the native schemas it complements; `'both'` carries two representations. Prefix stability + provider caching amortize per-session cost; the mode is per-deployment; the RFC makes no unconditional-savings claim. Measured guidance (when to prefer which mode) is explicitly post-ship learning. + +**Registry scope growth.** `dsh-tools` absorbs codegen, a tool, a bridge, and an event. Contained by module boundaries inside the package (`ts-types.ts`, `code-mode.ts` beside `schema.ts`/`json-schema.ts`/`presentation.ts`) and by the seam: everything substrate-shaped lives behind `ctx.codeRuntime`. + +**Structured-clone limits at the binding boundary.** The seam's clone boundary admits values JSON does not (`Date`, `Map`, `BigInt`), and the session log accepts only JSON — left unhandled, a sub-call could execute and then fail at `tool/code-dispatch` append time. Closed by the bridge's JSON-normalization step (§ the dispatch bridge): what does not survive the round-trip rejects that binding call before dispatch, so every executed sub-call is loggable by construction. The seam itself keeps the wider structured-clone contract (it is about the port, and stated so a future binding producer cannot discover it in production); consumers with stricter payload needs enforce them at their own boundary, as the bridge does. Non-text sub-result content is reduced to placeholders — a known MVP limitation, recorded in the SDK instructions. + +**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs. + +**Budget metering reads the event loop, not a flag.** Busy-time polling (`eventLoopUtilization()`) is coarser than an exact CPU meter — a budget expires up to one poll interval late — and its correctness claim ("a pending dispatch cannot pause it") is load-bearing against a hostile program. Both sides are unit-tested (hot loop with a pending decoy dispatch dies at `computeMs`; idle-on-slow-binding survives to `maxWallMs`), and the poll interval is an internal constant, not config — nothing a deployment could mis-tune into a bypass. diff --git a/docs/rfc/proposed/feature/2026-06-15-optional-code-mode.md b/docs/rfc/proposed/feature/2026-06-15-optional-code-mode.md deleted file mode 100644 index e221618ce1..0000000000 --- a/docs/rfc/proposed/feature/2026-06-15-optional-code-mode.md +++ /dev/null @@ -1,119 +0,0 @@ -# RFC: Optional Code Mode — model writes TypeScript against an SDK of all tools - -Status: proposed - -> Premise partially stale: this proposal predates [request reconstructability](../../implemented/architecture/2026-07-05-reconstructable-requests.md) — `agent/request` now shapes call config only (no request/content mutation), so the interception points named below need re-mapping onto the log channels and `system-prompt/assemble` before implementation. - -## Problem - -Today the agent loop advertises every registered tool to the model as a native JSON-schema function definition. `ToolRegistry` feeds its schemas into `ctx.systemPrompt`, the loop puts them on `GenerateOptions.tools`, and the adapter serializes them to the provider's function-calling wire format. The model then invokes one `tool-call` block per step, the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../architecture.md)), and **every** intermediate `tool-result` re-enters the model's context on the next request. - -For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each of those round-trips drags the entire intermediate result back into context whether the model needs it or not. - -Cloudflare's [Code Mode](https://blog.cloudflare.com/code-mode/) (shipped as the `@cloudflare/codemode` npm package) proposes an alternative grounded in a simple observation: LLMs are better at writing code than at emitting tool calls, because they have seen millions of lines of real code and comparatively few contrived tool-calling traces. Instead of one tool call per step, the model writes a TypeScript program against a generated SDK that wraps all the tools, and that program is executed. The model curates what comes back — only what it `console.log`s and/or returns — instead of every intermediate result. The SDK functions are async, so the model can *express* fan-out (`Promise.all`) naturally in code; this RFC initially **serializes** those dispatches (§ Concurrency) until the tool contract grows concurrency-safety metadata, so the early win is composition and fewer round-trips, not parallelism. - -This RFC proposes an **optional** Code Mode for the DeepSeek Harness, covering **all** tools uniformly — built-in and future MCP — with no per-tool work, implemented Cordis-style with **zero core-package changes**. It fully specifies the code-execution seam and the SDK-generation pipeline, but ships only a minimal `node:vm` reference stub for execution; the hardened, sandboxed execution substrate is **deferred to a follow-up RFC** (see Risks). This RFC does not change the agent loop, and it leaves native tool-calling exactly as it is — Code Mode is a plugin you load, not a replacement. - -## Proposal - -The design follows the codebase's capability-seam pattern ([capability seams](../../implemented/architecture/2026-06-13-capability-seams.md), the `bash` template) as a three-package split, plus one consumer plugin. Nothing in `dsh-session`, `dsh-agent`, `dsh-agent-loop`, `dsh-llm`, `dsh-tools`, or `dsh-system-prompt` changes. - -**Prior art.** `@cloudflare/codemode` validates this shape directly and several of its decisions are adopted below. Its `Executor` interface is deliberately tiny — `execute(code, fns) → { result, error?, logs? }` — with a production `DynamicWorkerExecutor` (isolated Workers) and a six-line `NodeVMExecutor` example as two implementations behind it: exactly the interface/implementation split [the capability-seam pattern](../../implemented/architecture/2026-06-13-capability-seams.md) prescribes. It generates TypeScript type definitions from tools for the model's context and runs the generated JavaScript in a sandbox, capturing console output alongside the return value. It normalizes model output into an async arrow function via AST parsing (acorn) and sanitizes tool names into valid JS identifiers (`my-tool` → `my_tool`, `delete` → `delete_`). It blocks outbound network by default. The transferable lessons — minimal executor contract, host-side type derivation, capture-output-and-return-value, name sanitization, AST-normalize the code, isolate by default — are folded into the design below. What does **not** transfer is the substrate: Cloudflare's isolation is Workers-specific; our equivalent hardened substrate is the deferred follow-up. - -**Prompt-budget tradeoff (Code Mode is not unconditionally cheaper).** Deriving the SDK types host-side costs no extra *discovery* round-trip, but the generated `.d.ts` is injected into the system prompt (§3a), so the type definitions themselves **do** consume context — and for an all-tools SDK that cost scales with every registered tool and can be comparable to, or larger than, the native JSON schemas it replaces. Code Mode's saving is on the **output/result** side (the model curates what comes back; intermediate results never re-enter context) and on **round-trips** (compose many calls in one program), not on the input-side tool description. The net win is workload-dependent: it pays off for multi-call, large-intermediate-result workflows and can cost *more* for a single call against a large tool surface. The `.d.ts` section is a prefix-stable prompt prefix, so prompt caching amortizes its per-turn cost across a session; the RFC notes that caching is what keeps the injected SDK affordable, and that a deployment with a very large tool surface should weigh the SDK size against native schemas rather than assume Code Mode is strictly cheaper. - -**1. Interface package `packages/code-runtime/`** — a new package `@deepseek-ai/dsh-code-runtime` owning `ctx.codeRuntime`, depending only on `cordis`. It defines an abstract `CodeRuntime extends Service` plus the execution vocabulary. The runtime knows **nothing** about `ctx.tools`: it is handed a set of named async functions (the resolved SDK bindings), runs the program, and captures output. The result shape mirrors Cloudflare's proven-minimal contract so an error is a *field on a resolved result*, not a throw the runtime is expected to make: - -- `CodeRunRequest = { code: string; sdk: SdkBinding[]; signal?: AbortSignal }` -- `CodeRunResult = { result: unknown; logs: string[]; error?: string }` -- a readonly `safe: boolean` on the `CodeRuntime` service — `false` for an unsandboxed stub, `true` only for a real isolating substrate; consumers gate on it (§2). -- `SdkBinding = { namespace: string; fns: Record Promise> }` - -Per the "explicit > implicit at seams" convention, the request spells out every field the runtime acts on; defaulting (e.g. an output cap, a timeout derived from `signal`) is the implementation's explicit job, not a hidden `?? default` inside `run()`. The split into interface + implementation is justified under [the capability-seam pattern](../../implemented/architecture/2026-06-13-capability-seams.md) because there is **genuinely more than one planned implementation** — the node:vm stub *and* the hardened substrate (a real isolate, or the generated program run as a sandboxed process through the existing `ctx.bash` seam) that is scheduled follow-up work, not speculative optionality. The capability-seam pattern warns against splitting preemptively when only one implementation is conceivable; here a second is not just conceivable but required before any untrusted use, so the seam earns its keep. - -**Backends can differ by language/runtime, not only by trust level.** The two implementations above (unsafe stub vs. hardened substrate) differ along the *trust* axis while staying TypeScript/JS, but nothing in the `CodeRuntime` contract — a program string plus a set of named async SDK bindings in, and a `{ result, logs, error? }` out — is bound to one source language. The same seam can host backends that differ along the *language* axis, executing a program written in something other than TypeScript. Two illustrative directions: - -- **An AssemblyScript backend.** AssemblyScript is a strict TypeScript subset that compiles to WebAssembly, so a program stays familiar to a TS-fluent model while the WASM boundary supplies exactly the sandboxing the hardened substrate is meant to provide — memory isolation and no ambient host authority come from the runtime rather than from after-the-fact hardening of `node:vm`. This is an appealing route to a `safe = true` backend. -- **A Python backend.** Python is arguably the model's most native language — it has seen far more real Python than any tool-calling trace — which is the same "LLMs write better code than tool calls" argument that motivates Code Mode, taken one step further. A Python backend is itself a sub-seam over different Python *runtimes*: **CPython** (in-process or a sandboxed subprocess via `ctx.bash`) for maximum fidelity and ecosystem access, or a more controllable / embeddable interpreter — Pyodide (CPython on WASM), RustPython, or a restricted embedded interpreter — when isolation, deterministic resource limits, or a clean capability boundary matter more than running arbitrary native extensions. - -These are illustrations of the seam's reach, **not commitments** — the MVP ships only the TypeScript path. The honest caveat is that the *execution* contract is language-agnostic but the *presentation* is not: the SDK-generation pipeline below (§3a and the `jsonSchemaToTs` codegen, which emits a TypeScript `.d.ts`) is TypeScript-specific, so a non-TS backend pairs the shared `CodeRuntime` contract with its own language-appropriate SDK generator and system-prompt section (a `.pyi` stub and Python usage instructions for the Python backend, AssemblyScript-flavored types for that one). The runtime seam is reused as-is; only the codegen/prompt half is per-language. - -**2. Implementation package `packages/code-runtime-vm/`** — a new package `@deepseek-ai/dsh-code-runtime-vm`, the `node:vm` reference stub. It type-erases the model's TypeScript via the compiler's `transpileModule` (or sucrase) — the types exist only to guide the model; the runtime is plain JS — then wraps the body in an async IIFE for top-level `await` (Cloudflare's `NodeVMExecutor` does literally `new AsyncFunction("codemode", "return await (${code})()")`), runs it in a `vm.Context` whose globals are a capturing `console` and the SDK namespace objects, awaits the IIFE, and captures the return value, the buffered logs, and any thrown error (as `error: string`). It applies an **output cap** (truncate captured logs) and a **timeout tied to `request.signal`**. These caps limit blast radius; **they are not a security boundary**. node:vm is **not** isolation: withholding `require`/`process` does not contain anything (code escapes via `constructor`/prototype reflection), and per [AGENTS.md](../../../../AGENTS.md) the harness must never hand model output the ambient environment. - -**The unsafe-runtime guard is enforceable, not a README warning.** Because a README caveat is not a control — and AGENTS.md's "never hand model output ambient authority" is a hard rule, not advice — the design makes the danger refuse to run by construction. Two layers: - -- **The runtime declares its trust level.** `CodeRuntime` carries a readonly `safe: boolean` (a `node:vm`-class stub returns `safe = false`; a real isolate/sandboxed-process substrate returns `safe = true`). The `code-runtime-vm` constructor *additionally* requires an explicit opt-in — `new VmCodeRuntime({ unsafe: true })` — and **throws** if that flag is absent, so merely depending on the package and wiring it cannot silently produce a live unsafe runtime; the operator must type the word `unsafe`. -- **The consumer refuses to expose `run_code` over an unsafe runtime by default.** When `code-mode` initializes, if `ctx.codeRuntime.safe === false` it does **not** register `run_code` unless the plugin itself is configured with an explicit acknowledgement (e.g. `code-mode` config `allowUnsafeRuntime: true`). Absent that, it logs a typed error and registers nothing — so a real model never reaches an unsandboxed runtime by a single config slip. The refusal path is tested: with the acknowledgement unset and an unsafe runtime, `run_code` is absent (and the wire tool list is unchanged from native); with both opt-ins set, it registers and runs. This keeps the unsafe reference backend usable for tests and trusted local demos while making production misuse take two deliberate, greppable flags rather than one mistake. - -`code-runtime-vm` is therefore documented as **reference / test-only / unsafe-for-untrusted-input**, acceptable in the MVP only because the code runs at harness trust *and* both opt-in flags must be set. Signal handling is best-effort: it aborts in-flight sub-dispatches but cannot reliably interrupt a hot synchronous loop (`while(true){}`) in node:vm — another reason the hardened substrate is deferred, not optional-forever. - -**3. Consumer plugin `packages/code-mode/`** — a new package `@deepseek-ai/dsh-code-mode`, the plugin that wires everything together. It declares `inject = ['tools', 'systemPrompt', 'codeRuntime']` — Cordis throws on access to a service that is not injected, and keeps the plugin inactive until all three exist (the same pattern as `tool-bash`'s `inject = ['tools', 'bash']`), which also gives correct load-ordering relative to `code-runtime`/`code-runtime-vm`. The plugin contributes four things, all through existing seams: - -**3a. Tool presentation — a lazy system-prompt section (the injection seam already exists).** `dsh-system-prompt` already provides the Cordis-idiomatic way for any plugin to inject prompt snippets: `ctx.systemPrompt.section({ name, order, text })`, fiber-scoped and auto-disposed via `ctx.effect()`, where `text` may be a lazy `() => string` re-evaluated at each assembly. No new mechanism is needed or invented. Code Mode registers a lazy section (high `order` so it lands last) whose thunk reads `ctx.tools.schemas()` at assembly time and regenerates the SDK `.d.ts` plus usage instructions from the currently-registered tool set. Because the thunk reads the live registry, coverage of every tool — built-in, MCP, future — is automatic. - -**3b. Wire tool-list enforcement — an `agent/request` listener (the authoritative seam).** The goal "exactly one tool reaches the wire" must be enforced where the wire request is finalized. The loop calls `ctx.systemPrompt.assemble()` first, *then* builds `GenerateOptions` (seeding `tools` from `assembly.tools`), *then* runs the `agent/request` waterfall, *then* calls `ctx.llm.stream()`. A `system-prompt/assemble` listener can only influence the *seed*; `agent/request` is the last seam before the model call, so it is authoritative. The plugin registers an `agent/request` listener that does `const final = await next(); return { ...final, tools: [runCodeSchema] }` — overriding the value *returned by* `next()`, not the inbound argument, so it dominates the cooperative request listeners it wraps. It registers with `prepend: true` to sit at the outer edge of the waterfall chain. One honest caveat, stated in the RFC body: `ctx.llm.stream()` itself runs a further `llm/stream` waterfall before the adapter, so the guarantee is "authoritative within the agent request pipeline," not an absolute wire invariant; if a hard invariant is ever required, a defensive `llm/stream` assertion with a spy adapter covers it in tests. - -**3c. The single tool — `run_code`.** Registered normally in `ctx.tools` with one parameter `{ code: string (required) }`. Because it is an ordinary tool, the unchanged loop dispatches it through the normal path — this is the crux of "zero loop changes." Its `execute(args, exec)`: - -1. Builds the SDK bindings. For each real tool, an async `invoke(callArgs)` that **checks `exec.signal?.aborted` (throwing if set) before and after** calling `ctx.tools.execute({ callId: , name, arguments: callArgs, agent: exec.agent, signal: exec.signal })`, then maps the resulting `ContentBlock[]` to a simplified `{ output, isError }` (text blocks for the MVP), and emits an observability event. The explicit abort check matters because `ctx.tools.execute()` *catches* thrown tool errors and converts them to `isError` results — without the check, an aborted sub-call would look like ordinary error data and the program would keep running instead of stopping. Sub-dispatch still flows through the `tools/pre-execute`/`tools/post-execute` waterfalls, so permission/sandbox/hook plugins apply to code-mode calls exactly as to native ones. -2. Calls `ctx.codeRuntime.run({ code: args.code, sdk: bindings, signal: exec.signal })`. -3. Surfaces the outcome. A *successful* run returns `[{ type: 'text', text: }]`. A *runtime-error* result cannot be reported by returning content, because a normal `ToolDefinition.execute()` returns only `Promise` and `ToolRegistry.execute()` hardcodes `isError: false` on any successful return — `isError: true` arises only from the registry's catch path. So on an error result the tool **throws a `CodeRunError extends HarnessError`** (`HarnessError` is exported from `dsh-llm`; the registry catch turns any throw into `isError: true` with the message as text, and a `HarnessError` additionally carries structured `{ name, code }`). An alternative — registering `run_code` handling as a `tools/execute` listener that returns a full `ToolExecutionResult` and can set `isError` directly — is noted; the throw is simpler and preferred. - -**3d. Result discipline — what the model receives.** The model gets back **only the captured console output and/or the program's return value** (the model chooses which to surface). Intermediate sub-call results are **never** returned to the model. This is the core context-saving benefit: the agent curates its own output, exactly as a script's stdout curates a pipeline's intermediate state. - -**Sub-call CallIds.** Real tool calls dispatched from inside `run_code` need ids, but `CallId` is normally provider-issued (a branded string for correlating a call with its result — only brand-wrapped via `CallId()`, with no generator and no documented session-global-uniqueness guarantee). The plugin mints deterministic sub-ids scoped to the parent: `` `${exec.callId}:code:${n}` `` with a per-run counter `n`. These are unique within one `run_code` run (assuming the parent `callId` is unique, which the provider guarantees per turn); the `code/dispatch` event additionally carries the session log's `seq` so the UI and persistence can order and disambiguate globally without relying on the id alone. `ToolExecution.agent` is optional; the normal loop always supplies it (and with it `exec.agent.session`, the log `code/dispatch` appends to). A `run_code` execution arriving without `exec.agent` still runs (sub-calls propagate `agent: undefined`, exactly as the loop's own contract allows) but **skips session-log observability** — with no session to append to, those direct runs are simply not logged. - -**Observability without context cost.** Each sub-dispatch emits a session event **declared by the `dsh-code-mode` plugin itself** via `SessionEventMap` declaration merging (the map is merge-extensible precisely so plugins can add events without touching `dsh-session`). Shape: `code/dispatch` with `{ parentCallId, subCallId, name, arguments (or redacted), isError, summary }`, ordered by the session log's own `seq`. `deriveMessages()` does **not** translate it into a model message — an unknown event type falls through its `default`, per the merge-extensible-union convention — so the UI and persistence ([session persistence](../../implemented/architecture/2026-06-14-session-persistence.md)) can render every sub-call while the model's context only ever receives the single `run_code` tool-result. Because the event lives in the plugin, this adds no core change. - -**SDK codegen.** A pure `jsonSchemaToTs(schema)` in `code-mode` maps the JSON-schema subset the `defineTool` DSL produces (object/string/number/boolean/array, `properties`, `required[]`, `enum` → string-literal union, nested objects, array `items`) to a TS type literal. It is **total**: any unsupported construct (`$ref`, `oneOf`/`anyOf`, `integer`, `null`, `additionalProperties`, or any raw MCP shape it does not recognize) degrades to `unknown` without throwing — it never crashes codegen. Typing is best-effort, not a guarantee, because MCP tools accept arbitrary JSON Schema and `ToolSchema.parameters` is typed only as `Record`. Because `ToolSchema.name` is an arbitrary string (not necessarily a valid TS identifier), the SDK is generated as a **namespace with quoted access** (e.g. `tools["some-mcp-tool"](args)`) plus safe camelCase aliases where the name is a clean identifier; alias collisions and TS reserved words fall back to quoted-only access (no duplicate alias emitted). This mirrors Cloudflare's `sanitizeToolName`. `run_code` itself is filtered out of the SDK. The MVP surfaces text content only; image and other block types in sub-results are deferred (noted as a limitation). - -**Concurrency — serialized by default (the binding must enforce it).** The SDK functions are async, so a model writing `await Promise.all([tools.a(...), tools.b(...)])` would *start both* immediately, and each would call `ctx.tools.execute` right away — i.e. the binding shape makes concurrent dispatch the **default**, not an opt-in. Because the tool contract carries **no concurrency-safety metadata today** (parallel tool execution and a concurrency-safety hint are an open TODO in both `dsh-tools` and [docs/architecture.md](../../../architecture.md): "phase 1 executes tool calls sequentially"), concurrent dispatch through a not-yet-hardened tool may race. So a prose "may serialize" is not sufficient. **Decision: the MVP SDK bindings enforce serialization** — each `run_code` invocation owns a per-run dispatch queue, and every `invoke()` chains onto it (`tail = tail.then(() => ctx.tools.execute(...))`), so even `Promise.all` over SDK calls executes them one at a time in submission order. This is a hard acceptance criterion, with a test that issues `Promise.all([...])` from a program and asserts the underlying `ctx.tools.execute` calls did **not** overlap (e.g. a probe tool records enter/exit and the test asserts no interleaving). The `.d.ts` may *describe* the model-visible functions as async (they are), but the implementation guarantees serial execution. Lifting serialization is deferred: only once a tool can declare itself read-only / concurrency-safe does the binding allow those specific tools to overlap. The same per-run queue is where the before/after abort checks (§3c) live, so an aborted run drains no further queued dispatches. - -**Tool visibility tiers (design intentionally skipped).** A natural extension is to mark each tool with a *visibility tier*: some tools "direct-call eligible" (still offered as native wire tools alongside `run_code`), some "code-mode only" (reachable solely from within a `run_code` program, never on the wire), and the default "both." This would let a deployment keep a few high-frequency or approval-gated tools as direct calls while routing the long tail through Code Mode, or hide composition-only primitives from the native surface entirely. This RFC notes the possibility but **intentionally skips the detailed design** — the per-tool metadata, how it interacts with the `agent/request` enforcement in 3b, and the presentation split in 3a are left to a follow-up. The MVP is the simple two-state model: Code Mode on (everything via `run_code`) or off (everything native). - -**Optionality / toggle.** Loading the `code-mode` plugin enables Code Mode for that context; not loading it leaves today's native tool-calling untouched. The two are mutually exclusive within one ctx, because Code Mode rewrites the wire tool list down to `[run_code]`. Per-agent selection via ctx forks, and the visibility tiers above, are future work; the MVP toggle is plugin presence. - -## Alternatives considered - -**Result elision / summarization over native tool-calling (the narrower route).** The Problem has two halves — context bloat (every intermediate `tool-result` re-enters context) and serial composition (one tool call per round-trip). The context-bloat half can be addressed *without* any code-execution runtime: keep provider tool-calling exactly as it is, and add a plugin on the `agent/request` waterfall (or a compaction pass akin to [the session-persistence work](../../implemented/architecture/2026-06-14-session-persistence.md)) that elides or summarizes older `tool-result` blocks before they re-enter the model's context — drop them past a window, replace large payloads with a digest, or keep only the blocks the model still references. This is strictly less invasive than Code Mode: no new runtime seam, no model-written programs, no new safety surface. It is the right tool if context growth is the only pain. - -It is insufficient for the **composition / round-trip** half, which is the decisive reason this RFC does not stop there. Elision still pays one model round-trip per tool call: a loop over N items is N turns, a branch on an intermediate value is a turn to fetch then a turn to act, and post-processing (filter, join, reduce) either happens in the model's head over full payloads or not at all. Code Mode collapses all of that into one program — the loop, the branch, the join run in the runtime, and only the curated result returns. Elision also cannot express fan-out or data-dependent control flow; it only shrinks what comes back. So the two are complementary, not competing: elision could even layer *under* Code Mode for the residual native-tool paths. The RFC chooses Code Mode because the round-trip/composition cost is the larger structural limit, and accepts the new code-execution surface as the price — which is exactly why the execution substrate is gated behind the enforceable safety guard (§2) and the hardened backend is a hard prerequisite for untrusted use. - -**Why not change the loop to dispatch native tool calls in parallel instead?** That is the other obvious answer to the round-trip cost, and it remains valid future work (it is the open `dsh-tools`/architecture.md TODO). But it is a core-loop change requiring the same concurrency-safety metadata Code Mode defers, and it still does not give the model *composition* (branch/loop/post-process between calls) — only parallelism of independent calls the model already decided to make in one step. Code Mode delivers composition with zero core change; parallel native dispatch and Code Mode can coexist later. - -## Plan - -1. Scaffold the interface package `packages/code-runtime/` per [the cookbook](../../../cookbook/adding-a-package.md): abstract `CodeRuntime extends Service` (`super(ctx, 'codeRuntime')`) with a readonly `safe: boolean`, the `declare module 'cordis'` ctx key, the `CodeRunRequest`/`CodeRunResult`/`SdkBinding` vocabulary, method contracts documented in JSDoc (what `run` captures, abort semantics, that an error is a result field not a throw, what `safe` means). HMR-safety test (dispose the contributing fiber, assert `ctx.codeRuntime` is gone). -2. Scaffold the implementation package `packages/code-runtime-vm/`: the node:vm stub — `safe = false`, a constructor that **throws unless given `{ unsafe: true }`**, transpile/type-erase, async-IIFE wrap, capturing `console`, SDK globals, return-value/logs/error capture, output cap, signal-tied timeout. Tests for output capture, return value, error-as-field, abort, the constructor refusal without `unsafe`, and a README documenting the "not a sandbox, trusted-only" caveat prominently. -3. Scaffold the consumer plugin `packages/code-mode/`: `jsonSchemaToTs` codegen with namespace/quoted-access + alias handling (unit tests, including non-identifier MCP names and unsupported-shape → `unknown`); the registered lazy `ctx.systemPrompt.section()` carrying the SDK `.d.ts`; the `agent/request` listener (`prepend: true`) collapsing `request.tools` to `[run_code]` after `await next()`; the **unsafe-runtime gate** (refuse to register `run_code` when `ctx.codeRuntime.safe === false` unless `allowUnsafeRuntime` is set); the `run_code` tool with the dispatch bridge (per-run serialization queue, deterministic sub-call ids, before/after abort checks, `CodeRunError` on error results); and the `code/dispatch` event declared here via `SessionEventMap` merge. Declare `inject = ['tools', 'systemPrompt', 'codeRuntime']`. -4. Tests: HMR-safety (dispose removes the tool, the section, and the listener); a waterfall test that the wire tool list is exactly `[run_code]` (spy adapter, asserting via `agent/request` and optionally `llm/stream`); an integration test that a program calling two tools returns only its printed/returned output (verify the world, not the self-report); a **serialization test** that `Promise.all([...])` over SDK calls does not overlap the underlying `ctx.tools.execute` invocations (a probe tool records enter/exit; assert no interleaving); `deriveMessages()` ignores `code/dispatch`; abort mid-program stops further dispatches; `CodeRunError` surfaces as `isError: true`; and the **unsafe-runtime refusal test** (§3, the VM-guard): with the unsafe flag unset, a non-mock agent's `run_code` is refused; with it set, the program runs. -5. Wire an example: `examples/coding-agent-code-mode` (or a config flag on the existing example) loading the trio. Running it against the node:vm stub requires both opt-ins (`VmCodeRuntime({ unsafe: true })` and `code-mode`'s `allowUnsafeRuntime`); the example sets them explicitly and comments why, or uses a mock model — a real model never reaches the unsandboxed stub without those deliberate flags. Add a `pnpm run demo:*` entry. -6. Docs: update [docs/architecture.md](../../../architecture.md) (a `ctx.codeRuntime` row in the service map, a Code Mode note under the tool pipeline / capability seams sections); add a [cookbook](../../../cookbook) note on writing a `CodeRuntime` backend; and **file the follow-up RFC for the hardened execution substrate** (the isolate/sandboxed-process design, the additional-language backends sketched in §1 — AssemblyScript/WASM, Python — with their per-language SDK generators, plus the tool-visibility-tier design skipped here). On landing, move this file to `implemented/` and update its row in [the RFC index](../../README.md). - -## Acceptance criteria - -- The three packages exist and pass their suites: `dsh-code-runtime` (the abstract seam), `dsh-code-runtime-vm` (the reference stub whose constructor throws without `{ unsafe: true }`), and `dsh-code-mode` (SDK codegen, the lazy prompt section, the `agent/request` collapse, the `run_code` tool). -- The wire tool list is exactly `[run_code]` under the plugin (spy-adapter test); the generated SDK covers every registered tool, with non-identifier names reachable via quoted access. -- A program calling two tools returns only its curated output; `code/dispatch` events land in the session log and never enter derived history. -- `Promise.all` over SDK calls does not overlap the underlying `ctx.tools.execute` invocations (the per-run serialization queue holds); an abort stops further dispatches. -- With `allowUnsafeRuntime` unset over an unsafe runtime, `run_code` is not registered and the wire tool list is unchanged from native. - -## Risks - -node:vm is not a sandbox. This is the single biggest caveat. Withholding `require`/`process` is not a boundary; the MVP runs at harness trust only; the hardened substrate is a hard prerequisite before any untrusted use and is the explicit subject of a follow-up RFC. The guard is enforceable, not just documented: the runtime exposes `safe: boolean`, the VM stub throws unless constructed with `{ unsafe: true }`, and `code-mode` refuses to register `run_code` over an unsafe runtime unless separately acknowledged (`allowUnsafeRuntime`) — production misuse requires two deliberate, greppable flags, and the refusal path is tested. - -Wrong seam would leak tools. If the wire tool list were enforced only in `system-prompt/assemble`, a later `agent/request` listener could re-add tools. Mitigation: enforce `request.tools = [run_code]` in the `agent/request` waterfall (the authoritative seam, run last before `llm.stream()`) with `prepend: true`, and assert exactly one wire tool in tests. The residual `llm/stream` caveat is documented, not hidden. - -Concurrency before the contract supports it. The binding shape makes concurrent dispatch the default, and the tool contract has no concurrency-safety metadata yet, so unguarded `Promise.all` over SDK calls could race a not-yet-hardened tool. Mitigation: the MVP bindings enforce a per-run serialization queue (every `invoke` chains onto the previous), with a test asserting `Promise.all` from a program does not overlap the underlying `ctx.tools.execute` calls. Per-tool parallelism is unlocked only once a tool can declare itself concurrency-safe. - -Two presentation modes to keep coherent. A tool added later must work in both native and Code Mode. Mitigation: both the codegen thunk and the `agent/request` listener read `ctx.tools.schemas()`, so coverage is automatic; a test asserts every registered schema produces valid `.d.ts`, including non-identifier MCP names via quoted access. - -Type-erased runtime is not type-checked. The model can write code that type-checks against the advisory `.d.ts` but throws at runtime, and MCP-schema typing is best-effort. Mitigation: errors are captured as `CodeRunResult.error` and surfaced so the model can self-correct; the `.d.ts` is explicitly advisory. - -Lost observability of sub-calls. Routing everything through one `run_code` result hides the individual calls from the model — and could hide them from operators too. Mitigation: the plugin-declared `code/dispatch` event keeps every sub-call in the session log and UI without polluting model context. - -Abort granularity. node:vm cannot reliably interrupt hot synchronous code, and `ctx.tools.execute()` converts thrown aborts into `isError` data. Mitigation: the SDK bindings check `signal.aborted` and throw before/after each dispatch so an aborted sub-call stops the program; the vm stub wraps the run in a signal-tied timeout; the hardened substrate addresses the hot-loop case. - -Unsafe example wiring. A demo running a real model through the node:vm stub would hand model output ambient authority. Mitigation: examples are mock-model or explicitly marked unsafe; `code-runtime-vm` is labeled reference/test-only. - -Non-text sub-results dropped in the MVP. Image and other block types from sub-calls are not surfaced into the program yet. Mitigation: noted as a known limitation; block-type handling deferred. diff --git a/docs/rfc/proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md b/docs/rfc/proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md new file mode 100644 index 0000000000..3de3c7d9be --- /dev/null +++ b/docs/rfc/proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md @@ -0,0 +1,87 @@ +# RFC: Claude Code and Codex subagent backends (out-of-process delegation to external coding agents) + +Status: proposed + +## Problem + +The subagent seam ([the seam RFC](../../implemented/feature/2026-06-21-subagent-capability-seam.md)) hosts multiple named providers on `ctx.subagents`, and the ACP backend ([the ACP backend RFC](../../implemented/feature/2026-06-22-acp-subagent-backend.md)) proved the seam generalizes across a process boundary; its Future-providers section explicitly named the Codex app-server and the Claude Code Agent SDK as mechanically similar siblings. Those two are the engines actually worth delegating to today: a harness turn should be able to hand a self-contained task to a real Claude Code or a real Codex — a separate product with its own model, tools, and sandbox — and get back one final answer, without the parent deployment leaking its secrets into the child or the child's behavior silently depending on whatever `~/.claude` / `~/.codex` state exists on the host machine. + +## Proposal + +Two sibling provider packages, structural variants of the ACP backend, plus one extraction: + +- `@deepseek-ai/dsh-subagent-claude-code` — drives a Claude Code child through `@anthropic-ai/claude-agent-sdk`'s `query()` (the SDK runs in the parent process and spawns its bundled `claude` CLI as the subprocess). Provider name `claude-code`: the child is the Claude Code *product*, not an Anthropic model adapter — "claude" stays reserved for a future `dsh-llm` adapter. +- `@deepseek-ai/dsh-subagent-codex` — spawns `codex app-server` and drives one thread/turn over its JSON-RPC-over-stdio protocol with a hand-rolled newline-JSON client (~200–300 lines) in the package. +- `@deepseek-ai/dsh-subagent-process` — a pure library (the `subagent-inprocess` precedent) extracting what `dsh-subagent-acp` already carries and both new backends need: the credential env scrub (`SENSITIVE_ENV_PATTERN`/`buildChildEnv`), the EOF → SIGTERM → SIGKILL dispose ladder, and new isolated-config-dir helpers (`mkdtemp` create, best-effort remove). The ACP backend migrates onto it; `bash-local`'s sibling copy is left alone to bound the change. + +Both providers copy the ACP backend's seam posture verbatim: fresh child per `start`, exactly one prompt round-trip, capabilities all `false`, `inheritsParentContext: false`, `request.parent`/`request.agentOptions` ignored, `id = AgentId(randomUUID())`, `result` never rejects — child-level failure flattens to a stop reason and the original error goes to `ctx.logger` via an `onError` spec callback. Model exposure is zero new code: `dsh-tool-subagent` is loaded once per provider with a distinct `toolName` (`subagent_claude_code`, `subagent_codex`). No new session events are needed — the only model-visible artifact is the tool result, so reconstructability holds exactly as it did for ACP. To be explicit about the boundary: the session log reconstructs the model-visible transcript, not workspace mutation history — a child granted write access mutates files as an ambient side effect outside the log, exactly as the bash tools and the ACP backend already do; replay reproduces requests, not the disk. + +## Verified interface facts (pinned versions) + +Both integration surfaces were verified against pinned implementations before this proposal — types and bundled source read, keyless spikes run — not from vendor docs alone. The pins are the verification baseline, not a runtime contract: the backends perform no runtime version probe (no `codex --version` gate, no SDK version sniffing). Compatibility is enforced at development time — every dependency bump re-runs the keyless suites against the real load path — and at runtime by failing loudly: a protocol-level surprise settles `error` via `onError`, never a silent misbehavior. + +**`@anthropic-ai/claude-agent-sdk` 0.3.202.** `options.env` REPLACES the child environment (no merge with `process.env`), which is exactly what the scrub needs. `settingSources` defaults to loading ALL filesystem settings — isolation requires explicitly passing `[]`. Result subtypes are `success` | `error_during_execution` | `error_max_turns` | `error_max_budget_usd` | `error_max_structured_output_retries`. On abort the SDK escalates the CLI child itself: stdin EOF immediately, SIGTERM ~2s later if the child ignores it (observed; no leftover processes) — no bespoke kill fallback needed. `outputFormat: {type: 'json_schema'}` and an `agents` option exist, giving future landing points for the seam's `outputSchema` capability and named subagent types; both are out of scope here. + +**codex CLI 0.142.5, `codex app-server` (v2 vocabulary).** LF-delimited JSON, JSON-RPC 2.0 shapes with the `"jsonrpc"` header omitted. + +- Lifecycle: `initialize{clientInfo}` + `initialized` → `thread/start` (accepts `cwd`, `model`, `sandbox`, `approvalPolicy`, `ephemeral`; succeeds unauthenticated) → `turn/start{threadId, input:[{type:'text',text}]}` returns an `inProgress` turn immediately; the terminal signal is the `turn/completed` notification carrying `Turn{status: completed|interrupted|failed|inProgress, error}`. +- Approvals are server-initiated requests — `item/commandExecution/requestApproval`, `item/fileChange/requestApproval`, `item/permissions/requestApproval`, `item/tool/requestUserInput`, `mcpServer/elicitation/request` — answered with `accept`/`decline`-family decisions. +- Auth: `account/login/start{type:'apiKey', apiKey}` is a first-class RPC and `account/read` reports `requiresOpenaiAuth` — and an unauthenticated `turn/start` does NOT fail fast (it hangs in retry), so the backend MUST pre-check auth and settle `error` loudly instead of waiting on the turn. +- Isolation: `CODEX_HOME` redirection is honored (the `initialize` response echoes it, so tests can assert isolation), and `ephemeral: true` threads leave no session files at all. + +## Isolation and credentials + +Deployments authenticate with API keys only, and the child must not see the host user's Claude Code / Codex configuration: behavior has to be a function of `cordis.yml` alone. Each run gets a fresh `mkdtemp` config dir — `CLAUDE_CONFIG_DIR` for Claude Code (paired with an explicit `settingSources: []`), `CODEX_HOME` for Codex — removed best-effort on dispose; a config field can pin a persistent dir instead. The child env reuses the ACP backend's `buildChildEnv` semantics verbatim via the extraction: the ambient env is forwarded MINUS credential-shaped vars (`/KEY|SECRET|TOKEN/i`), with `config.env` layered on top — so `PATH`, `HOME`, `TMPDIR`, locale, and proxy vars survive and the CLIs run normally, while only credential-shaped ambient vars are scrubbed (`ANTHROPIC_API_KEY` enters explicitly through `config.env` for Claude Code), and the Codex key travels via the `account/login/start` RPC into the isolated `CODEX_HOME` rather than a hand-written `auth.json`. + +## Permission and approval policy + +Instead of collapsing to ACP's single `permission: allow|reject` knob, each backend exposes its engine's native vocabulary as config, with conservative defaults: Claude Code gets `permissionMode` (default `default`) plus `permission: allow|reject` (default `reject`) as the `canUseTool` auto-answer for whatever falls through; Codex gets `sandboxMode` (default `read-only`) and `approvalPolicy` (default `never`) plus the same `permission` fallback for approval requests that still arrive. Defaults are deliberately do-no-harm (the out-of-box child cannot write files); examples demonstrate opening up (`acceptEdits` / `workspace-write`). The mechanical rule: EVERY server-initiated request is settled programmatically and promptly — the enumerated approval/user-input/elicitation requests by the configured policy, an unknown request method with a JSON-RPC method-not-found error response (never left pending), unknown notifications consumed — so no child request can wedge a turn waiting on an answer that will never come. Prompts never reach a human in this cut, matching ACP. + +## StopReason mapping + +Claude Code: `success` → `completed`; `error_max_turns`, `error_during_execution`, `error_max_budget_usd`, `error_max_structured_output_retries` → `error` (aligning with the ACP call on `max_turn_requests`: an unfinished task is not success); generator abort → `aborted`; anything unknown → `error`. Codex: `Turn.status` `completed` → `completed`; `interrupted` → `aborted`; `failed` with `codexErrorInfo: 'contextWindowExceeded'` → `max-tokens`, any other `failed` → `error`; transport/spawn/auth-precheck failure → `error` (or `aborted` if cancel was requested). In both, `cancel()` is the ACP shape: flag + abort/interrupt + a cancel-settled race arm so an uncooperative child cannot stall the result. + +Liveness posture, stated explicitly: teardown timing is config, turn duration is not. Both backends take the dispose ladder's grace periods as defaulted validated config fields (the ACP backend's `disposeEofGraceMs`/`disposeGraceMs` shape, carried by the extraction), but there is deliberately NO turn-duration or startup timeout — matching ACP, liveness during a turn belongs to the caller via `cancel()`/the abort signal, a subagent turn is legitimately minutes long, and the Codex auth precheck removes the one verified guaranteed-hang; a deployment wanting a wall-clock bound cancels from the parent. + +## Testing + +Named at every tier per the root AGENTS.md rule, and de-risked up front: + +- **Keyless unit/integration**, mirroring the ACP spec list per backend (round-trip and output accumulation, every stop mapping, both cancel paths, already-aborted, permission auto-answer under both policies, unknown-message tolerance, bad-command spawn failure, HMR provider cleanup, export shape, isolation assertions on child env and temp-dir removal; Codex adds the auth-precheck failure path). Claude Code's harness is a scripted fake `claude` executable behind `pathToClaudeCodeExecutable` driven by the REAL SDK — a spike already passed end-to-end keyless in 24ms (the fake CLI answers one `control_request/initialize` and speaks plain stream-json, ~40 lines). Codex's harness is a scripted mock app-server subprocess speaking the verified wire protocol, the `mock-acp-server.ts` shape. +- **With-key e2e** per backend: the real engine does real file work verified on disk, under a pinned opened-up config so acceptance and the do-no-harm defaults don't collide — `permissionMode: 'acceptEdits'` for Claude Code, `sandboxMode: 'workspace-write'` + `approvalPolicy: 'never'` for Codex; self-skips report exactly what is missing (binary vs key). CI has no secrets, so these run locally per the with-key policy. +- **Snapshot**: deferred as `TODO(claude-code-subagent-replay)` / `TODO(codex-subagent-replay)` — the same distinct replay shape the ACP backend deferred ([the per-session replay RFC](../../implemented/testing/2026-06-22-subagent-snapshot-replay.md)); the keyless suites carry deterministic coverage meanwhile. + +## Alternatives considered + +### Why not the official `@openai/codex-sdk` instead of a hand-rolled client? + +The dispose ladder and env scrub require owning the child process (spawn args, env, signals, exit await); the SDK hides the process. The wire format is trivial to frame (LF JSON), the shapes are generatable per pinned version (`codex app-server generate-json-schema`), and the repo precedent (`hook-protocol`) is to own thin protocol cores rather than wrap someone's runtime. The SDK would save protocol-evolution maintenance but costs the exact control this backend exists to have. + +### Why not a model-visible `subagent_type` parameter (one Task-style tool)? + +Claude Code's own Task tool puts the subagent type in the model-facing schema, selecting a prompt-plus-toolset persona. Here the choice is between EXECUTION ENGINES, and only the deployer knows which engines have credentials configured — so selection stays deployment config, preserving `dsh-tool-subagent`'s documented one-provider-per-tool contract. A persona-style type selector would be a separate RFC against the tool, not the backends. + +### Why not login-state credentials and the user's own config? + +Inheriting `~/.claude` / `~/.codex` (subscription login, user settings, skills, MCP servers) would make child behavior depend on host-machine state and punch an implicit exception through the "credentials enter explicitly via `config.env`, never ambiently" rule the ACP backend and bash executor established. API-key-only plus forced config-dir isolation keeps runs reproducible; deployments wanting shared state can point the config-dir field at a persistent directory deliberately. + +### Why not a driver-injection seam for the Claude Code keyless tests? + +Injecting a fake `query()` would mock our own boundary and leave the real SDK load path untested (the real-over-mock policy in docs/testing.md). The risk that justified considering it — the SDK↔CLI stream-json control protocol being internal — was retired by the spike: the fake-CLI harness works against the real pinned SDK today. If an SDK upgrade breaks the mock, the keyless suite fails the upgrade PR, which is the gate working. + +### Why not ACP adapters (e.g. `claude-code-acp`) reusing the existing backend? + +Community shims wrap both engines in ACP, which would make them "just config" on `dsh-subagent-acp`. But that inserts an unofficial third-party layer between the harness and the engine, erases the native control surfaces this RFC exposes (permissionMode, sandboxMode/approvalPolicy, config-dir isolation, apiKey RPC), and trades first-party protocol stability for a shim's release cadence. First-party surfaces — the Agent SDK and the app-server — are the supported integration points. + +## Acceptance criteria + +On a machine with both engines and keys configured: a REPL-driven model completes one real file task through `subagent_claude_code` and one through `subagent_codex`, the tool result being the child's final answer, with only `tool/call` + `tool/result` in the parent session log. Keyless suites pass at 100% per-file coverage in a credential-less environment, asserting isolation (scrubbed child env, no temp config dirs left after dispose) and that child behavior is unchanged by the presence or absence of `~/.claude` / `~/.codex`. Cancelling a parent turn quiesces both backends in bounded time with no leftover child processes. E2e suites self-skip cleanly, naming the missing prerequisite. + +## Risks + +- `codex app-server` is CLI-flagged experimental and its v1/v2 vocabularies coexist; the client pins 0.142.5, implements v2 only, and consumes unknown methods/notifications without crashing, but a future codex bump can still force rework (regenerate schemas and re-run the keyless suite on every bump — the development-time enforcement behind the no-runtime-version-probe stance above). +- The Claude Code fake-CLI mock rides an internal protocol: any SDK upgrade must go through the keyless suite, and a breaking control-protocol change means reworking the mock (fallback: the driver-injection seam rejected above becomes the escape hatch). +- The SDK's optionalDependencies weigh ~280MB per platform — accepted, and confined to the one backend package. +- The SDK's SIGKILL branch beyond EOF→SIGTERM was not observed and is trusted; e2e keeps a no-leftover-process assertion. +- Codex is a deployment prerequisite (no npm-bundled binary); a missing or incompatible binary surfaces as a loud spawn/protocol `error`, not a version probe. +- Every run pays a fresh child process and only the final answer surfaces — thoughts, tool cards, and usage are consumed and dropped; pooling, intermediate-progress surfacing, `sendMessage`/`resume`, `outputSchema` via the SDK's `outputFormat`, and named subagent types via the SDK's `agents` option are all deliberate deferrals. diff --git a/docs/testing.md b/docs/testing.md index d7ed14ecb9..a4a77c9d0c 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -25,9 +25,9 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword - A plugin shipped via `cordis.yml` needs at least one test through the REAL Loader path: hand-built `ctx.plugin({...})` mounts bypass `unwrapExports` and cannot catch a broken export shape ([postmortem 0001](postmortem/0001-acp-default-export-drops-inject.md); export-shape rules in [packages/AGENTS.md](../packages/AGENTS.md)). - A guard only guards if the regression actually fails it. For a plugin without `inject` (bundle/composition plugins), a Loader smoke stays green under a broken export shape — add an explicit `expect('default' in mod).toBe(false)` plus an `unwrapExports` round-trip assertion, and prove it: introduce the regression, watch red, revert. -- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). Keep the built-bin smokes green (`packages/ui/*/tests/built-bin.e2e.ts`), and assert a genuinely-missing config exits non-zero. +- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). The same applies to any non-index runtime entry the built package resolves at run time (the worker-thread runtime's sibling `lib/worker.js`). Keep the built-artifact smokes green (`packages/ui/*/tests/built-bin.e2e.ts`, `packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts`), and assert a genuinely-missing config exits non-zero. - An e2e that spawns an example from a temp cwd sets `TSX_TSCONFIG_PATH` to the repo-root tsconfig, or it silently falls back to stale built `lib/` ([examples/AGENTS.md](../examples/AGENTS.md)). ## When a snapshot test is required -Any change affecting the editor-facing transcript or end-to-end agent UX — the ACP bridge, the loop's observable output, tool presentation — adds or updates a scenario under `examples/acp-agent/tests/snapshots/` (or states in the PR why none applies). New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise. +Any change affecting the editor-facing transcript or end-to-end agent UX — the ACP bridge, the loop's observable output, tool presentation — adds or updates a scenario in the owning example's snapshot suite (`examples//tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory; `examples/acp-agent` is the primary suite), or states in the PR why none applies. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise. diff --git a/docs/tool-catalog.md b/docs/tool-catalog.md index a621d3299d..3645ff40e9 100644 --- a/docs/tool-catalog.md +++ b/docs/tool-catalog.md @@ -289,10 +289,6 @@ Fetch the content of a specific HTTP(S) URL and return it decoded to text. "url": { "type": "string", "description": "The HTTP(S) URL to fetch." - }, - "timeout_ms": { - "type": "number", - "description": "Optional fetch timeout in milliseconds (capped by the provider)." } }, "required": [ diff --git a/docs/tool-execution-pipeline.md b/docs/tool-execution-pipeline.md index db50a3beec..c28c934ab2 100644 --- a/docs/tool-execution-pipeline.md +++ b/docs/tool-execution-pipeline.md @@ -3,7 +3,7 @@ # Tool Execution Pipeline -This graph shows where policy, hooks, sandboxing, filesystem guards, result rewriting, and UI rendering fit without changing the loop. The key extension points are the `tools/pre-execute` and `tools/post-execute` waterfalls. +This graph shows where policy, hooks, sandboxing, filesystem guards, result rewriting, and UI rendering fit without changing the loop. The key extension points are the `tools/pre-execute`, `tools/execute`, and `tools/post-execute` waterfalls. ```mermaid flowchart TD @@ -12,6 +12,7 @@ flowchart TD presentCall["UI pending card
presentCall(args)"] pre["tools/pre-execute waterfall
hooks, permission, sandbox"] denied["deny or ask
tool body skipped"] + around["tools/execute waterfall
timeout, retry, metrics (around dispatch)"] toolBody["Registered tool execute() body"] fsGate["fs/write-intent or fs/edit-intent
tool-fs mutations only"] owned["Tool-owned session events
todo/write, fs/observed, hook/invoked, hook/result"] @@ -22,18 +23,20 @@ flowchart TD model --> toolCall toolCall --> presentCall toolCall --> pre - pre -->|allow| toolBody + pre -->|allow| around + around --> toolBody pre -->|deny or ask| denied denied --> post toolBody --> fsGate fsGate --> toolBody toolBody --> owned - toolBody --> post + toolBody --> around + around --> post post --> context post --> toolResult toolResult --> presentResult ``` -Filesystem read-before-edit checks live below `tool-fs` on the `fs/*` event gate, while hook bridges and future permission prompts live on the generic tool waterfalls. That split lets the same hooks observe bash, fs, web, todo, and subagent calls without coupling those tools to one policy service. +Filesystem read-before-edit checks live below `tool-fs` on the `fs/*` event gate; hook bridges and future permission prompts live on the generic pre/post tool waterfalls; and around-dispatch concerns like the tool-call timeout policy (`@deepseek-ai/dsh-timeout-policy`) wrap core dispatch on `tools/execute`. That split lets the same hooks observe bash, fs, web, todo, and subagent calls without coupling those tools to one policy service. Maintenance mode: curated Mermaid flow; exact tool schemas and event signatures live in generated catalogs. diff --git a/examples/acp-agent/README.md b/examples/acp-agent/README.md index 58a5984301..a1f6e818ab 100644 --- a/examples/acp-agent/README.md +++ b/examples/acp-agent/README.md @@ -6,7 +6,7 @@ The DeepSeek Harness SDK agent demo exposed as an **Agent Client Protocol (ACP)* pnpm run demo:acp # needs DEEPSEEK_API_KEY (repo-root .env or env) ``` -This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand), the swappable DeepSeek, bash, and filesystem backends, and the model-facing `read`/`write`/`edit`/`subagent`/`subagent_fork`/`todo_write` tool entries. The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC. +This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand), the swappable DeepSeek, bash, and filesystem backends, the model-facing `read`/`write`/`edit`/`subagent`/`subagent_fork`/`todo_write` tool entries, and the advisory `repeat-tool-guard` loop-hygiene plugin. The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC. ## stdout is the protocol diff --git a/examples/acp-agent/composition.md b/examples/acp-agent/composition.md index 2644a2b112..7fd5e25875 100644 --- a/examples/acp-agent/composition.md +++ b/examples/acp-agent/composition.md @@ -33,6 +33,8 @@ flowchart LR cfg --> plugin_acp_tool_subagent_fork plugin_acp_tool_todo["tool-todo
@deepseek-ai/dsh-tool-todo"] cfg --> plugin_acp_tool_todo + plugin_acp_repeat_tool_guard["repeat-tool-guard
@deepseek-ai/dsh-repeat-tool-guard"] + cfg --> plugin_acp_repeat_tool_guard plugin_acp_fs_local["fs-local
@deepseek-ai/dsh-fs-local"] cfg --> plugin_acp_fs_local plugin_acp_fs_policy["fs-policy
@deepseek-ai/dsh-fs-policy"] @@ -56,6 +58,7 @@ flowchart LR | `tool-subagent` | `@deepseek-ai/dsh-tool-subagent` | | `tool-subagent-fork` | `@deepseek-ai/dsh-tool-subagent` | | `tool-todo` | `@deepseek-ai/dsh-tool-todo` | +| `repeat-tool-guard` | `@deepseek-ai/dsh-repeat-tool-guard` | | `fs-local` | `@deepseek-ai/dsh-fs-local` | | `fs-policy` | `@deepseek-ai/dsh-fs-policy` | | `tool-fs` | `@deepseek-ai/dsh-tool-fs` | diff --git a/examples/acp-agent/cordis.yml b/examples/acp-agent/cordis.yml index 83ac07ce55..af2da25b86 100644 --- a/examples/acp-agent/cordis.yml +++ b/examples/acp-agent/cordis.yml @@ -86,6 +86,14 @@ - id: tool-todo name: '@deepseek-ai/dsh-tool-todo' +# The repeat-tool-call guard: advisory reminders (injected context, never a +# block) when the model re-issues the same tool call with identical arguments; +# defaults [3, 5, 8]. Loaded here so the snapshot tier exercises the reminder +# transcript (the repeat-tool-guard scenario) — no other scenario repeats a +# call three times, so it is inert everywhere else. +- id: repeat-tool-guard + name: '@deepseek-ai/dsh-repeat-tool-guard' + # Filesystem capability stack: local provider, read-before-write/edit policy # gate, then the model-facing read/write/edit tools. Relative filesystem paths # resolve from the server launch cwd; the documented Zed setup launches this diff --git a/examples/acp-agent/tests/acp.e2e.ts b/examples/acp-agent/tests/acp.e2e.ts index 3245fe1750..714791fa3e 100644 --- a/examples/acp-agent/tests/acp.e2e.ts +++ b/examples/acp-agent/tests/acp.e2e.ts @@ -57,8 +57,8 @@ interface Spawned { } // TODO(acp-test-harness): this subprocess/client boot glue is duplicated with -// hooks.e2e.ts and partly with snapshot-harness.ts. Extract one shared ACP test -// launcher before the TSX/env/permission-stub details drift again. +// hooks.e2e.ts and partly with dsh-acp-snapshot's harness. Migrate both e2e +// files onto that launcher before the TSX/env/permission-stub details drift. function spawnAcpAgent(cwd: string, env: NodeJS.ProcessEnv = process.env): Spawned { const child = spawn( process.execPath, diff --git a/examples/acp-agent/tests/acp.snapshot.ts b/examples/acp-agent/tests/acp.snapshot.ts index bde4f4dbb9..647a37e9df 100644 --- a/examples/acp-agent/tests/acp.snapshot.ts +++ b/examples/acp-agent/tests/acp.snapshot.ts @@ -1,86 +1,24 @@ -import { readFile, readdir, writeFile } from 'node:fs/promises' -import { existsSync } from 'node:fs' import { fileURLToPath } from 'node:url' import { dirname, join } from 'node:path' -import { describe, expect, it } from 'vitest' -import { type HarvestedLog, type InputScript, runScenario } from './snapshot-harness.ts' -import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from './snapshot-normalize.ts' +import { defineAcpSnapshotSuite, type Scenario } from '@deepseek-ai/dsh-acp-snapshot' /** - * ACP snapshot tests (REPLAY by default, keyless). Each scenario under - * `snapshots//` ships an `input.json` (the client stdin script) and a - * `session.jsonl` fixture; replay boots the real acp-agent subprocess, drives - * it, and diffs the normalized stdout transcript against the committed - * `stdout.golden.jsonl`. For model scenarios it ALSO checks the re-persisted - * session log — against the `session.jsonl` fixture itself, not a separate - * golden: the fixture doubles as the replay source (recorded scenarios) and the - * expected produced log (both sides normalized before comparing). - * - * Request-header content (the composed system prompt + tool schemas riding on - * `request/header` events) is pinned by exactly ONE scenario — the one with - * `pinsHeader` — and scrubbed to `{{system}}`/`{{tools}}` tokens in every - * other fixture and compare, so a prompt or tool-schema edit churns one - * committed line instead of every fixture. A per-run uniformity guard keeps - * the single pin sound: every live header must equal the pinned one, and no - * header-delta may appear outside the pinning scenario (see the - * pinned-header RFC, - * docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md). - * - * `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the - * `session.jsonl` fixtures against the real API and refreshes the stdout golden - * in one pass. + * The acp-agent example's snapshot suite: the scenario table for + * `dsh-acp-snapshot`'s suite factory, which owns every compare/guard mechanic + * (golden + re-persisted-log diffs, record write-back, the pinned-header + * uniformity guard, the fixture guards). Fixtures live under `snapshots//`; + * `pnpm run test:snapshot:record` re-records the `recorded` scenarios against + * the real API. See the package README (packages/support/acp-snapshot) and the + * snapshot RFC, docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md. */ -const SNAPSHOTS_DIR = join(dirname(fileURLToPath(import.meta.url)), 'snapshots') -const RECORDING = process.env.DSH_SNAPSHOT === 'record' - -/** A snapshot scenario and how its fixtures are produced. */ -interface Scenario { - name: string - /** Whether the scenario drives at least one model turn (so a JSONL golden applies). */ - hasModelTurn: boolean - /** - * Whether the run persists a comparable session log to diff against the - * `session.jsonl` fixture. Defaults to {@link hasModelTurn} (a model turn - * always produces a log worth comparing). Set it independently for a scenario - * that produces a non-trivial log WITHOUT a model turn — e.g. a prompt blocked - * by a `UserPromptSubmit` hook, which opens a `rejected` turn carrying `hook/*` - * events but never calls the model. - */ - comparesLog?: boolean - /** - * Whether `test:snapshot:record` regenerates this scenario's `session.jsonl` - * from the LIVE API. `recorded` scenarios are model-driven and reproducible; - * `authored` scenarios (a hand-written `replay.override.json` sidecar drives - * replay — e.g. a provider error or a cancel, which the live API can't be - * coaxed into deterministically — or a deterministic hook scenario whose - * derived empty script needs no sidecar) are NEVER re-recorded. - */ - recorded: boolean - /** - * How many SUBAGENT child sessions this scenario records beyond the top-level - * one (0 for a single-session scenario). Each child rides in a sibling fixture - * `session..jsonl` (1-based); replay forwards them to `dsh-llm-replay` so - * each child session replays from its own script, and record mode writes the - * harvested child logs back to those files. Defaults to 0. - */ - childSessions?: number - /** - * Whether THIS scenario's fixtures keep the full request-header content (the - * composed system prompt and tool schema list on `request/header` / - * `request/header-delta` events) and compare it verbatim. Exactly one - * scenario pins it; every other scenario stores and compares that content as - * `{{system}}`/`{{tools}}` tokens ({@link scrubRequestHeaders}), so a system - * prompt or tool-schema change shows up as ONE committed-fixture diff, not - * one per scenario. One pin suffices because header composition is - * suite-uniform (parent, spawn child, and fork child all compose the same - * prompt-modulo-cwd and the same tools) — and that premise is ASSERTED, not - * assumed: every non-pinning run's live headers must equal the pinned - * fixture's (normalized), so a session-dependent header (say, a restricted - * subagent toolset) fails loud until it gets its own pinning scenario. - * Defaults to false. - */ - pinsHeader?: boolean +// The dsh-acp-agent bin (the demo:acp entry), this example's cordis.yml, and +// the repo-root tsconfig (four levels up from examples/acp-agent/tests) — all +// ABSOLUTE: the subprocess cwd is a temp dir outside the repo. +const AGENT = { + binScript: fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)), + configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)), + tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)), } const SCENARIOS: Scenario[] = [ @@ -101,8 +39,13 @@ const SCENARIOS: Scenario[] = [ { name: 'fs-read-window', hasModelTurn: true, recorded: true }, { name: 'fs-policy-reject', hasModelTurn: true, recorded: true }, { name: 'multi-turn', hasModelTurn: true, recorded: true }, - { name: 'error-finish', hasModelTurn: true, recorded: false }, - { name: 'cancel', hasModelTurn: true, recorded: false }, + { name: 'error-finish', hasModelTurn: true, recorded: false, overridden: true }, + // Keyless, authored (like error-finish/cancel): deterministically forcing a + // LIVE model to repeat one call three times is not a stable recording, so + // the fixture scripts five identical todo_write calls and pins BOTH reminder + // tiers (gentle at 3, detailed at 5) as context/message in transcript and log. + { name: 'repeat-tool-guard', hasModelTurn: true, recorded: false }, + { name: 'cancel', hasModelTurn: true, recorded: false, overridden: true }, { name: 'subagent-spawn', hasModelTurn: true, recorded: true, childSessions: 1 }, { name: 'subagent-multi', hasModelTurn: true, recorded: true, childSessions: 2 }, { name: 'subagent-fork', hasModelTurn: true, recorded: true, childSessions: 1 }, @@ -148,238 +91,9 @@ const SCENARIOS: Scenario[] = [ { name: 'hook-codex-stop-continue', hasModelTurn: true, recorded: true }, ] -/** The single header-pinning scenario. Guarded here (and by a meta-test) so the pin cannot silently vanish. */ -const pinningScenario = SCENARIOS.find(s => s.pinsHeader === true) -if (pinningScenario === undefined) throw new Error('acp.snapshot: no scenario pins the request-header content') - -/** The sibling child-fixture paths for a scenario (`session.1.jsonl` …). */ -function childFixturePaths(dir: string, childSessions: number): string[] { - return Array.from({ length: childSessions }, (_, i) => join(dir, `session.${i + 1}.jsonl`)) -} - -/** - * Derive the {@link NormalizeContext} for a `session.jsonl` fixture from its own - * header line (`{ type: 'session', id, cwd }`). A committed fixture carries the - * session id and cwd of the run that harvested it — different from the live - * replay run — so normalizing it against the live run's ctx would leave those - * recorded values unscrubbed. Reading them from the header scrubs the fixture's - * own id/cwd to the same `{{sessionId}}`/`{{cwd}}` tokens the replay output gets. - * An authored fixture whose header is already normalized (`id:'{{sessionId}}'`, - * `cwd:'{{cwd}}'`) yields those tokens as the volatile values, so scrubbing them - * is an idempotent no-op. A header with no `cwd` falls back to a sentinel that - * cannot occur in a log (NOT `''`, which `String.split` would match on every - * character boundary and corrupt the output). - */ -function fixtureContext(fixture: string): NormalizeContext { - const firstLine = fixture.split('\n').find(line => line.trim().length > 0) ?? '{}' - const header = JSON.parse(firstLine) as { id?: unknown; cwd?: unknown } - return { - sessionIds: typeof header.id === 'string' ? [header.id] : [], - cwd: typeof header.cwd === 'string' ? header.cwd : '\0no-cwd\0', - } -} - -/** - * The `data.header` payload of every `request/header` event in a session - * JSONL, in log order, with the log's volatile values scrubbed first - * ({@link normalizeSessionLog}) so headers harvested from different runs — - * each embedding its own temp cwd in the composed prompt — compare on equal - * footing. - */ -function normalizedHeaders(rawLog: string, ctx: NormalizeContext): unknown[] { - return normalizeSessionLog(rawLog, ctx) - .split('\n') - .filter(line => line.trim().length > 0) - .map(line => JSON.parse(line) as { type?: unknown; data?: { header?: unknown } }) - .filter(record => record.type === 'request/header') - .map(record => record.data?.header) -} - -/** Count the `request/header-delta` events in a session JSONL. */ -function headerDeltaCount(rawLog: string): number { - return rawLog.split('\n') - .filter(line => line.trim().length > 0) - .filter(line => (JSON.parse(line) as { type?: unknown }).type === 'request/header-delta') - .length -} - -for (const scenario of SCENARIOS) { - describe(`snapshot: ${scenario.name}`, () => { - // In RECORD mode, only re-run the `recorded` (live-API) scenarios; the - // `authored` ones (sidecar-driven errors/cancel) are never re-recorded. - it.skipIf(RECORDING && !scenario.recorded)('matches the goldens', async () => { - const dir = join(SNAPSHOTS_DIR, scenario.name) - const input = JSON.parse(await readFile(join(dir, 'input.json'), 'utf8')) as InputScript - const overrideFile = join(dir, 'replay.override.json') - const workspaceDir = join(dir, 'workspace') - const childSessions = scenario.childSessions ?? 0 - const result = await runScenario(input, { - mode: RECORDING ? 'record' : 'replay', - fixtureFile: join(dir, 'session.jsonl'), - ...existsSync(overrideFile) ? { overrideFile } : {}, - // In REPLAY, forward the recorded child fixtures so each subagent session - // replays from its own script. In RECORD they are harvested, not read. - ...!RECORDING && childSessions > 0 ? { childFiles: childFixturePaths(dir, childSessions) } : {}, - ...existsSync(workspaceDir) ? { workspaceDir } : {}, - }) - - // Scrub every volatile id the run produced: the ACP server-issued session - // id plus every harvested log's recorded id (a subagent child id never - // surfaces over ACP, but it appears in the child's own log header). The - // normalizer's UUID catch-all covers any we don't enumerate. - const ctx: NormalizeContext = { - sessionIds: [ - ...result.sessionId !== undefined ? [result.sessionId] : [], - ...result.sessionLogs.map(l => l.id), - ], - cwd: result.cwd, - } - - // RECORD mode (recorded model scenarios only): persist the freshly-harvested - // logs back to their fixtures — the primary to session.jsonl, each child to - // session..jsonl in harvest order. `--update` refreshes the Vitest - // goldens but NOT these fixtures, so write them here. A non-pinning - // scenario's fixtures are written header-scrubbed, so a re-record can - // never smuggle the full prompt/schema content back into every fixture. - const scrub = scenario.pinsHeader === true - ? (log: string): string => log - : scrubRequestHeaders - if (RECORDING && scenario.recorded && scenario.hasModelTurn) { - expect(result.sessionLogs.length, 'record produced no session log to harvest').toBeGreaterThan(0) - expect(result.sessionLogs.length, `expected ${childSessions + 1} session logs (parent + children)`) - .toBe(childSessions + 1) - await writeFile(join(dir, 'session.jsonl'), scrub((result.sessionLogs[0] as HarvestedLog).content)) - for (let i = 1; i < result.sessionLogs.length; i++) { - await writeFile(join(dir, `session.${i}.jsonl`), scrub((result.sessionLogs[i] as HarvestedLog).content)) - } - } - - await expect(normalizeStdout(result.rawStdout, ctx)) - .toMatchFileSnapshot(join(dir, 'stdout.golden.jsonl')) - - // A model turn always produces a log worth comparing; a hook scenario can - // produce one without a model turn (a `rejected` turn carrying `hook/*`). - const comparesLog = scenario.comparesLog ?? scenario.hasModelTurn - if (comparesLog) { - // The harvested logs (primary-first) must match their committed fixtures - // 1:1. Each side passes through normalizeSessionLog, scrubbed against ITS - // OWN volatile values — the live run's via `ctx`, the committed fixture's - // via its own header (a committed file cannot share the live run's ids). - // Unless this scenario pins the header, both sides ALSO pass through - // scrubRequestHeaders: the live log carries the real prompt/schemas, the - // fixture carries the `{{system}}`/`{{tools}}` tokens, and the scrub is - // idempotent — so the compare checks the header's presence, position, - // reason, and config, but not its bulk content (pinned once, in the - // `pinsHeader` scenario). - expect(result.sessionLogs.length, 'this scenario must persist a session log').toBe(childSessions + 1) - const fixtureFiles = ['session.jsonl', ...Array.from({ length: childSessions }, (_, i) => `session.${i + 1}.jsonl`)] - for (let i = 0; i < fixtureFiles.length; i++) { - const harvested = scrub((result.sessionLogs[i] as HarvestedLog).content) - const fixture = scrub(await readFile(join(dir, fixtureFiles[i] as string), 'utf8')) - expect(normalizeSessionLog(harvested, ctx), `${fixtureFiles[i]} mismatch`) - .toEqual(normalizeSessionLog(fixture, fixtureContext(fixture))) - } - } - - // Header-uniformity guard: the single pin is sound only while every - // session in the suite composes the SAME header and keeps it for the - // whole run. Assert both halves live. (1) Every request/header the run - // produced (parent, spawn child, fork child, initial or resume) must - // equal the pinned fixture's header after each side is normalized - // against its own volatile values. (2) No request/header-delta may - // appear at all — a mid-run header change diverges from the pin by - // construction, and its content would be invisible under the scrub. If - // either fails, either the header changed (update the pin: re-record or - // hand-edit the pinning scenario's fixture) or composition became - // session-dependent by design (give the divergent shape its own - // pinning scenario). - if (scenario.pinsHeader !== true) { - const pinnedFixture = await readFile(join(SNAPSHOTS_DIR, pinningScenario.name, 'session.jsonl'), 'utf8') - const pinned = normalizedHeaders(pinnedFixture, fixtureContext(pinnedFixture)) - expect(pinned.length, `the pinning fixture (${pinningScenario.name}) must carry exactly one request/header`) - .toBe(1) - for (const log of result.sessionLogs) { - expect(headerDeltaCount(log.content), `session ${log.id}: a request/header-delta in a non-pinning scenario`) - .toBe(0) - const headers = normalizedHeaders(log.content, ctx) - for (const [k, header] of headers.entries()) { - expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`) - .toEqual(pinned[0]) - } - } - } - }) - }) -} - -describe('snapshot fixtures', () => { - it('every scenario directory is registered (no orphans)', async () => { - // toMatchFileSnapshot does not prune orphaned golden/fixture files, so a - // renamed/removed scenario could leave a stale dir that nothing exercises. - // Fail loud on any snapshots/ not present in SCENARIOS. - const entries = await readdir(SNAPSHOTS_DIR, { withFileTypes: true }) - const onDisk = entries.filter(e => e.isDirectory()).map(e => e.name).sort() - const registered = SCENARIOS.map(s => s.name).sort() - expect(onDisk).toEqual(registered) - }) - - it('every registered scenario has its required fixture files', async () => { - // Every scenario has an input script and an stdout golden. EVERY scenario - // also needs `session.jsonl`: the harness boots `llm-replay` with that path - // as the replay source for ALL scenarios (acp.snapshot.ts passes - // `fixtureFile: /session.jsonl` unconditionally), and `loadReplayScript` - // throws "fixture not found" when it is absent and no override replaces it. - // A no-model scenario ships a header-only `session.jsonl` (it derives to an - // empty script — no model call is made); a model scenario's fixture also - // doubles as the expected-log artifact the run is diffed against. An authored - // (non-`recorded`) model scenario additionally ships a `replay.override.json` - // sidecar for the throw/hang cases a derived script cannot express. - for (const { name, hasModelTurn, recorded, childSessions } of SCENARIOS) { - const dir = join(SNAPSHOTS_DIR, name) - expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true) - expect(existsSync(join(dir, 'stdout.golden.jsonl')), `${name}/stdout.golden.jsonl`).toBe(true) - expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true) - if (hasModelTurn && !recorded) { - expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json`).toBe(true) - } - // A nested-agent scenario ships one child fixture per recorded subagent - // session (`session.1.jsonl` …), the replay source for that child session. - for (const childFixture of childFixturePaths(dir, childSessions ?? 0)) { - expect(existsSync(childFixture), childFixture).toBe(true) - } - } - }) - - it('exactly one scenario pins the request-header content', () => { - // Zero pins would drop the prompt/schema surface from the suite entirely; - // two would split it. The single pin is the design (pinned-header RFC). - expect(SCENARIOS.filter(s => s.pinsHeader === true).map(s => s.name)).toEqual(['text-turn']) - }) - - it('committed fixtures carry request-header content ONLY in the pinning scenario', async () => { - // The whole point of the pin: a system-prompt or tool-schema change must - // churn exactly one committed line. A non-pinning fixture that carries the - // full header (a hand-recorded file, or a header line hand-edited out of - // its canonical JSON form) silently reopens the suite-wide churn, so fail - // loud here: every non-pinning session*.jsonl must be a fixed point of - // scrubRequestHeaders (apply the scrub to fix a violation), and the - // pinning scenario's fixtures must NOT be (their content IS the pin). - for (const scenario of SCENARIOS) { - const dir = join(SNAPSHOTS_DIR, scenario.name) - const files = [ - 'session.jsonl', - ...Array.from({ length: scenario.childSessions ?? 0 }, (_, i) => `session.${i + 1}.jsonl`), - ] - for (const file of files) { - const fixture = await readFile(join(dir, file), 'utf8') - if (scenario.pinsHeader === true) { - expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} must PIN the full header content`) - .not.toEqual(fixture) - } else { - expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} carries unscrubbed header content`) - .toEqual(fixture) - } - } - } - }) +defineAcpSnapshotSuite({ + agent: AGENT, + snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'), + scenarios: SCENARIOS, + mode: process.env.DSH_SNAPSHOT === 'record' ? 'record' : 'replay', }) diff --git a/examples/acp-agent/tests/snapshots/fs-policy-reject/session.jsonl b/examples/acp-agent/tests/snapshots/fs-policy-reject/session.jsonl index 566f26e3ed..a55328ea33 100644 --- a/examples/acp-agent/tests/snapshots/fs-policy-reject/session.jsonl +++ b/examples/acp-agent/tests/snapshots/fs-policy-reject/session.jsonl @@ -180,7 +180,7 @@ {"type":"assistant/chunk","seq":178,"time":1783279386389,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" use"}}} {"type":"assistant/chunk","seq":179,"time":1783279386416,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" bash"}}} {"type":"assistant/chunk","seq":180,"time":1783279386416,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} -{"type":"assistant/chunk","seq":181,"time":1783279386416,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" sed"}}} +{"type":"assistant/chunk","seq":181,"time":1783279386416,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" awk"}}} {"type":"assistant/chunk","seq":182,"time":1783279386416,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} {"type":"assistant/chunk","seq":183,"time":1783279386444,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" replace"}}} {"type":"assistant/chunk","seq":184,"time":1783279386444,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} @@ -197,47 +197,47 @@ {"type":"assistant/chunk","seq":195,"time":1783279386528,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} {"type":"assistant/chunk","seq":196,"time":1783279386614,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":1,"blockType":"tool-call"}}} {"type":"assistant/chunk","seq":197,"time":1783279386615,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":""}}} -{"type":"assistant/chunk","seq":198,"time":1783279386615,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"{"}}} -{"type":"assistant/chunk","seq":199,"time":1783279386615,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":200,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"command"}}} -{"type":"assistant/chunk","seq":201,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":202,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":": "}}} -{"type":"assistant/chunk","seq":203,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":204,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"sed"}}} -{"type":"assistant/chunk","seq":205,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" -"}}} -{"type":"assistant/chunk","seq":206,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"i"}}} -{"type":"assistant/chunk","seq":207,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" '"}}} -{"type":"assistant/chunk","seq":208,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"s"}}} -{"type":"assistant/chunk","seq":209,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"/"}}} -{"type":"assistant/chunk","seq":210,"time":1783279386699,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"blue"}}} -{"type":"assistant/chunk","seq":211,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"/g"}}} -{"type":"assistant/chunk","seq":212,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"reen"}}} -{"type":"assistant/chunk","seq":213,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"/g"}}} -{"type":"assistant/chunk","seq":214,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"'"}}} -{"type":"assistant/chunk","seq":215,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" settings"}}} -{"type":"assistant/chunk","seq":216,"time":1783279386726,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":".txt"}}} -{"type":"assistant/chunk","seq":217,"time":1783279386727,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":218,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":", "}}} -{"type":"assistant/chunk","seq":219,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":220,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"description"}}} -{"type":"assistant/chunk","seq":221,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":222,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":": "}}} -{"type":"assistant/chunk","seq":223,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":224,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"Replace"}}} -{"type":"assistant/chunk","seq":225,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" blue"}}} -{"type":"assistant/chunk","seq":226,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" with"}}} -{"type":"assistant/chunk","seq":227,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" green"}}} -{"type":"assistant/chunk","seq":228,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" in"}}} -{"type":"assistant/chunk","seq":229,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" settings"}}} -{"type":"assistant/chunk","seq":230,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":".txt"}}} -{"type":"assistant/chunk","seq":231,"time":1783279386837,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\""}}} -{"type":"assistant/chunk","seq":232,"time":1783279386838,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"}"}}} -{"type":"assistant/chunk","seq":233,"time":1783279386896,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The edit tool requires reading the file first due to the default fs-policy. However, the user explicitly instructed me NOT to use the read tool. I need to follow the user's instructions. The edit tool returned an error saying it requires reading the file first. I cannot comply with the user's request to not read the file because the tool enforces it. Let me try a different approach - maybe I can use subagent to do this, or use bash to do the replacement. Let me use bash with sed to replace \"blue\" with \"green\" without reading the file."}}}} -{"type":"assistant/chunk","seq":234,"time":1783279386896,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","arguments":"{\"command\": \"sed -i 's/blue/green/g' settings.txt\", \"description\": \"Replace blue with green in settings.txt\"}"}}}} +{"type":"assistant/chunk","seq":198,"time":1783279386615,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"{\"com"}}} +{"type":"assistant/chunk","seq":199,"time":1783279386615,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"mand\""}}} +{"type":"assistant/chunk","seq":200,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":":\"awk"}}} +{"type":"assistant/chunk","seq":201,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" '{gs"}}} +{"type":"assistant/chunk","seq":202,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"ub(/b"}}} +{"type":"assistant/chunk","seq":203,"time":1783279386643,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"lue/,"}}} +{"type":"assistant/chunk","seq":204,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\\\"gre"}}} +{"type":"assistant/chunk","seq":205,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"en\\\")"}}} +{"type":"assistant/chunk","seq":206,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"}1' s"}}} +{"type":"assistant/chunk","seq":207,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"ettin"}}} +{"type":"assistant/chunk","seq":208,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"gs.tx"}}} +{"type":"assistant/chunk","seq":209,"time":1783279386671,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"t > s"}}} +{"type":"assistant/chunk","seq":210,"time":1783279386699,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"ettin"}}} +{"type":"assistant/chunk","seq":211,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"gs.tm"}}} +{"type":"assistant/chunk","seq":212,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"p && "}}} +{"type":"assistant/chunk","seq":213,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"mv se"}}} +{"type":"assistant/chunk","seq":214,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"tting"}}} +{"type":"assistant/chunk","seq":215,"time":1783279386700,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"s.tmp"}}} +{"type":"assistant/chunk","seq":216,"time":1783279386726,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" sett"}}} +{"type":"assistant/chunk","seq":217,"time":1783279386727,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"ings."}}} +{"type":"assistant/chunk","seq":218,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"txt\","}}} +{"type":"assistant/chunk","seq":219,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\"des"}}} +{"type":"assistant/chunk","seq":220,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"crip"}}} +{"type":"assistant/chunk","seq":221,"time":1783279386754,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"tion"}}} +{"type":"assistant/chunk","seq":222,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"\":\"R"}}} +{"type":"assistant/chunk","seq":223,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"epla"}}} +{"type":"assistant/chunk","seq":224,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"ce b"}}} +{"type":"assistant/chunk","seq":225,"time":1783279386781,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"lue "}}} +{"type":"assistant/chunk","seq":226,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"with"}}} +{"type":"assistant/chunk","seq":227,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":" gre"}}} +{"type":"assistant/chunk","seq":228,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"en i"}}} +{"type":"assistant/chunk","seq":229,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"n se"}}} +{"type":"assistant/chunk","seq":230,"time":1783279386809,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"ttin"}}} +{"type":"assistant/chunk","seq":231,"time":1783279386837,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"gs.t"}}} +{"type":"assistant/chunk","seq":232,"time":1783279386838,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","argumentsDelta":"xt\"}"}}} +{"type":"assistant/chunk","seq":233,"time":1783279386896,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The edit tool requires reading the file first due to the default fs-policy. However, the user explicitly instructed me NOT to use the read tool. I need to follow the user's instructions. The edit tool returned an error saying it requires reading the file first. I cannot comply with the user's request to not read the file because the tool enforces it. Let me try a different approach - maybe I can use subagent to do this, or use bash to do the replacement. Let me use bash with awk to replace \"blue\" with \"green\" without reading the file."}}}} +{"type":"assistant/chunk","seq":234,"time":1783279386896,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","arguments":"{\"command\":\"awk '{gsub(/blue/,\\\"green\\\")}1' settings.txt > settings.tmp && mv settings.tmp settings.txt\",\"description\":\"Replace blue with green in settings.txt\"}"}}}} {"type":"assistant/chunk","seq":235,"time":1783279386896,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":227,"outputTokens":195,"cacheReadTokens":2048,"reasoningTokens":118}}}} {"type":"assistant/chunk","seq":236,"time":1783279386896,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":237,"time":1783279386897,"data":{"turn":1,"step":2,"content":[{"type":"reasoning","text":"The edit tool requires reading the file first due to the default fs-policy. However, the user explicitly instructed me NOT to use the read tool. I need to follow the user's instructions. The edit tool returned an error saying it requires reading the file first. I cannot comply with the user's request to not read the file because the tool enforces it. Let me try a different approach - maybe I can use subagent to do this, or use bash to do the replacement. Let me use bash with sed to replace \"blue\" with \"green\" without reading the file."},{"type":"tool-call","id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","arguments":"{\"command\": \"sed -i 's/blue/green/g' settings.txt\", \"description\": \"Replace blue with green in settings.txt\"}"}],"usage":{"inputTokens":227,"outputTokens":195,"cacheReadTokens":2048,"reasoningTokens":118}},"sourceEventSeqs":[77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236],"surfaceOp":"append"} -{"type":"tool/call","seq":238,"time":1783279386897,"data":{"turn":1,"step":2,"callId":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","arguments":"{\"command\": \"sed -i 's/blue/green/g' settings.txt\", \"description\": \"Replace blue with green in settings.txt\"}"}} +{"type":"assistant/message","seq":237,"time":1783279386897,"data":{"turn":1,"step":2,"content":[{"type":"reasoning","text":"The edit tool requires reading the file first due to the default fs-policy. However, the user explicitly instructed me NOT to use the read tool. I need to follow the user's instructions. The edit tool returned an error saying it requires reading the file first. I cannot comply with the user's request to not read the file because the tool enforces it. Let me try a different approach - maybe I can use subagent to do this, or use bash to do the replacement. Let me use bash with awk to replace \"blue\" with \"green\" without reading the file."},{"type":"tool-call","id":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","arguments":"{\"command\":\"awk '{gsub(/blue/,\\\"green\\\")}1' settings.txt > settings.tmp && mv settings.tmp settings.txt\",\"description\":\"Replace blue with green in settings.txt\"}"}],"usage":{"inputTokens":227,"outputTokens":195,"cacheReadTokens":2048,"reasoningTokens":118}},"sourceEventSeqs":[77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236],"surfaceOp":"append"} +{"type":"tool/call","seq":238,"time":1783279386897,"data":{"turn":1,"step":2,"callId":"call_00_SvvpTh6bWybYXoO77NHg8535","name":"bash","arguments":"{\"command\":\"awk '{gsub(/blue/,\\\"green\\\")}1' settings.txt > settings.tmp && mv settings.tmp settings.txt\",\"description\":\"Replace blue with green in settings.txt\"}"}} {"type":"tool/result","seq":239,"time":1783279386915,"data":{"turn":1,"step":2,"callId":"call_00_SvvpTh6bWybYXoO77NHg8535","content":[{"type":"text","text":"(no output)"}],"isError":false},"sourceEventSeqs":[238],"surfaceOp":"append"} {"type":"step/end","seq":240,"time":1783279386916,"data":{"turn":1,"step":2}} {"type":"step/start","seq":241,"time":1783279386916,"data":{"turn":1,"step":3}} diff --git a/examples/acp-agent/tests/snapshots/fs-policy-reject/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/fs-policy-reject/stdout.golden.jsonl index 638dac6373..e21f8cda14 100644 --- a/examples/acp-agent/tests/snapshots/fs-policy-reject/stdout.golden.jsonl +++ b/examples/acp-agent/tests/snapshots/fs-policy-reject/stdout.golden.jsonl @@ -137,7 +137,7 @@ {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" use"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" bash"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} -{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" sed"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" awk"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" replace"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} @@ -152,7 +152,7 @@ {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" the"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" file"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} -{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_SvvpTh6bWybYXoO77NHg8535","title":"sed -i 's/blue/green/g' settings.txt","kind":"execute","status":"in_progress","rawInput":"sed -i 's/blue/green/g' settings.txt","content":[{"type":"content","content":{"type":"text","text":"Replace blue with green in settings.txt"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_SvvpTh6bWybYXoO77NHg8535","title":"awk '{gsub(/blue/,\"green\")}1' settings.txt > settings.tmp && mv settings.tmp settings.txt","kind":"execute","status":"in_progress","rawInput":"awk '{gsub(/blue/,\"green\")}1' settings.txt > settings.tmp && mv settings.tmp settings.txt","content":[{"type":"content","content":{"type":"text","text":"Replace blue with green in settings.txt"}}]}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_SvvpTh6bWybYXoO77NHg8535","status":"completed","content":[{"type":"content","content":{"type":"text","text":"```console\n(no output)\n```"}}]}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" bash"}}}} diff --git a/examples/acp-agent/tests/snapshots/repeat-tool-guard/input.json b/examples/acp-agent/tests/snapshots/repeat-tool-guard/input.json new file mode 100644 index 0000000000..9d2203ed57 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/repeat-tool-guard/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "prompt", "text": "Write the todo list 'watch the kettle boil' five times in a row without changing it, then reply DONE." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/repeat-tool-guard/session.jsonl b/examples/acp-agent/tests/snapshots/repeat-tool-guard/session.jsonl new file mode 100644 index 0000000000..7e50e71b3b --- /dev/null +++ b/examples/acp-agent/tests/snapshots/repeat-tool-guard/session.jsonl @@ -0,0 +1,70 @@ +{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"} +{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"Write the todo list 'watch the kettle boil' five times in a row without changing it, then reply DONE."}],"source":{"kind":"user"}},"surfaceOp":"append"} +{"type":"step/start","seq":2,"time":0,"data":{"turn":1,"step":1}} +{"type":"request/header","seq":3,"time":0,"data":{"header":{"config":{"model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"assistant/chunk","seq":4,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":5,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"call_1","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}} +{"type":"assistant/chunk","seq":6,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_1","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}} +{"type":"assistant/chunk","seq":7,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}} +{"type":"assistant/chunk","seq":8,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":9,"time":0,"data":{"turn":1,"step":1,"content":[{"type":"tool-call","id":"call_1","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[4,5,6,7,8],"surfaceOp":"append"} +{"type":"tool/call","seq":10,"time":0,"data":{"turn":1,"step":1,"callId":"call_1","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}} +{"type":"todo/write","seq":11,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}} +{"type":"tool/result","seq":12,"time":0,"data":{"turn":1,"step":1,"callId":"call_1","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[10],"surfaceOp":"append"} +{"type":"step/end","seq":13,"time":0,"data":{"turn":1,"step":1}} +{"type":"step/start","seq":14,"time":0,"data":{"turn":1,"step":2}} +{"type":"assistant/chunk","seq":15,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":16,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":0,"id":"call_2","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}} +{"type":"assistant/chunk","seq":17,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_2","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}} +{"type":"assistant/chunk","seq":18,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}} +{"type":"assistant/chunk","seq":19,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":20,"time":0,"data":{"turn":1,"step":2,"content":[{"type":"tool-call","id":"call_2","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[15,16,17,18,19],"surfaceOp":"append"} +{"type":"tool/call","seq":21,"time":0,"data":{"turn":1,"step":2,"callId":"call_2","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}} +{"type":"todo/write","seq":22,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}} +{"type":"tool/result","seq":23,"time":0,"data":{"turn":1,"step":2,"callId":"call_2","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[21],"surfaceOp":"append"} +{"type":"step/end","seq":24,"time":0,"data":{"turn":1,"step":2}} +{"type":"step/start","seq":25,"time":0,"data":{"turn":1,"step":3}} +{"type":"assistant/chunk","seq":26,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":27,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"tool-call-delta","index":0,"id":"call_3","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}} +{"type":"assistant/chunk","seq":28,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_3","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}} +{"type":"assistant/chunk","seq":29,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}} +{"type":"assistant/chunk","seq":30,"time":0,"data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":31,"time":0,"data":{"turn":1,"step":3,"content":[{"type":"tool-call","id":"call_3","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[26,27,28,29,30],"surfaceOp":"append"} +{"type":"tool/call","seq":32,"time":0,"data":{"turn":1,"step":3,"callId":"call_3","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}} +{"type":"todo/write","seq":33,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}} +{"type":"tool/result","seq":34,"time":0,"data":{"turn":1,"step":3,"callId":"call_3","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[32],"surfaceOp":"append"} +{"type":"context/message","seq":35,"time":0,"data":{"content":[{"type":"text","text":"You are repeating the exact same tool call with identical arguments. Carefully analyze the previous result before calling again: if the task is not complete, try a different approach or different arguments instead of repeating the call."}],"source":{"kind":"plugin","plugin":"repeat-tool-guard"}},"surfaceOp":"append"} +{"type":"step/end","seq":36,"time":0,"data":{"turn":1,"step":3}} +{"type":"step/start","seq":37,"time":0,"data":{"turn":1,"step":4}} +{"type":"assistant/chunk","seq":38,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":39,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"tool-call-delta","index":0,"id":"call_4","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}} +{"type":"assistant/chunk","seq":40,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_4","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}} +{"type":"assistant/chunk","seq":41,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}} +{"type":"assistant/chunk","seq":42,"time":0,"data":{"turn":1,"step":4,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":43,"time":0,"data":{"turn":1,"step":4,"content":[{"type":"tool-call","id":"call_4","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[38,39,40,41,42],"surfaceOp":"append"} +{"type":"tool/call","seq":44,"time":0,"data":{"turn":1,"step":4,"callId":"call_4","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}} +{"type":"todo/write","seq":45,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}} +{"type":"tool/result","seq":46,"time":0,"data":{"turn":1,"step":4,"callId":"call_4","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[44],"surfaceOp":"append"} +{"type":"step/end","seq":47,"time":0,"data":{"turn":1,"step":4}} +{"type":"step/start","seq":48,"time":0,"data":{"turn":1,"step":5}} +{"type":"assistant/chunk","seq":49,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":50,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"tool-call-delta","index":0,"id":"call_5","name":"todo_write","argumentsDelta":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}} +{"type":"assistant/chunk","seq":51,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_5","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}}}} +{"type":"assistant/chunk","seq":52,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}}} +{"type":"assistant/chunk","seq":53,"time":0,"data":{"turn":1,"step":5,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":54,"time":0,"data":{"turn":1,"step":5,"content":[{"type":"tool-call","id":"call_5","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}],"usage":{"inputTokens":10,"outputTokens":5}},"sourceEventSeqs":[49,50,51,52,53],"surfaceOp":"append"} +{"type":"tool/call","seq":55,"time":0,"data":{"turn":1,"step":5,"callId":"call_5","name":"todo_write","arguments":"{\"todos\": [{\"content\": \"watch the kettle boil\", \"status\": \"in_progress\"}]}"}} +{"type":"todo/write","seq":56,"time":0,"data":{"todos":[{"content":"watch the kettle boil","status":"in_progress"}]}} +{"type":"tool/result","seq":57,"time":0,"data":{"turn":1,"step":5,"callId":"call_5","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}],"isError":false},"sourceEventSeqs":[55],"surfaceOp":"append"} +{"type":"context/message","seq":58,"time":0,"data":{"content":[{"type":"text","text":"Repeated tool call detected:\n- tool: todo_write\n- consecutive_calls: 5\n- arguments: {\"todos\":[{\"content\":\"watch the kettle boil\",\"status\":\"in_progress\"}]}\nThe repeated calls are not making progress. Do not call this tool with these exact arguments again. Inspect the latest result and choose a different action, different arguments, or finish the task if enough evidence has been gathered."}],"source":{"kind":"plugin","plugin":"repeat-tool-guard"}},"surfaceOp":"append"} +{"type":"step/end","seq":59,"time":0,"data":{"turn":1,"step":5}} +{"type":"step/start","seq":60,"time":0,"data":{"turn":1,"step":6}} +{"type":"assistant/chunk","seq":61,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","seq":62,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"text-delta","index":0,"text":"DONE."}}} +{"type":"assistant/chunk","seq":63,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE."}}}} +{"type":"assistant/chunk","seq":64,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":65,"time":0,"data":{"turn":1,"step":6,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":66,"time":0,"data":{"turn":1,"step":6,"content":[{"type":"text","text":"DONE."}],"usage":{"inputTokens":10,"outputTokens":3}},"sourceEventSeqs":[61,62,63,64,65],"surfaceOp":"append"} +{"type":"step/end","seq":67,"time":0,"data":{"turn":1,"step":6}} +{"type":"turn/end","seq":68,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/repeat-tool-guard/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/repeat-tool-guard/stdout.golden.jsonl new file mode 100644 index 0000000000..a3ae2e3870 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/repeat-tool-guard/stdout.golden.jsonl @@ -0,0 +1,19 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_1","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_1","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_2","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_2","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_3","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_3","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_4","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_4","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_5","title":"Update todo list","kind":"other","status":"in_progress","rawInput":[{"content":"watch the kettle boil","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"plan","entries":[{"content":"watch the kettle boil","priority":"medium","status":"in_progress"}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_5","status":"completed","content":[{"type":"content","content":{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 0 completed."}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"DONE."}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/text-turn/session.jsonl b/examples/acp-agent/tests/snapshots/text-turn/session.jsonl index afdd312983..2a71c46b18 100644 --- a/examples/acp-agent/tests/snapshots/text-turn/session.jsonl +++ b/examples/acp-agent/tests/snapshots/text-turn/session.jsonl @@ -2,7 +2,7 @@ {"type":"turn/start","seq":0,"time":1783279329596,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} {"type":"user/message","seq":1,"time":1783279329596,"data":{"content":[{"type":"text","text":"Reply with exactly the word: PONG. Do not use any tools."}],"source":{"kind":"user"}},"surfaceOp":"append"} {"type":"step/start","seq":2,"time":1783279329598,"data":{"turn":1,"step":1}} -{"type":"request/header","seq":3,"time":1783279329598,"data":{"header":{"config":{"model":"deepseek-v4-flash"},"system":"You are an AI agent powered by the DeepSeek Harness SDK.\n\nYou are a coding assistant powered by the deepseek-v4-flash model. Your working\ndirectory is /tmp/acp-snap-cwd-q0sbE9.\n\nVerify your work by running the code or tests. Keep answers brief and\nfactual.\n\n\nUse the read tool — not shell commands like cat — to inspect text files. Results include line numbers. Use offset and limit to continue reading large files.\n\nUse the write tool to create files or completely replace file contents. Existing files are overwritten, so read an existing file first (the default fs-policy requires it) and prefer edit for targeted changes.\n\nUse the edit tool for targeted changes to existing UTF-8 text files. It replaces literal old_string with new_string; by default old_string must appear exactly once. If old_string appears multiple times, provide a more specific old_string or set replace_all to true. Read the file first (the default fs-policy requires it), unless you just created or edited it in this session.\n\nCheck the [exit code: N] marker on every bash result; investigate failures before moving on.","tools":[{"name":"subagent","description":"Delegate a self-contained task to a subagent (a separate agent that works in its own context) and return its final result. Use this to offload focused, independent work — research, a scoped implementation, an analysis — so it does not consume this conversation's context. The subagent runs to completion and you receive only its final answer, not its intermediate steps. Give it a complete, standalone prompt: it does not see this conversation.","parameters":{"type":"object","properties":{"description":{"type":"string","description":"A short (3-5 word) description of the delegated task, for display."},"prompt":{"type":"string","description":"The complete, self-contained task for the subagent. It does not share this conversation's context, so include everything it needs."}},"required":["description","prompt"]}},{"name":"subagent_fork","description":"Delegate a task to a subagent that INHERITS this conversation: a child agent seeded with all completed turns so far (it does not see the current in-flight turn), returning only its final result. Use this when the subtask builds on this conversation's context — a follow-up analysis, a review, a continuation — without consuming this conversation's context for the work itself. You receive only its final answer, not its intermediate steps.","parameters":{"type":"object","properties":{"description":{"type":"string","description":"A short (3-5 word) description of the delegated task, for display."},"prompt":{"type":"string","description":"The task for the subagent. It already sees this conversation's completed turns, so build on them freely and state only what is new."}},"required":["description","prompt"]}},{"name":"todo_write","description":"Record and update a structured task list for the current work. Send the ENTIRE list every call — it REPLACES the previous list (there are no partial updates, no per-item edits). Use it to plan multi-step work and show progress: add one todo per concrete step before you start. Keep AT MOST ONE todo `in_progress` at a time; while work remains, exactly one active task should be `in_progress`. Mark a todo `completed` the moment it is done (do not batch completions), and allow no `in_progress` item only once all work is complete. Skip the list for trivial single-step tasks. Statuses: `pending` (not started), `in_progress` (being worked on now), `completed` (finished).","parameters":{"type":"object","properties":{"todos":{"type":"array","description":"The COMPLETE task list, replacing any previous list.","items":{"type":"object","properties":{"content":{"type":"string","description":"What the task is — a short imperative line."},"status":{"type":"string","description":"pending (not started) | in_progress (now) | completed (done).","enum":["pending","in_progress","completed"]}},"required":["content","status"]}}},"required":["todos"]}},{"name":"read","description":"Read a UTF-8 text file and return line-numbered content.","parameters":{"type":"object","properties":{"file_path":{"type":"string","description":"Path to read, resolved by the filesystem backend."},"offset":{"type":"number","description":"1-based first line to return. Defaults to 1."},"limit":{"type":"number","description":"Maximum number of lines to return. Defaults to 2000."}},"required":["file_path"]}},{"name":"write","description":"Create or fully replace a UTF-8 text file.","parameters":{"type":"object","properties":{"file_path":{"type":"string","description":"Path to write, resolved by the filesystem backend."},"content":{"type":"string","description":"Full UTF-8 text content to write."}},"required":["file_path","content"]}},{"name":"edit","description":"Edit an existing UTF-8 text file by replacing literal text.","parameters":{"type":"object","properties":{"file_path":{"type":"string","description":"Path to edit, resolved by the filesystem backend."},"old_string":{"type":"string","description":"Literal text to replace. Must match exactly."},"new_string":{"type":"string","description":"Literal replacement text. Use an empty string to delete the match."},"replace_all":{"type":"boolean","description":"Replace all matches. Defaults to false; when false, old_string must appear exactly once."}},"required":["file_path","old_string","new_string"]}},{"name":"bash","description":"Execute a bash command (`bash -c`) and return its stdout/stderr. Each call runs in a fresh shell: no state (cwd, variables, functions) persists between calls — pass `workdir` instead of using `cd`. Non-zero exits are reported as `[exit code: N]`. Long output is truncated to its tail; the full output is saved to a file whose path is reported when available. Set `run_in_background: true` for long-running commands: the call returns a task id immediately; poll it with `bash_output` and stop it with `bash_kill`.","parameters":{"type":"object","properties":{"command":{"type":"string","description":"The bash command to execute."},"description":{"type":"string","description":"Clear, concise description of what this command does in active voice, 5-10 words (shown in the UI). Examples: \"ls\" → \"List files in current directory\"; \"git status\" → \"Show working tree status\"; \"npm install\" → \"Install package dependencies\"."},"timeoutMs":{"type":"number","description":"Timeout in milliseconds. The executor applies its configured default and cap, and kills the command on expiry."},"workdir":{"type":"string","description":"Working directory for this command. Defaults to the session workspace; a relative path is resolved against it."},"run_in_background":{"type":"boolean","description":"Run in the background and return a task id immediately. No timeout applies."}},"required":["command","description"]}},{"name":"bash_output","description":"Read new output from a background bash task started with `bash` + `run_in_background`. Returns only output produced since the previous bash_output call, plus the task status. Tasks keep running while you do other work; poll again later for more output.","parameters":{"type":"object","properties":{"task_id":{"type":"string","description":"Task id returned by the bash tool."}},"required":["task_id"]}},{"name":"bash_kill","description":"Ask the executor to kill a running background bash task by task id.","parameters":{"type":"object","properties":{"task_id":{"type":"string","description":"Task id returned by the bash tool."}},"required":["task_id"]}}]},"reason":"initial"}} +{"type":"request/header","seq":3,"time":1783279329598,"data":{"header":{"config":{"model":"deepseek-v4-flash"},"system":"You are an AI agent powered by the DeepSeek Harness SDK.\n\nYou are a coding assistant powered by the deepseek-v4-flash model. Your working\ndirectory is /tmp/acp-snap-cwd-q0sbE9.\n\nVerify your work by running the code or tests. Keep answers brief and\nfactual.\n\n\nUse the read tool — not shell commands like cat — to inspect text files. Results include line numbers. Use offset and limit to continue reading large files.\n\nUse the write tool to create files or completely replace file contents. Existing files are overwritten, so read an existing file first (the default fs-policy requires it) and prefer edit for targeted changes.\n\nUse the edit tool for targeted changes to existing UTF-8 text files. It replaces literal old_string with new_string; by default old_string must appear exactly once. If old_string appears multiple times, provide a more specific old_string or set replace_all to true. Read the file first (the default fs-policy requires it), unless you just created or edited it in this session.\n\nCheck the [exit code: N] marker on every bash result; investigate failures before moving on.","tools":[{"name":"bash","description":"Execute a bash command (`bash -c`) and return its stdout/stderr. Each call runs in a fresh shell: no state (cwd, variables, functions) persists between calls — pass `workdir` instead of using `cd`. Non-zero exits are reported as `[exit code: N]`. Long output is truncated to its tail; the full output is saved to a file whose path is reported when available. Set `run_in_background: true` for long-running commands: the call returns a task id immediately; poll it with `bash_output` and stop it with `bash_kill`.","parameters":{"type":"object","properties":{"command":{"type":"string","description":"The bash command to execute."},"description":{"type":"string","description":"Clear, concise description of what this command does in active voice, 5-10 words (shown in the UI). Examples: \"ls\" → \"List files in current directory\"; \"git status\" → \"Show working tree status\"; \"npm install\" → \"Install package dependencies\"."},"timeoutMs":{"type":"number","description":"Timeout in milliseconds. The executor applies its configured default and cap, and kills the command on expiry."},"workdir":{"type":"string","description":"Working directory for this command. Defaults to the session workspace; a relative path is resolved against it."},"run_in_background":{"type":"boolean","description":"Run in the background and return a task id immediately. No timeout applies."}},"required":["command","description"]}},{"name":"bash_kill","description":"Ask the executor to kill a running background bash task by task id.","parameters":{"type":"object","properties":{"task_id":{"type":"string","description":"Task id returned by the bash tool."}},"required":["task_id"]}},{"name":"bash_output","description":"Read new output from a background bash task started with `bash` + `run_in_background`. Returns only output produced since the previous bash_output call, plus the task status. Tasks keep running while you do other work; poll again later for more output.","parameters":{"type":"object","properties":{"task_id":{"type":"string","description":"Task id returned by the bash tool."}},"required":["task_id"]}},{"name":"edit","description":"Edit an existing UTF-8 text file by replacing literal text.","parameters":{"type":"object","properties":{"file_path":{"type":"string","description":"Path to edit, resolved by the filesystem backend."},"old_string":{"type":"string","description":"Literal text to replace. Must match exactly."},"new_string":{"type":"string","description":"Literal replacement text. Use an empty string to delete the match."},"replace_all":{"type":"boolean","description":"Replace all matches. Defaults to false; when false, old_string must appear exactly once."}},"required":["file_path","old_string","new_string"]}},{"name":"read","description":"Read a UTF-8 text file and return line-numbered content.","parameters":{"type":"object","properties":{"file_path":{"type":"string","description":"Path to read, resolved by the filesystem backend."},"offset":{"type":"number","description":"1-based first line to return. Defaults to 1."},"limit":{"type":"number","description":"Maximum number of lines to return. Defaults to 2000."}},"required":["file_path"]}},{"name":"subagent","description":"Delegate a self-contained task to a subagent (a separate agent that works in its own context) and return its final result. Use this to offload focused, independent work — research, a scoped implementation, an analysis — so it does not consume this conversation's context. The subagent runs to completion and you receive only its final answer, not its intermediate steps. Give it a complete, standalone prompt: it does not see this conversation.","parameters":{"type":"object","properties":{"description":{"type":"string","description":"A short (3-5 word) description of the delegated task, for display."},"prompt":{"type":"string","description":"The complete, self-contained task for the subagent. It does not share this conversation's context, so include everything it needs."}},"required":["description","prompt"]}},{"name":"subagent_fork","description":"Delegate a task to a subagent that INHERITS this conversation: a child agent seeded with all completed turns so far (it does not see the current in-flight turn), returning only its final result. Use this when the subtask builds on this conversation's context — a follow-up analysis, a review, a continuation — without consuming this conversation's context for the work itself. You receive only its final answer, not its intermediate steps.","parameters":{"type":"object","properties":{"description":{"type":"string","description":"A short (3-5 word) description of the delegated task, for display."},"prompt":{"type":"string","description":"The task for the subagent. It already sees this conversation's completed turns, so build on them freely and state only what is new."}},"required":["description","prompt"]}},{"name":"todo_write","description":"Record and update a structured task list for the current work. Send the ENTIRE list every call — it REPLACES the previous list (there are no partial updates, no per-item edits). Use it to plan multi-step work and show progress: add one todo per concrete step before you start. Keep AT MOST ONE todo `in_progress` at a time; while work remains, exactly one active task should be `in_progress`. Mark a todo `completed` the moment it is done (do not batch completions), and allow no `in_progress` item only once all work is complete. Skip the list for trivial single-step tasks. Statuses: `pending` (not started), `in_progress` (being worked on now), `completed` (finished).","parameters":{"type":"object","properties":{"todos":{"type":"array","description":"The COMPLETE task list, replacing any previous list.","items":{"type":"object","properties":{"content":{"type":"string","description":"What the task is — a short imperative line."},"status":{"type":"string","description":"pending (not started) | in_progress (now) | completed (done).","enum":["pending","in_progress","completed"]}},"required":["content","status"]}}},"required":["todos"]}},{"name":"write","description":"Create or fully replace a UTF-8 text file.","parameters":{"type":"object","properties":{"file_path":{"type":"string","description":"Path to write, resolved by the filesystem backend."},"content":{"type":"string","description":"Full UTF-8 text content to write."}},"required":["file_path","content"]}}]},"reason":"initial"}} {"type":"assistant/chunk","seq":4,"time":1783279330062,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} {"type":"assistant/chunk","seq":5,"time":1783279330062,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} {"type":"assistant/chunk","seq":6,"time":1783279330154,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" user"}}} diff --git a/knip.json b/knip.json index 89cd2fffe7..407f892775 100644 --- a/knip.json +++ b/knip.json @@ -9,7 +9,7 @@ "examples/echo-agent/tests/**/*.e2e.ts", "examples/coding-agent/tests/**/*.e2e.ts", "examples/acp-agent/tests/**/*.e2e.ts", - "examples/acp-agent/tests/**/*.snapshot.ts" + "examples/*/tests/**/*.snapshot.ts" ], "project": ["scripts/**/*.ts", "examples/**/*.ts"] }, @@ -21,10 +21,24 @@ "project": ["src/**/*.ts"], "ignoreDependencies": ["cordis"] }, + "packages/util/timeout": { + "entry": ["tests/**/*.spec.ts"], + "project": ["src/**/*.ts", "tests/**/*.ts"], + "ignoreDependencies": ["cordis"] + }, + "packages/support/acp-snapshot": { + "entry": ["tests/**/*.spec.ts", "tests/fixtures/fake-acp-agent.ts"], + "project": ["src/**/*.ts", "tests/**/*.ts"], + "ignoreDependencies": ["cordis"] + }, "packages/core/agent-loop": { "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], "project": ["src/**/*.ts", "tests/**/*.ts"] }, + "packages/code-runtime/code-runtime-worker": { + "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], + "project": ["src/**/*.ts", "tests/**/*.ts"] + }, "packages/llm/llm-deepseek": { "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], "project": ["src/**/*.ts", "tests/**/*.ts"] diff --git a/package.json b/package.json index dc8f487bd9..bba471ea5f 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "type": "module", "packageManager": "pnpm@11.7.0", "engines": { - "node": ">=24" + "node": "^22.19.0 || >=24.0.0" }, "workspaces": [ "vendor/*", @@ -47,6 +47,7 @@ "gen-cordis-catalog": "tsx scripts/gen-cordis-catalog.ts", "gen-rfc-index": "tsx scripts/gen-rfc-index.ts", "verify-cordis-catalog": "tsx scripts/gen-cordis-catalog.ts --check", + "verify-export-jsdoc": "tsx scripts/verify-export-jsdoc.ts", "gen-tool-catalog": "tsx scripts/gen-tool-catalog.ts", "verify-tool-catalog": "tsx scripts/gen-tool-catalog.ts --check", "gen-config-catalog": "tsx scripts/gen-config-catalog.ts", @@ -58,7 +59,7 @@ "gen-module-graph": "tsx scripts/gen-module-graph.ts", "verify-module-graph": "tsx scripts/gen-module-graph.ts --check", "constraints": "tsx scripts/check-workspace-constraints.ts", - "doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-tool-catalog && pnpm run verify-config-catalog && pnpm run verify-persistence-catalog && pnpm run verify-doc-graphs && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-mermaid && pnpm run verify-rfc-classification && pnpm run verify-rfc-format && pnpm run verify-type-equiv && pnpm run verify-translation-pairing && pnpm run verify-doc-budgets", + "doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-export-jsdoc && pnpm run verify-tool-catalog && pnpm run verify-config-catalog && pnpm run verify-persistence-catalog && pnpm run verify-doc-graphs && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-mermaid && pnpm run verify-rfc-classification && pnpm run verify-rfc-format && pnpm run verify-type-equiv && pnpm run verify-translation-pairing && pnpm run verify-doc-budgets", "hygiene": "pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types", "demo:echo": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/echo-agent/cordis.yml", "demo:repl": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/coding-agent/cordis.yml", @@ -70,7 +71,7 @@ "@stylistic/eslint-plugin": "^5.10.0", "@types/jsdom": "^28.0.3", "@types/mdast": "^4.0.4", - "@types/node": "^25.3.5", + "@types/node": "^22.20.0", "@vitest/coverage-v8": "^4.1.8", "eslint": "^10.4.1", "fast-check": "^4.8.0", diff --git a/packages/README.md b/packages/README.md index f07e0d5a36..bc09826777 100644 --- a/packages/README.md +++ b/packages/README.md @@ -11,11 +11,14 @@ Packages are grouped by modular role at `packages///`. The group dir | [`core/`](core/README.md) | Product API spine: session, system-prompt, tools, agent, and the concrete loop | Product — stable surface | | [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: the executor seam, a local impl, and the model-facing tool | Product — stable surface | +| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the abstract runtime seam for model-written programs + a worker-thread backend | Product — stable surface | | [`fs/`](fs/README.md) | Filesystem capability family: the abstract seam, a local impl, and the model-facing file tools | Product — stable surface | | [`compact/`](compact/README.md) | Compaction capability family: the abstract seam + a basic backend (tool deferred) | Product — stable surface | | [`subagent/`](subagent/README.md) | Subagent capability family: the provider-registry seam and the model-facing delegation tool | Product — stable surface | | [`web/`](web/README.md) | Web capability family: the abstract seam, search/fetch provider impls, and the model-facing web tools | Product — stable surface | -| [`todo/`](todo/README.md) | Todo/planning family: the model-facing `todo_write` tool (whole-list task tracking on the session log) | Product — stable surface | +| [`timeout/`](timeout/README.md) | Tool-call timeout policy: the `tools/execute` deadline enforcer | Product — stable surface | +| [`todo/`](todo/README.md) | Todo/planning family: the model-facing `todo_write` tool | Product — stable surface | +| [`guard/`](guard/README.md) | Loop-hygiene guards: advisory repeat-call reminders | Product — stable surface | | [`hooks/`](hooks/README.md) | Hook bridges + the shared Claude Code / Codex wire-protocol library | Product — stable surface | | [`session-persistence/`](session-persistence/README.md) | Persistence capability family: the seam + JSONL/SQLite backends | Product — stable surface | | [`ui/`](ui/README.md) | Editor/client integration surfaces (the ACP bridge) + the app packages | Product — stable surface | @@ -28,6 +31,6 @@ The split is the point: a package's group says whether it is part of the product The inter-package dependency graph is generated: [docs/module-graph.md](../docs/module-graph.md) (`pnpm run gen-module-graph`, freshness-gated in CI). -The rule it must obey: **extension plugins depend on interfaces, never on the concrete loop.** `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. The sanctioned exception is a **composition/bundle** package like `dsh-agent-core`, whose whole job is to assemble the concrete spine: it depends on `dsh-agent-loop` (and the other concrete spine plugins) on purpose. The rule constrains plugins that EXTEND the system, not the bundle that COMPOSES it — swapping the loop means shipping a different bundle, not rewiring every extension. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)). +The rule it must obey: **extension plugins depend on interfaces, never on the concrete loop.** `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. The sanctioned exception is a **composition/bundle** package like `dsh-agent-core`, whose whole job is to assemble the concrete spine: it depends on `dsh-agent-loop` (and the other concrete spine plugins) on purpose. The rule constrains plugins that EXTEND the system, not the bundle that COMPOSES it. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)). Each package has its own `README.md` with purpose, service API, events, extension points, and deliberate non-goals (TODOs). diff --git a/packages/bash/bash-local/package.json b/packages/bash/bash-local/package.json index bc1dc7eb40..e3c7ffe33b 100644 --- a/packages/bash/bash-local/package.json +++ b/packages/bash/bash-local/package.json @@ -23,6 +23,7 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-bash": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "dependencies": { @@ -30,6 +31,7 @@ }, "devDependencies": { "@deepseek-ai/dsh-bash": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", "cordis": "^4.0.0-rc.6" } } diff --git a/packages/bash/bash-local/src/index.ts b/packages/bash/bash-local/src/index.ts index cdac3985b8..3e09d7e35b 100644 --- a/packages/bash/bash-local/src/index.ts +++ b/packages/bash/bash-local/src/index.ts @@ -18,6 +18,7 @@ import { Context } from 'cordis' import z from 'schemastery' import { BashExecutor, BashTaskId } from '@deepseek-ai/dsh-bash' import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash' +import { clampTimeout, deadline, timeoutOf } from '@deepseek-ai/dsh-timeout' import { DEFAULT_GRACE_MS, runBash } from './run.ts' import type { RunInternals, RunningBash } from './run.ts' @@ -114,8 +115,12 @@ export class LocalBashExecutor extends BashExecutor { * values and never re-default. */ resolve(request: BashExecRequest): BashExecSpec { - if (request.timeoutMs !== undefined) assertPositiveFinite('request.timeoutMs', request.timeoutMs) - const timeoutMs = Math.min(request.timeoutMs ?? this.config.timeoutMs, this.config.maxTimeoutMs) + const timeoutMs = clampTimeout( + request.timeoutMs, + this.config.timeoutMs, + this.config.maxTimeoutMs, + 'bash-local: request.timeoutMs', + ) return { command: request.command, workdir: request.workdir ?? this.config.cwd ?? process.cwd(), @@ -132,29 +137,39 @@ export class LocalBashExecutor extends BashExecutor { } async run(spec: BashExecSpec): Promise { + // One fused deadline drives both the timeout and upstream cancellation; + // runBash listens on d.signal and runs the SIGTERM→grace→SIGKILL kill. + // `using` clears the timer across the awaited process lifetime. + using d = deadline(spec.signal, spec.timeoutMs, 'BASH_TIMEOUT') const outcome = await runBash({ command: spec.command, cwd: spec.workdir, - timeoutMs: spec.timeoutMs, maxOutputBytes: this.config.maxOutputBytes, graceMs: this.config.graceMs, - signal: spec.signal, + signal: d.signal, stdin: spec.stdin, env: spec.env, }, this.internals).done - return { ...outcome, timeoutMs: spec.timeoutMs } + // Classify the FIRST abort reason: a BASH_TIMEOUT TimeoutReason means our + // timeout cut the command short; any other abort — an upstream cancel, or a + // foreign (outer) deadline's timeout under nesting — is aborted. Scoping to + // our own code keeps a nested outer deadline from reading as our timeout. + // Mutually exclusive by construction — the fused signal reports one cause. + const timedOut = timeoutOf(d.signal, 'BASH_TIMEOUT') !== undefined + const aborted = d.signal.aborted && !timedOut + return { ...outcome, timedOut, aborted, timeoutMs: spec.timeoutMs } } start(spec: BashExecSpec): BashTask { // No timeout for background tasks (matches Claude Code, which detaches // the timeout when backgrounding); callers stop tasks via kill() — or // via spec.signal, which the seam contract honors for background runs - // too (runBash wires it to the group kill). spec.timeoutMs is ignored - // here by design. + // too (runBash wires it to the group kill). No deadline is created here, + // so spec.timeoutMs is ignored by design — background tasks stay + // timeout-free (see the timeout-library RFC). const running = runBash({ command: spec.command, cwd: spec.workdir, - timeoutMs: 0, maxOutputBytes: this.config.maxOutputBytes, graceMs: this.config.graceMs, signal: spec.signal, @@ -174,8 +189,10 @@ export class LocalBashExecutor extends BashExecutor { stdoutOffset: 0, stderrOffset: 0, done: running.done.then((outcome) => { - // Abort-killed tasks report as killed, not completed. - if (task.status === 'running') task.status = outcome.aborted ? 'killed' : 'completed' + // Abort-killed tasks report as killed, not completed. Background runs + // forward only the upstream signal (no timeout), so its aborted state + // is the authoritative "was this cancelled" signal. + if (task.status === 'running') task.status = spec.signal?.aborted === true ? 'killed' : 'completed' task.exitCode = outcome.exitCode task.signal = outcome.signal this.notifyTaskDone(task) diff --git a/packages/bash/bash-local/src/run.ts b/packages/bash/bash-local/src/run.ts index 489d380787..bc4a017dea 100644 --- a/packages/bash/bash-local/src/run.ts +++ b/packages/bash/bash-local/src/run.ts @@ -6,6 +6,12 @@ * Everything here is deliberately free of Cordis concepts so it can be unit * tested in isolation; `LocalBashExecutor` owns lifecycle and configuration. * + * runBash owns NO timing: it kills the process group when its `spec.signal` + * fires and does not distinguish a timeout from a cancel. The executor fuses + * timeout + upstream cancellation into that one signal via + * `@deepseek-ai/dsh-timeout`'s `deadline`, and classifies the outcome from the + * signal afterward — the timing/classification half is shared, the kill is not. + * * Design notes (surveyed against Claude Code, OpenCode, Codex, and pi — see * the package README): spawn-per-call with `detached: true` so the child * leads its own process group; kills target the group (`kill(-pid)`) so @@ -56,6 +62,8 @@ export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i * by in-process plugins (the hooks bridges), not the model — `dsh-tool-bash` * builds its request from named fields only and does not forward model input * here (see its README, § "The tool builds its request from named args only"). + * @param extra - caller-supplied entries merged last; an explicit entry wins even against the scrub and the overrides. + * @returns the environment to hand to `spawn` for the child process. */ export function childEnv(extra?: Record): NodeJS.ProcessEnv { const env: NodeJS.ProcessEnv = {} @@ -69,13 +77,17 @@ export function childEnv(extra?: Record): NodeJS.ProcessEnv { export interface SpawnSpec { command: string cwd: string - /** Kill the process group after this many milliseconds. 0 = no timeout. */ - timeoutMs: number /** Per-stream in-memory cap; overflow spills to disk (tail kept in memory). */ maxOutputBytes: number /** Grace period between the SIGTERM and the SIGKILL escalation on a kill. */ graceMs: number - /** Abort signal — kills the process group when fired. */ + /** + * Abort signal — kills the process group when it fires. The executor owns + * timing: `run()` passes a fused timeout/cancel deadline signal (see + * `@deepseek-ai/dsh-timeout`), `start()` passes the bare upstream signal. + * runBash only listens and kills; it does NOT classify why (the executor + * reads the signal's reason afterward). + */ signal?: AbortSignal | undefined /** * Bytes to write to the child's stdin, then close it. Absent (or empty) @@ -92,12 +104,15 @@ export interface SpawnSpec { env?: Record | undefined } -/** Raw outcome of one closed process (before result shaping). */ +/** + * Raw outcome of one closed process (before result shaping). Deliberately + * carries NO timeout/cancel classification: runBash kills on abort but does not + * decide why — the executor's `run()`/`start()` reads the deadline signal it + * owns to classify `timedOut`/`aborted` (see the package README). + */ export interface SpawnOutcome { exitCode: number | null signal: NodeJS.Signals | null - timedOut: boolean - aborted: boolean stdout: CollectedOutput stderr: CollectedOutput } @@ -147,6 +162,14 @@ export class OutputCollector { private readonly spillDir: string, ) {} + /** + * Ingest one stream chunk, counting it toward the whole-stream total. On + * first overflow of the in-memory cap a spill file is opened and every chunk + * (already-collected ones included) is appended there from then on; the + * in-memory tail then drops whole chunks from its head (or the head of a + * single over-cap chunk) until it fits the cap again. + * @param chunk - the raw bytes from one stream 'data' event. + */ push(chunk: Buffer): void { this.total += chunk.length const overflows = this.bytes + chunk.length > this.maxBytes @@ -190,7 +213,10 @@ export class OutputCollector { // the bottom of this file) and `totalBytes` is read only by a test. The live // background-poll path goes through `readFrom()`, so inline snapshot() into // finalize() and drop or privatize the totalBytes getter. - /** Read the collected tail without finalizing (the final-result snapshot). */ + /** + * Read the collected tail without finalizing (the final-result snapshot). + * @returns the retained tail text, the truncation flag, and the spill path when one was created. + */ snapshot(): CollectedOutput { return { text: Buffer.concat(this.chunks).toString('utf8'), @@ -209,6 +235,8 @@ export class OutputCollector { * pushed since `fromByte`. When `fromByte` has already slid out of the * in-memory tail window, the read is `lossy` — it returns the whole * retained tail and the gap is only recoverable from the spill file. + * @param fromByte - whole-stream offset to resume from (a prior read's `nextOffset`; 0 for the first read). + * @returns the delta text, the offset for the next read, the `lossy` flag, and the spill path when one was created. */ readFrom(fromByte: number): { text: string; nextOffset: number; lossy: boolean; spillPath?: string } { const windowStart = this.total - this.bytes @@ -223,7 +251,12 @@ export class OutputCollector { } } - /** Close the spill file (if any) and return the final output. */ + /** + * Close the spill file (if any) and return the final output. A failed close + * (delayed writeback fault) stops advertising the spill path — the file may + * be missing its tail — but still returns the in-memory result. + * @returns the final collected output: tail text, truncation flag, and the spill path when intact. + */ finalize(): CollectedOutput { if (this.spillFd !== undefined) { try { @@ -249,6 +282,8 @@ export class OutputCollector { * host process — a kill that cannot be delivered is reported by the process * NOT dying, which callers already handle via escalation/timeouts. No-op for * non-positive pids (spawn never started a process). + * @param pid - the group leader's pid; non-positive means the spawn failed and the call is a no-op. + * @param sig - the signal to deliver to the whole group. */ export function killGroup(pid: number, sig: NodeJS.Signals): void { if (pid <= 0) return @@ -290,6 +325,9 @@ export interface RunningBash { * exec sessions addressable via session ids + stdin writes. We deliberately * spawn a fresh non-login `bash -c` per call for determinism (no rc files, * no inherited shell state); revisit when real workflows demand it. + * @param spec - the fully-resolved run (command, cwd, limits); no defaulting happens here. + * @param internals - test-only knobs; omitted fields fall back to the private per-process spill dir. + * @returns the live handle: pid, the two live collectors, the outcome promise, and `kill()`. */ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningBash { const spillDir = internals.spillDir ?? privateSpillDir() @@ -318,9 +356,6 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB child.stdout.on('data', (chunk: Buffer) => { stdout.push(chunk) }) child.stderr.on('data', (chunk: Buffer) => { stderr.push(chunk) }) - let timedOut = false - let aborted = false - let killTimer: NodeJS.Timeout | undefined let graceTimer: NodeJS.Timeout | undefined // pid is undefined when the spawn itself fails (bad cwd, missing binary); @@ -333,17 +368,12 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB graceTimer = setTimeout(() => { killGroup(pid, 'SIGKILL') }, spec.graceMs) } - if (spec.timeoutMs > 0) { - killTimer = setTimeout(() => { - timedOut = true - kill() - }, spec.timeoutMs) - } - - const onAbort = (): void => { - aborted = true - kill() - } + // runBash owns no timer: the executor's `run()` fuses timeout+cancel into one + // deadline signal (`@deepseek-ai/dsh-timeout`) and passes it here; we only + // listen and run the SIGTERM→grace→SIGKILL kill. Whether the abort was a + // timeout or an upstream cancel is classified by the executor from that + // signal, not tracked here. + const onAbort = (): void => { kill() } spec.signal?.addEventListener('abort', onAbort, { once: true }) // Write stdin and close it, but ONLY when the caller supplied bytes — with no @@ -376,14 +406,11 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB resolve({ exitCode, signal, - timedOut, - aborted, stdout: stdout.finalize(), stderr: stderr.finalize(), }) }) function cleanup(): void { - if (killTimer !== undefined) clearTimeout(killTimer) if (graceTimer !== undefined) clearTimeout(graceTimer) spec.signal?.removeEventListener('abort', onAbort) } diff --git a/packages/bash/bash-local/tests/executor.spec.ts b/packages/bash/bash-local/tests/executor.spec.ts index ce89b2a0ae..98095b8d47 100644 --- a/packages/bash/bash-local/tests/executor.spec.ts +++ b/packages/bash/bash-local/tests/executor.spec.ts @@ -90,8 +90,8 @@ describe('LocalBashExecutor.run', () => { it('kill escalation uses the configured graceMs (a TERM-trapping task dies by SIGKILL)', async () => { const { bash } = await setup() // setup pins graceMs: 200 via config - const task = bash.start(bash.resolve({ command: 'trap \'\' TERM; sleep 60' })) - await new Promise(resolve => setTimeout(resolve, 100)) + const task = bash.start(bash.resolve({ command: 'trap \'\' TERM; echo ready; while :; do sleep 60 & wait $!; done' })) + await readUntil(bash, task.id, 'ready\n') bash.kill(task.id) await task.done expect(task.signal).toBe('SIGKILL') @@ -101,6 +101,8 @@ describe('LocalBashExecutor.run', () => { const { bash } = await setup({ timeoutMs: 60_000 }) const result = await bash.run(bash.resolve({ command: 'sleep 60', timeoutMs: 100 })) expect(result.timedOut).toBe(true) + // Mutually exclusive: a timeout classifies as timedOut, never also aborted. + expect(result.aborted).toBe(false) expect(result.timeoutMs).toBe(100) }) @@ -111,6 +113,20 @@ describe('LocalBashExecutor.run', () => { setTimeout(() => { controller.abort() }, 50) const result = await pending expect(result.aborted).toBe(true) + // Mutually exclusive: an upstream cancel classifies as aborted, never also timedOut. + expect(result.timedOut).toBe(false) + }) + + it('classifies a self-killed command as neither timed out nor aborted', async () => { + // The command kills itself (SIGTERM) with no timeout and no upstream abort: + // the deadline signal never fires, so both classifications are false — the + // fused-signal classification reports the cause that cut the command short, + // and here nothing the executor owns did. + const { bash } = await setup({ timeoutMs: 60_000 }) + const result = await bash.run(bash.resolve({ command: 'kill -TERM $$' })) + expect(result.signal).toBe('SIGTERM') + expect(result.timedOut).toBe(false) + expect(result.aborted).toBe(false) }) it('rejects on spawn failure (bad workdir)', async () => { diff --git a/packages/bash/bash-local/tests/run.spec.ts b/packages/bash/bash-local/tests/run.spec.ts index d2888e2fee..1d6e93afe3 100644 --- a/packages/bash/bash-local/tests/run.spec.ts +++ b/packages/bash/bash-local/tests/run.spec.ts @@ -26,7 +26,6 @@ function spec(command: string, overrides: Partial[0]> return { command, cwd: process.cwd(), - timeoutMs: 0, maxOutputBytes: 64_000, graceMs: 3_000, ...overrides, @@ -56,13 +55,25 @@ async function waitForStdout(running: RunningBash, expected: string, timeoutMs = throw new Error(`stdout did not include ${JSON.stringify(expected)} after ${timeoutMs}ms`) } +async function waitForPidFile(path: string, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs + while (Date.now() < deadline) { + try { + const pid = Number(readFileSync(path, 'utf8').trim()) + if (Number.isSafeInteger(pid) && pid > 0) return pid + } catch { + // The child shell has not written the pid file yet. + } + await new Promise(resolve => setTimeout(resolve, 20)) + } + throw new Error(`pid file ${path} was not written after ${timeoutMs}ms`) +} + describe('runBash', () => { it('captures stdout on success', async () => { const result = await runBash(spec('echo hello')).done expect(result.exitCode).toBe(0) expect(result.signal).toBeNull() - expect(result.timedOut).toBe(false) - expect(result.aborted).toBe(false) expect(result.stdout.text).toBe('hello\n') expect(result.stdout.truncated).toBe(false) expect(result.stderr.text).toBe('') @@ -97,17 +108,22 @@ describe('runBash', () => { expect(result.stdout.text.trim()).toMatch(/\/tmp$/) }) - it('kills with SIGTERM on timeout', async () => { + it('kills the process group with SIGTERM when the signal fires', async () => { + // runBash owns no timer: it kills on abort. The executor drives the timeout + // by firing this signal via a deadline (see executor.spec.ts); here we + // assert the kill itself lands as SIGTERM. + const controller = new AbortController() const start = Date.now() - const result = await runBash(spec('sleep 60', { timeoutMs: 100 })).done + const running = runBash(spec('sleep 60', { signal: controller.signal })) + setTimeout(() => { controller.abort('deadline') }, 100) + const result = await running.done expect(Date.now() - start).toBeLessThan(5_000) - expect(result.timedOut).toBe(true) expect(result.signal).toBe('SIGTERM') expect(result.exitCode).toBeNull() }) it('escalates to SIGKILL when SIGTERM is trapped', async () => { - const running = runBash(spec('trap \'\' TERM; echo ready; sleep 60', { graceMs: 200 })) + const running = runBash(spec('trap \'\' TERM; echo ready; while :; do sleep 60 & wait $!; done', { graceMs: 200 })) await waitForStdout(running, 'ready\n') running.kill() const result = await running.done @@ -119,8 +135,7 @@ describe('runBash', () => { // group must take the sleep down with bash. const pidFile = join(spillDir, `grandchild-${Date.now()}.pid`) const running = runBash(spec(`sleep 60 & echo $! > ${pidFile}; wait`)) - await new Promise(resolve => setTimeout(resolve, 300)) - const grandchild = Number(readFileSync(pidFile, 'utf8').trim()) + const grandchild = await waitForPidFile(pidFile) expect(grandchild).toBeGreaterThan(0) running.kill() @@ -134,7 +149,6 @@ describe('runBash', () => { const running = runBash(spec('sleep 60', { signal: controller.signal })) setTimeout(() => { controller.abort('user cancelled') }, 50) const result = await running.done - expect(result.aborted).toBe(true) expect(result.signal).toBe('SIGTERM') }) @@ -211,7 +225,6 @@ describe('stdin and extra env (set by in-process plugins)', () => { const big = 'x'.repeat(1024 * 1024) const result = await runBash(spec('exit 7', { stdin: big })).done expect(result.exitCode).toBe(7) - expect(result.aborted).toBe(false) }) }) @@ -339,11 +352,11 @@ describe('abort edge cases', () => { .toThrow(/aborted before spawn: aborted/) }) - it('reports an externally self-killed command without the timeout marker', async () => { + it('reports the terminating signal of an externally self-killed command', async () => { + // runBash reports the raw signal; whether it counts as timeout/cancel is the + // executor's classification (a self-kill is neither) — see executor.spec.ts. const result = await runBash(spec('kill -TERM $$')).done expect(result.signal).toBe('SIGTERM') - expect(result.timedOut).toBe(false) - expect(result.aborted).toBe(false) }) }) @@ -396,10 +409,9 @@ describe('review fixes: env scrubbing and spill hardening', () => { it('honors AbortSignal on background-style runs (no timeout)', async () => { const controller = new AbortController() - const running = runBash(spec('sleep 60', { timeoutMs: 0, signal: controller.signal })) + const running = runBash(spec('sleep 60', { signal: controller.signal })) setTimeout(() => { controller.abort() }, 50) const result = await running.done - expect(result.aborted).toBe(true) expect(result.signal).toBe('SIGTERM') }) }) diff --git a/packages/bash/bash-local/tsconfig.json b/packages/bash/bash-local/tsconfig.json index ae31546543..02448770f4 100644 --- a/packages/bash/bash-local/tsconfig.json +++ b/packages/bash/bash-local/tsconfig.json @@ -20,6 +20,9 @@ { "path": "../../util/brand" }, + { + "path": "../../util/timeout" + }, { "path": "../../bash/bash" } diff --git a/packages/bash/bash/src/types.ts b/packages/bash/bash/src/types.ts index 9acd5c7cb7..4715ace318 100644 --- a/packages/bash/bash/src/types.ts +++ b/packages/bash/bash/src/types.ts @@ -11,7 +11,11 @@ import type { Branded } from '@deepseek-ai/dsh-brand' /** Identifies one background task within an executor (generated `bash-N`). */ export type BashTaskId = Branded<'BashTaskId'> -/** Brand a string as a {@link BashTaskId}. */ +/** + * Brand a string as a {@link BashTaskId}. + * @param id - the raw task-id string (the executor generates `bash-N`). + * @returns the same string, branded; no validation is performed. + */ export function BashTaskId(id: string): BashTaskId { return id as BashTaskId } @@ -26,7 +30,12 @@ export function BashTaskId(id: string): BashTaskId { */ export type OwnerToken = Branded<'OwnerToken'> -/** Brand a string as an {@link OwnerToken}. */ +/** + * Brand a string as an {@link OwnerToken}. Only the consuming boundary + * (`dsh-tool-bash`) should cast its own id vocabulary in — see the type's doc. + * @param id - the consumer's raw owner identity (the tool layer passes the owning agent's session id). + * @returns the same string, branded; no validation is performed. + */ export function OwnerToken(id: string): OwnerToken { return id as OwnerToken } diff --git a/packages/bash/tool-bash/src/index.ts b/packages/bash/tool-bash/src/index.ts index 478ed70ca5..d4a3105165 100644 --- a/packages/bash/tool-bash/src/index.ts +++ b/packages/bash/tool-bash/src/index.ts @@ -99,6 +99,8 @@ function streamText(output: CollectedOutput): string { * stderr section, then exit-status markers. Non-zero exits are REPORTED, not * errored — the model decides how to react; only infrastructure failures * (spawn errors, aborts) surface as isError results. + * @param result - the completed foreground run from the executor. + * @returns the model-facing text: output body (or `(no output)`), then any timeout/signal/exit markers, each on its own line. */ export function renderResult(result: BashRunResult): string { const out = streamText(result.stdout) diff --git a/packages/code-runtime/README.md b/packages/code-runtime/README.md new file mode 100644 index 0000000000..0310a57b1a --- /dev/null +++ b/packages/code-runtime/README.md @@ -0,0 +1,10 @@ +# code-runtime/ — code-execution capability family + +The code-execution capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's Code Mode, specified alongside the seam in the [Code Mode RFC](../../docs/rfc/proposed/feature/2026-06-15-code-mode.md). **Product** packages. + +| Package | Role | ctx key | +|---|---|---| +| `code-runtime/` | Abstract code-execution seam (interface + vocabulary) | `ctx.codeRuntime` | +| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend: fresh worker per run, TypeScript via host-side type-strip (annotations advisory, never type-checked), port-bridged bindings, budget/heap containment | registers `ctx.codeRuntime` | + +The interface lives at `code-runtime/code-runtime/`; the shipped backend at `code-runtime/code-runtime-worker/`. Backends differ by execution substrate (worker thread, process, container) and by source language — both readonly descriptors on the service — and register `ctx.codeRuntime` without touching the interface or its consumer; that split is what makes a hardened backend a drop-in later. diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md new file mode 100644 index 0000000000..b8f440397a --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -0,0 +1,32 @@ +# @deepseek-ai/dsh-code-runtime-worker + +Worker-thread implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam: `WorkerCodeRuntime` runs each program in ONE fresh Node `worker_threads.Worker` — TypeScript in, type-stripped host-side, bindings bridged over the message port, `{ value, logs, error? }` out. **Containment, not a security boundary**: trust posture is bash-equivalent by design (the [Code Mode RFC](../../../docs/rfc/proposed/feature/2026-06-15-code-mode.md) § Trust posture), with containment bash does not have — separate isolate, empty environment, heap cap, hard termination. + +## Config + +```yaml +- id: code-runtime + name: '@deepseek-ai/dsh-code-runtime-worker' + config: + computeMs: 60000 # busy-time budget (measured event-loop active time) + maxWallMs: 600000 # wall-clock ceiling; never pauses for anything + maxLogBytes: 65536 # shared byte budget for captured log text + maxValueBytes: 32768 # rendered-completion-value cap + maxOldGenerationSizeMb: 512 # worker heap cap (resourceLimits) +``` + +Every field is validated (positive numbers) and defaulted; there are no other tunables. + +## Design + +- **One fresh worker per run, no pooling** — a program's world dies with its worker: no cross-run state to log, state bleed unrepresentable, runs reconstructable from the session log alone. +- **Type-strip host-side, in execution context** — the program is wrapped in an async-function shell, stripped with `node:module`'s `stripTypeScriptTypes` (erasable syntax only — `enum`/namespaces are rejected as a program `exception` and no worker spawns), and sliced back out byte-positioned; it then executes as the body of an `AsyncFunction`, so top-level `await`/`return` work. +- **The port assumes a hostile peer** — model code can reach `parentPort` and forge traffic, so every inbound message is shape-validated and REBUILT before anything reads it (`null`, primitives, junk types, and malformed payloads drop without a throw; forged extra fields never ride along), the host answers each call id at most once, resolves binding names as OWN properties only (a forged `constructor` cannot walk a prototype chain), drops post-settlement replies, and converts a non-cloneable binding resolution into an error reply. Forged `log`/`done` messages cannot bypass the caps: one host-side ledger bounds everything that lands in `logs`, and the completion value is re-capped host-side. Worker-side namespaces are null-prototype with `defineProperty`, so `__proto__`-shaped binding names are ordinary keys. +- **Two independent budgets, because the peer is hostile** — `computeMs` meters the worker's MEASURED busy time (`worker.performance.eventLoopUtilization()` polling): a hot loop cannot hide behind a pending decoy dispatch, and a program awaiting a slow tool accrues nothing. `maxWallMs` backstops what busy time cannot see (awaiting a promise nobody resolves). Both funnel into `worker.terminate()`, which ends hot synchronous loops too; heap overflow surfaces as the worker's OOM exit (`kind: 'worker-exit'`). +- **Logs stream eagerly** — console/stdout/stderr entries cross the port as they happen, so a timed-out or killed program still shows what it printed. ONE shared `maxLogBytes` ledger bounds everything: streamed entries, forged port traffic, and pipe bytes that bypass the patched streams (appended after), with the overflow marked in-band once. +- **Empty environment** — the worker gets `env: {}` and `execArgv: []`: no ambient credentials (stronger than the scrubbed-env rule for spawned commands) and no inherited loader flags. +- **Dispose to quiescence** — teardown fails in-flight runs as `abort` and AWAITS each worker's exit before resolving. + +## The worker entry, unbuilt and built + +`worker.ts` is deliberately erasable-only TypeScript with type-only cross-package imports: unbuilt (vitest/tsx), the host spawns `src/worker.ts` directly and Node's native type stripping loads it; built, the entry ships as the sibling bundle `lib/worker.js` (its own tsdown entry). The built path is pinned by `tests/built-lib.e2e.ts`, the real-load-path guard from [docs/testing.md](../../../docs/testing.md). diff --git a/packages/code-runtime/code-runtime-worker/package.json b/packages/code-runtime/code-runtime-worker/package.json new file mode 100644 index 0000000000..85df6a0d78 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/package.json @@ -0,0 +1,36 @@ +{ + "name": "@deepseek-ai/dsh-code-runtime-worker", + "description": "Worker-thread implementation of the DeepSeek Harness code-execution seam", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/worker.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-code-runtime": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-code-runtime": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/code-runtime/code-runtime-worker/src/bootstrap.ts b/packages/code-runtime/code-runtime-worker/src/bootstrap.ts new file mode 100644 index 0000000000..f2e0d343f3 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/src/bootstrap.ts @@ -0,0 +1,301 @@ +/** + * Worker-side execution logic, written as plain functions over an injected + * port so the unit suite can run every line IN-PROCESS against a fake port + * (a real worker thread is a separate V8 isolate the coverage provider + * cannot observe). The real worker entry (`worker.ts`) is a thin + * self-executing glue file over {@link runWorkerMain}, excluded from + * coverage the same way `bin.ts` entrypoints are, and exercised end-to-end + * by the integration tests that spawn real workers. + * + * @module @deepseek-ai/dsh-code-runtime-worker/src/bootstrap + */ + +import { inspect } from 'node:util' +import { serialize } from 'node:v8' +import type { CodeLogEntry } from '@deepseek-ai/dsh-code-runtime' +import { logTruncationMarker } from './protocol.ts' +import type { DoneMessage, ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts' + +/** The port surface the bootstrap needs — satisfied by `parentPort` and by the tests' fake. */ +export interface BootstrapPort { + postMessage(message: WorkerToHost): void + on(event: 'message', listener: (message: ReplyMessage) => void): void +} + +/** + * A writable stream's `write` slot, as the bootstrap patches it (see + * {@link captureStreamWrites}). Method-typed so the real + * `process.stdout`/`process.stderr` (narrower chunk parameters) remain + * assignable. + */ +export interface PatchableStream { + write(chunk: unknown, ...rest: unknown[]): boolean +} + +/** + * Ordered log capture under one shared byte budget, delivered to a sink as + * each entry lands (the real sink streams entries over the port eagerly, so + * captured output survives a mid-run termination). Once the budget is + * exhausted it emits exactly one in-band marker entry (on the `stderr` + * diagnostics channel) and silently drops everything after — the cap is a + * blast-radius bound, so "how much was lost" intentionally stays unmeasured. + */ +export class LogBuffer { + private remaining: number + private truncated = false + // Explicit fields, not constructor parameter properties: this module loads + // under Node's native strip-only mode, which rejects non-erasable syntax — + // and parameter properties are non-erasable. + private readonly maxBytes: number + private readonly sink: (entry: CodeLogEntry) => void + + constructor(maxBytes: number, sink: (entry: CodeLogEntry) => void) { + this.maxBytes = maxBytes + this.sink = sink + this.remaining = maxBytes + } + + /** + * Emit one entry to the sink, charging its text against the budget (drops + marks once exhausted). + * @param entry - the log entry to deliver. + */ + push(entry: CodeLogEntry): void { + if (this.truncated) return + const cost = Buffer.byteLength(entry.text, 'utf8') + if (cost > this.remaining) { + this.truncated = true + this.sink({ source: 'stderr', text: logTruncationMarker(this.maxBytes) }) + return + } + this.remaining -= cost + this.sink(entry) + } +} + +/** The five console methods the shim captures, in the seam's level vocabulary. */ +const CONSOLE_LEVELS = ['log', 'info', 'warn', 'error', 'debug'] as const + +/** + * A `console` replacement whose five leveled methods render their arguments + * `util.inspect`-style (matching real console formatting closely enough for + * a model to recognize its own output) into the buffer. Only these five + * exist — the program gets a deliberately small console, not Node's full + * surface. + * @param logs - the buffer every rendered line is pushed into. + * @returns the five-method console object handed to the program. + */ +export function makeConsoleShim(logs: LogBuffer): Record<(typeof CONSOLE_LEVELS)[number], (...args: unknown[]) => void> { + const render = (args: unknown[]): string => + args.map(arg => typeof arg === 'string' ? arg : inspect(arg, INSPECT_OPTIONS)).join(' ') + const shim = Object.create(null) as Record<(typeof CONSOLE_LEVELS)[number], (...args: unknown[]) => void> + for (const level of CONSOLE_LEVELS) { + shim[level] = (...args: unknown[]) => { logs.push({ source: 'console', level, text: render(args) }) } + } + return shim +} + +/** + * Redirect a stream's `write` into the log buffer (the program-visible + * `process.stdout`/`process.stderr` in the real worker), so raw writes land + * in emission order alongside console output instead of racing down a pipe. + * The shim keeps Node's `write(chunk[, encoding][, callback])` contract: the + * callback fires asynchronously once the chunk is admitted (a program + * awaiting flush completion must complete, not sit until the wall timeout), + * even for writes the exhausted budget drops. + * @param logs - the buffer captured writes are pushed into. + * @param stream - the stream whose `write` slot is patched. + * @param source - the log source the captured writes are attributed to. + * @returns the restore function (the in-process tests un-patch; the real + * worker never needs to). + */ +export function captureStreamWrites(logs: LogBuffer, stream: PatchableStream, source: 'stdout' | 'stderr'): () => void { + // The slot's VALUE is stored for restore and reassigned — never invoked + // detached, so the unbound-method concern does not apply. + // eslint-disable-next-line @typescript-eslint/unbound-method + const original = stream.write + stream.write = (chunk: unknown, ...rest: unknown[]): boolean => { + logs.push({ source, text: typeof chunk === 'string' ? chunk : String(chunk) }) + // Node's optional-encoding shape: the callback is whichever of the next + // two positions holds a function (a non-function there is the encoding). + const callback = [rest[0], rest[1]].find( + (arg): arg is (error?: Error | null) => void => typeof arg === 'function', + ) + if (callback) queueMicrotask(() => { callback(null) }) + return true + } + return () => { stream.write = original } +} + +/** Bounded inspect options: deep enough to be useful, bounded so a pathological value cannot explode the rendering. */ +const INSPECT_OPTIONS = { depth: 4, maxArrayLength: 100, maxStringLength: 10_000 } as const + +/** + * The longest prefix of `text` whose UTF-8 encoding fits `maxBytes`, cut at + * a code-point boundary (never mid-surrogate-pair). The byte caps are BYTE + * caps — `String.prototype.slice` counts UTF-16 code units, up to 3× smaller + * than what a multibyte string actually costs across the boundary. + * @param text - the string to bound. + * @param maxBytes - the UTF-8 byte budget the prefix must fit. + * @returns the prefix (all of `text` when it already fits). + */ +export function truncateUtf8Bytes(text: string, maxBytes: number): string { + if (Buffer.byteLength(text, 'utf8') <= maxBytes) return text + let bytes = 0 + let end = 0 + for (const char of text) { + const cost = Buffer.byteLength(char, 'utf8') + if (bytes + cost > maxBytes) break + bytes += cost + end += char.length + } + return text.slice(0, end) +} + +/** + * Prepare the program's completion value for the done message: a value whose + * MEASURED cross-boundary size fits `maxValueBytes` crosses raw — exact + * bytes for a string, the structured-clone wire size (`v8.serialize`) for + * everything else, so a huge container whose BOUNDED inspect rendering + * happens to be small cannot smuggle itself past the cap. Anything else + * (non-cloneable, or oversized) is REPLACED by its bounded `util.inspect` + * rendering, byte-truncated ({@link truncateUtf8Bytes}) with an in-band + * marker — the seam contract's "a non-transferable value is replaced by a + * string rendering", extended to oversized ones so a huge return cannot + * flood the host. + * @param value - the program's completion value. + * @param maxValueBytes - the byte cap for the value. + * @returns the done-message fragment: `{}` for `undefined`, else `{ value }`. + */ +export function prepareValue(value: unknown, maxValueBytes: number): { value?: unknown } { + if (value === undefined) return {} + if (typeof value === 'string') { + if (Buffer.byteLength(value, 'utf8') <= maxValueBytes) return { value } + } else { + let size: number | undefined + try { + size = serialize(value).byteLength + } catch { + // Only the verdict matters: the value has parts the structured-clone + // algorithm rejects (functions, classes, …) and must cross as its + // rendering instead. + size = undefined + } + if (size !== undefined && size <= maxValueBytes) return { value } + } + const rendered = typeof value === 'string' ? value : inspect(value, INSPECT_OPTIONS) + const capped = Buffer.byteLength(rendered, 'utf8') > maxValueBytes + ? `${truncateUtf8Bytes(rendered, maxValueBytes)}… [truncated]` + : rendered + return { value: capped } +} + +/** One awaited binding call's settlement handles, keyed by call id in the pending map. */ +export interface PendingCall { + resolve(value: unknown): void + reject(error: Error): void +} + +/** + * Route host replies into the pending-call map: each reply settles its call + * at most once, and a reply for an unknown id (stray, or a duplicate answer + * to an id already settled) is ignored. Shared wiring between + * {@link runWorkerMain} and the tests that exercise {@link makeNamespaces} + * standalone. + * @param port - the port whose `message` events carry the replies. + * @param pending - the id-keyed map of unsettled binding calls. + */ +export function wireReplies(port: BootstrapPort, pending: Map): void { + port.on('message', (message: ReplyMessage) => { + const entry = pending.get(message.id) + if (!entry) return + pending.delete(message.id) + if (message.ok) entry.resolve(message.value) + else entry.reject(new Error(message.message)) + }) +} + +/** + * Build the binding namespace objects the program sees: one null-prototype + * global per namespace, each declared name an own enumerable async function + * that bridges over the port (`__proto__`/`constructor`/`toString` are + * ordinary keys, never prototype collisions). A non-cloneable argument + * rejects that one call with a descriptive error; the host's reply (`ok` + * false) rejects it likewise, so a failed tool call surfaces in the program + * as an ordinary promise rejection. + * @param data - the boot payload's namespace declarations (globals + names). + * @param port - the port binding calls are posted to. + * @param pending - the id-keyed map each posted call parks its handles in. + * @param nextId - the shared mutable id counter (worker-issued correlation ids). + * @returns one namespace object per declaration, in declaration order. + */ +export function makeNamespaces( + data: Pick, + port: BootstrapPort, + pending: Map, + nextId: { value: number }, +): Record[] { + return data.namespaces.map(({ global, names }) => { + const namespace = Object.create(null) as Record + for (const name of names) { + Object.defineProperty(namespace, name, { + enumerable: true, + value: (args: unknown): Promise => new Promise((resolve, reject) => { + const id = nextId.value++ + pending.set(id, { resolve, reject }) + try { + port.postMessage({ type: 'call', id, global, name, args }) + } catch (error: unknown) { + pending.delete(id) + reject(new Error(`binding arguments must be structured-cloneable: ${error instanceof Error ? error.message : String(error)}`)) + } + }), + }) + } + return namespace + }) +} + +/** + * Run one program to settlement and post the {@link DoneMessage}: wires the + * reply handler, materializes the namespaces and console shim, compiles the + * type-stripped body as an async function (top-level `await`/`return` + * work), and reports a thrown program error as the done message's `error` + * field. Exactly one done message is ever posted. + * @param port - the message port to the host (the real `parentPort`, or the tests' fake). + * @param data - the boot payload the host sent. + * @param streams - the stream objects whose `write` is captured (the real + * `process.stdout`/`process.stderr` in the worker; fakes in tests). + * @returns resolves after the done message is posted (the tests await it; + * the real entry lets the worker exit naturally). + */ +export async function runWorkerMain( + port: BootstrapPort, + data: WorkerBootData, + streams: { stdout: PatchableStream; stderr: PatchableStream }, +): Promise { + const logs = new LogBuffer(data.maxLogBytes, (entry) => { port.postMessage({ type: 'log', entry }) }) + captureStreamWrites(logs, streams.stdout, 'stdout') + captureStreamWrites(logs, streams.stderr, 'stderr') + + const pending = new Map() + wireReplies(port, pending) + + const nextId = { value: 1 } + const namespaces = makeNamespaces(data, port, pending, nextId) + const consoleShim = makeConsoleShim(logs) + + let done: DoneMessage + try { + // The async function constructor, reached through an instance because + // `AsyncFunction` is not a global. The program body is strict-mode. + /* v8 ignore next -- the arrow exists only to reach the AsyncFunction constructor; it is never invoked. */ + const AsyncFunction = (async () => {}).constructor as new (...args: string[]) => (...fnArgs: unknown[]) => Promise + const fn = new AsyncFunction(...data.namespaces.map(namespace => namespace.global), 'console', `'use strict';\n${data.code}`) + const value = await fn(...namespaces, consoleShim) + done = { type: 'done', ...prepareValue(value, data.maxValueBytes) } + } catch (error: unknown) { + const message = error instanceof Error ? error.stack ?? error.message : String(error) + done = { type: 'done', error: { message } } + } + port.postMessage(done) +} diff --git a/packages/code-runtime/code-runtime-worker/src/index.ts b/packages/code-runtime/code-runtime-worker/src/index.ts new file mode 100644 index 0000000000..f78f06cb0b --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/src/index.ts @@ -0,0 +1,441 @@ +/** + * Worker-thread implementation of the code-execution seam: one fresh Node + * worker per run, executing the model's TypeScript after a host-side + * type-strip, with bindings bridged over the message port. Containment, not + * a security boundary (bash-equivalent trust — see the Code Mode RFC's + * trust-posture section): the worker gets an EMPTY environment, a heap cap, + * and two independent budgets — `computeMs` metered on the worker's + * measured event-loop busy time (a hot loop cannot hide behind a pending + * binding call) and a never-pausing `maxWallMs` ceiling — all funneling + * into `worker.terminate()`, which ends hot synchronous loops too. + * + * @module @deepseek-ai/dsh-code-runtime-worker + */ + +import { Worker } from 'node:worker_threads' +import { stripTypeScriptTypes } from 'node:module' +import { Context } from 'cordis' +import z from 'schemastery' +import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime' +import type { CodeBindingFunction, CodeLogEntry, CodeRunFailure, CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime' +import { prepareValue, truncateUtf8Bytes } from './bootstrap.ts' +import { logTruncationMarker } from './protocol.ts' +import type { ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts' + +export type { BootstrapPort, PatchableStream } from './bootstrap.ts' +export type { CallMessage, DoneMessage, ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts' + +/** Plugin config: every execution cap, changeable from `cordis.yml` (no hardcoded tunables). */ +export interface Config { + /** + * Busy-time budget in milliseconds: the run fails with kind `'timeout'` + * once the worker's MEASURED event-loop active time + * (`worker.performance.eventLoopUtilization()`) exceeds this. Metering + * measured busy time — not wall time, not host-side pending-call + * bookkeeping — is what makes the budget both fair (a program awaiting a + * slow tool accrues nothing) and ungameable (a hot loop accrues whether + * or not a decoy dispatch is in flight). + */ + computeMs?: number + /** + * Wall-clock ceiling in milliseconds; never pauses for anything. The + * backstop for what busy-time cannot see (a program awaiting a promise + * nobody will resolve). + */ + maxWallMs?: number + /** Shared byte budget for captured log text (console + raw stream writes), truncation marked in-band. */ + maxLogBytes?: number + /** + * Byte cap for the completion value, measured by its real cross-boundary + * size (string bytes, or structured-clone wire size); an oversized or + * non-cloneable value crosses as a capped string rendering. + */ + maxValueBytes?: number + /** The worker's max old-generation heap in MiB (`resourceLimits`); overflow kills the worker, surfacing as kind `'worker-exit'`. */ + maxOldGenerationSizeMb?: number +} + +/** {@link Config} after schemastery fills the defaults (every field present). */ +type ResolvedConfig = Required + +/** + * How often the host samples the worker's event-loop utilization for the + * `computeMs` budget. An internal cadence, not config: the only effect of + * the interval is budget-expiry granularity (a run can overshoot by up to + * one interval), and nothing a deployment could tune here improves that + * without burning host CPU. + */ +const ELU_POLL_INTERVAL_MS = 25 + +/** ECMAScript reserved words that cannot be async-function parameter names — rejected as binding globals. */ +const RESERVED_WORDS = new Set([ + 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', + 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in', + 'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', + 'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package', + 'private', 'protected', 'public', 'arguments', 'eval', +]) + +/** Valid async-function parameter name (the binding global becomes one). */ +const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ + +/** + * The shell a program is wrapped in for the type-strip, matching the + * grammatical context it will execute in (an async function body, where + * top-level `return` and `await` are legal — a bare module parse would + * reject the `return`). Strip mode is position-preserving (removed syntax + * becomes whitespace, nothing shifts), so the wrapper survives the strip + * byte-identical and the body slices back out with the model's own + * line/column positions intact. + */ +const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const + +/** One in-flight run's host-side state, tracked for disposal. */ +interface LiveRun { + worker: Worker + settle(failure: CodeRunFailure): void + finished: Promise +} + +/** + * The worker entry module. Source runs unbuilt (`src/worker.ts`, loadable + * directly on this repo's Node range via native type stripping — the file + * is erasable-only with type-only relative imports); the built package + * ships it as a sibling bundle (`lib/worker.js`, its own tsdown entry). + * The URL *pathname*'s extension says which world this module is in — + * pathname, because dev-time module runners (vitest) may suffix + * `import.meta.url` with a query string; relative resolution drops it. + */ +/* v8 ignore next -- the './worker.js' arm is the built-lib world, unreachable unbuilt by construction; the built-lib e2e pins it. */ +const WORKER_URL = new URL(new URL(import.meta.url).pathname.endsWith('.ts') ? './worker.ts' : './worker.js', import.meta.url) + +/** Render an unknown thrown value as a message, `Error` or not. */ +function messageOf(error: unknown): string { + return error instanceof Error ? error.message : String(error) +} + +/** The log sources / console levels the seam vocabulary admits, as runtime sets for inbound-message validation. */ +const LOG_SOURCES = new Set(['console', 'stdout', 'stderr']) +const LOG_LEVELS = new Set(['log', 'info', 'warn', 'error', 'debug']) + +/** + * Runtime shape gate for inbound port traffic. The peer runs MODEL CODE and + * can post anything — `null`, primitives, objects with poisoned fields — so + * the compile-time `WorkerToHost` type means nothing here: everything is + * re-validated and REBUILT field by field (a forged extra field never rides + * along; a non-number call id can never be echoed into a reply). Junk returns + * `undefined` and is dropped — a throw in the host's `message` listener would + * crash the host process. + */ +function parseWorkerMessage(raw: unknown): WorkerToHost | undefined { + if (typeof raw !== 'object' || raw === null) return undefined + const m = raw as Record + switch (m.type) { + case 'call': { + if (typeof m.id !== 'number' || typeof m.global !== 'string' || typeof m.name !== 'string') return undefined + return { type: 'call', id: m.id, global: m.global, name: m.name, args: m.args } + } + case 'log': { + const entry = m.entry + if (typeof entry !== 'object' || entry === null) return undefined + const e = entry as Record + if (typeof e.text !== 'string') return undefined + if (typeof e.source !== 'string' || !LOG_SOURCES.has(e.source)) return undefined + if (e.level !== undefined && (typeof e.level !== 'string' || !LOG_LEVELS.has(e.level))) return undefined + return { + type: 'log', + entry: { + source: e.source as CodeLogEntry['source'], + ...e.level !== undefined ? { level: e.level as Exclude } : {}, + text: e.text, + }, + } + } + case 'done': { + if (m.error === undefined) return { type: 'done', ...m.value !== undefined ? { value: m.value } : {} } + const error = m.error + if (typeof error !== 'object' || error === null) return undefined + const message = (error as Record).message + if (typeof message !== 'string') return undefined + return { type: 'done', ...m.value !== undefined ? { value: m.value } : {}, error: { message } } + } + default: return undefined + } +} + +/** + * Headroom the host's value re-cap grants over `maxValueBytes`: exactly the + * truncation suffix {@link prepareValue} appends, so a value the WORKER + * already capped (byte-exact prefix + this marker) passes through unchanged + * instead of being marked twice. + */ +const VALUE_RENDER_SLACK = Buffer.byteLength('… [truncated]', 'utf8') + +/** + * The shipped {@link CodeRuntime} backend (`ctx.codeRuntime`). Registers as + * the `codeRuntime` service; every cap comes from validated config. See the + * module doc for the containment model and the class JSDoc on the seam for + * the contract this implements (error-as-field, hostile-peer port, + * no cross-run state, dispose to quiescence). + */ +export class WorkerCodeRuntime extends CodeRuntime { + static Config: z = z.object({ + computeMs: z.number().default(60_000), + maxWallMs: z.number().default(600_000), + maxLogBytes: z.number().default(65_536), + maxValueBytes: z.number().default(32_768), + maxOldGenerationSizeMb: z.number().default(512), + }) + + readonly language = 'typescript' + readonly isolation = 'worker-thread' + + private readonly config: ResolvedConfig + private readonly live = new Set() + private disposed = false + + constructor(ctx: Context, config: Config) { + super(ctx) + // Schemastery filled the defaults; the cast records that. Positivity is a + // semantic check the schema's plain number type does not carry. + this.config = config as ResolvedConfig + for (const [key, value] of Object.entries(this.config)) { + if (!(Number.isFinite(value) && value > 0)) throw new Error(`dsh-code-runtime-worker: config.${key} must be a positive number, got ${String(value)}`) + } + ctx.effect(() => () => this.teardown(), 'worker code-runtime teardown') + } + + /** + * Dispose to quiescence: mark the service unusable, fail every in-flight + * run as aborted, and AWAIT each worker's exit so no worker outlives the + * fiber. + */ + private async teardown(): Promise { + this.disposed = true + const runs = [...this.live] + for (const run of runs) run.settle({ kind: 'abort', message: 'runtime disposed' }) + await Promise.all(runs.map(run => run.finished)) + } + + /** + * Execute one program in a fresh worker. Program outcomes — including a + * type-strip syntax error, which never spawns a worker — resolve with + * `result.error`; the method rejects only for seam misuse (a disposed + * runtime, an invalid binding namespace). + * @param request - the program, its bindings, and the abort signal. + * @returns the run's outcome per the seam contract. + */ + async run(request: CodeRunRequest): Promise { + if (this.disposed) throw new Error('dsh-code-runtime-worker: run() after disposal') + const bindings = this.validateBindings(request) + if (request.signal?.aborted) { + return { logs: [], error: { kind: 'abort', message: String(request.signal.reason) } } + } + + let code: string + try { + const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + request.program + STRIP_WRAP.suffix) + code = stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length) + } catch (error: unknown) { + // A program that does not survive the type-strip (syntax error, + // non-erasable syntax like `enum`) is a program failure, reported the + // same way a thrown exception would be — and no worker ever spawns. + return { logs: [], error: { kind: 'exception', message: messageOf(error) } } + } + + return await this.execute(request, code, bindings) + } + + /** Reject (seam misuse) malformed binding namespaces: non-identifier or reserved globals, duplicates, and the `console` collision. */ + private validateBindings(request: CodeRunRequest): Map> { + const bindings = new Map>() + for (const namespace of request.bindings) { + if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) { + throw new Error(`dsh-code-runtime-worker: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`) + } + if (namespace.global === 'console' || bindings.has(namespace.global)) { + throw new Error(`dsh-code-runtime-worker: duplicate binding global ${JSON.stringify(namespace.global)}`) + } + bindings.set(namespace.global, namespace.functions) + } + return bindings + } + + /** Spawn the worker for one validated, type-stripped run and drive it to settlement. */ + private execute( + request: CodeRunRequest, + code: string, + bindings: Map>, + ): Promise { + const bootData: WorkerBootData = { + code, + namespaces: [...bindings].map(([global, functions]) => ({ global, names: Object.keys(functions) })), + maxLogBytes: this.config.maxLogBytes, + maxValueBytes: this.config.maxValueBytes, + } + const worker = new Worker(WORKER_URL, { + workerData: bootData, + // Model code gets NO ambient environment — stronger than the scrubbed + // env the defensive-patterns rule requires for spawned commands. + env: {}, + // Hermetic flags too: without this the worker inherits the host + // process's execArgv (a test runner's or tsx's loader hooks), which a + // bare isolate with an empty environment cannot satisfy. The entry + // needs nothing beyond native type stripping, on this repo's whole + // Node range. + execArgv: [], + resourceLimits: { maxOldGenerationSizeMb: this.config.maxOldGenerationSizeMb }, + // Backstop capture: the bootstrap patches JS-level writes into its own + // ordered buffer, so these pipes normally stay silent; anything that + // still arrives (native-level writes) is appended after the done logs. + stdout: true, + stderr: true, + }) + + return new Promise((resolve) => { + let settled = false + const answered = new Set() + const logs: CodeLogEntry[] = [] + const strayLogs: CodeLogEntry[] = [] + + // ONE host-side ledger for everything that lands in `logs`/`strayLogs`, + // whatever the path: honest port entries, FORGED port entries (model + // code posting `log` messages directly, bypassing the worker-side + // LogBuffer), and stray pipe bytes. On the first overflow it emits the + // same in-band marker the worker's LogBuffer would and drops the rest, + // so the documented cap is one shared `maxLogBytes` however it is hit. + let logBudget = this.config.maxLogBytes + let logsTruncated = false + const admit = (entry: CodeLogEntry, sink: CodeLogEntry[]): void => { + if (logsTruncated) return + const cost = Buffer.byteLength(entry.text, 'utf8') + if (cost > logBudget) { + logsTruncated = true + sink.push({ source: 'stderr', text: logTruncationMarker(this.config.maxLogBytes) }) + return + } + logBudget -= cost + sink.push(entry) + } + + // No settled guard: `finish` snapshots the arrays when it resolves, so + // a chunk flushing after settlement mutates only the discarded buffers, + // and the ledger bounds that growth until the pipes close. + const captureStray = (source: 'stdout' | 'stderr') => (chunk: Buffer) => { + admit({ source, text: chunk.toString('utf8') }, strayLogs) + } + worker.stdout.on('data', captureStray('stdout')) + worker.stderr.on('data', captureStray('stderr')) + + // Settlement: exactly one outcome wins; every path funnels through + // here, cleans up the timers/listeners, terminates the worker, and + // resolves only after the worker actually exited (quiescence). Logs + // streamed eagerly before the settlement are kept — a timed-out or + // killed program still shows the model what it printed. + let finishResolve!: () => void + const finished = new Promise((done) => { finishResolve = done }) + const finish = (result: Omit): void => { + if (settled) return + settled = true + clearInterval(eluTimer) + clearTimeout(wallTimer) + request.signal?.removeEventListener('abort', onAbort) + this.live.delete(live) + void worker.terminate().then(() => { + finishResolve() + resolve({ ...result, logs: [...logs, ...strayLogs] }) + }) + } + + const onDone = (message: WorkerToHost): void => { + if (message.type !== 'done') return + // Re-cap the completion value HOST-side: the honest path already + // capped it in the worker (prepareValue there), but a forged done + // message bypasses the bootstrap entirely — without this, model code + // could flood the host past maxValueBytes. Honest values pass + // unchanged (see VALUE_RENDER_SLACK); the error text is bounded too. + finish({ + ...prepareValue(message.value, this.config.maxValueBytes + VALUE_RENDER_SLACK), + ...message.error ? { error: { kind: 'exception' as const, message: truncateUtf8Bytes(message.error.message, this.config.maxValueBytes) } } : {}, + }) + } + + const onCall = (message: WorkerToHost): void => { + if (message.type !== 'call' || settled) return + // Hostile-peer rules: a duplicate id is ignored, an unknown name is + // answered with a failure, and a binding throw/reject becomes the + // program-side rejection — contained here, never a host crash. + if (answered.has(message.id)) return + answered.add(message.id) + const reply = (payload: ReplyMessage): void => { + if (settled) return + try { + worker.postMessage(payload) + } catch { + // The reply value failed structured clone; renegotiate as an error + // reply, which is always clone-plain. Nothing else throws here. + worker.postMessage({ type: 'reply', id: message.id, ok: false, message: 'binding resolution is not structured-cloneable' }) + } + } + const record = bindings.get(message.global) + // Own-property lookup only: a forged name like 'constructor' or + // 'hasOwnProperty' must not walk the record's prototype chain and + // reach a callable the consumer never declared. + const fn = record && Object.hasOwn(record, message.name) ? record[message.name] : undefined + if (typeof fn !== 'function') { + reply({ type: 'reply', id: message.id, ok: false, message: `unknown binding ${JSON.stringify(`${message.global}.${message.name}`)}` }) + return + } + void (async () => { + try { + reply({ type: 'reply', id: message.id, ok: true, value: await fn(message.args) }) + } catch (error: unknown) { + reply({ type: 'reply', id: message.id, ok: false, message: messageOf(error) }) + } + })() + } + + worker.on('message', (raw: unknown) => { + // Parse before touching: the peer can post ANY shape, and a throw in + // this listener would crash the host process. Junk drops silently. + const message = parseWorkerMessage(raw) + if (!message) return + if (message.type === 'log' && !settled) admit(message.entry, logs) + onCall(message) + onDone(message) + }) + worker.on('error', (error: Error) => { + finish({ error: { kind: 'worker-exit', message: `worker error: ${error.message}` } }) + }) + worker.on('exit', (exitCode: number) => { + finish({ error: { kind: 'worker-exit', message: `worker exited with code ${exitCode} before completing` } }) + }) + + // The compute budget reads the worker's own measured busy time, so a + // hot loop expires it no matter what dispatches are in flight, while a + // program idling on a slow binding accrues nothing. + const eluTimer = setInterval(() => { + const elu = worker.performance.eventLoopUtilization() + if (elu.active > this.config.computeMs) { + finish({ error: { kind: 'timeout', message: `compute budget exhausted (${this.config.computeMs}ms busy)` } }) + } + }, ELU_POLL_INTERVAL_MS) + const wallTimer = setTimeout(() => { + finish({ error: { kind: 'timeout', message: `wall-clock ceiling reached (${this.config.maxWallMs}ms)` } }) + }, this.config.maxWallMs) + const onAbort = (): void => { + finish({ error: { kind: 'abort', message: String(request.signal?.reason) } }) + } + request.signal?.addEventListener('abort', onAbort, { once: true }) + + const live: LiveRun = { + worker, + finished, + settle: (failure: CodeRunFailure) => { finish({ error: failure }) }, + } + this.live.add(live) + }) + } +} + +export default WorkerCodeRuntime diff --git a/packages/code-runtime/code-runtime-worker/src/protocol.ts b/packages/code-runtime/code-runtime-worker/src/protocol.ts new file mode 100644 index 0000000000..b8ea122c5b --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/src/protocol.ts @@ -0,0 +1,78 @@ +/** + * Wire protocol between the host runtime and the worker bootstrap. Everything + * crossing the message port is structured-clone-plain and versionless — both + * ends ship in this package, always at the same version. The host treats + * inbound traffic as HOSTILE (the worker runs model code, which can reach + * `parentPort` via `import('node:worker_threads')` and forge any of these + * shapes); the worker treats inbound traffic as trusted. + * + * @module @deepseek-ai/dsh-code-runtime-worker/src/protocol + */ + +import type { CodeLogEntry } from '@deepseek-ai/dsh-code-runtime' + +/** What the host hands the worker at spawn, via `workerData`. */ +export interface WorkerBootData { + /** The type-stripped (plain JS) program body. */ + code: string + /** Binding namespaces to materialize: the global name plus the function names (functions themselves stay host-side). */ + namespaces: { global: string; names: string[] }[] + /** Shared byte budget for captured log text; exceeding it drops further entries after one in-band marker. */ + maxLogBytes: number + /** Byte cap for the rendered completion value (see the value-preparation contract in bootstrap.ts). */ + maxValueBytes: number +} + +/** Worker → host: one bridged binding call. */ +export interface CallMessage { + type: 'call' + /** Worker-issued correlation id; the host answers each id at most once and ignores duplicates. */ + id: number + /** The namespace global the call targets. */ + global: string + /** The function name within the namespace. */ + name: string + /** The single argument, structured-clone-plain. */ + args: unknown +} + +/** Worker → host: one captured log entry, streamed eagerly so output survives a mid-run termination (timeout, abort, OOM). */ +export interface LogMessage { + type: 'log' + entry: CodeLogEntry +} + +/** + * Worker → host: the program settled. `error` carries a program exception + * (the only failure the bootstrap itself can report — budgets, aborts, and + * substrate death are observed host-side). `value` is present only on a + * clean completion that produced one (already size-capped and + * clone-safe per the bootstrap's value preparation). Logs are NOT carried + * here — they streamed eagerly as {@link LogMessage}s. + */ +export interface DoneMessage { + type: 'done' + value?: unknown + error?: { message: string } +} + +/** Every message the worker sends. */ +export type WorkerToHost = CallMessage | LogMessage | DoneMessage + +/** Host → worker: the answer to one {@link CallMessage}. */ +export type ReplyMessage = + | { type: 'reply'; id: number; ok: true; value: unknown } + | { type: 'reply'; id: number; ok: false; message: string } + +/** + * The in-band marker entry text announcing that log capture stopped at the + * byte budget. Shared wire vocabulary: the worker's LogBuffer emits it when + * ITS budget exhausts, and the host emits the identical text when its own + * ledger drops an entry first (forged port traffic, stray pipe bytes) — so + * a truncated run reads the same however the cap was hit. + * @param maxBytes - the configured `maxLogBytes` the marker names. + * @returns the marker line. + */ +export function logTruncationMarker(maxBytes: number): string { + return `[dsh-code-runtime-worker] log capture truncated at ${maxBytes} bytes` +} diff --git a/packages/code-runtime/code-runtime-worker/src/worker.ts b/packages/code-runtime/code-runtime-worker/src/worker.ts new file mode 100644 index 0000000000..efaafdb038 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/src/worker.ts @@ -0,0 +1,20 @@ +/** + * The worker-thread entrypoint: self-executing glue over + * `bootstrap.ts`'s {@link runWorkerMain}, kept to the spawn wiring alone. + * Like `bin.ts` CLI entrypoints, this file executes only inside a spawned + * worker isolate — a place the coverage provider cannot observe — so it is + * excluded from the coverage gate while every line of actual logic lives in + * `bootstrap.ts`, unit-tested in-process; the real spawn path is pinned by + * the integration tests that run genuine workers. + * + * @module @deepseek-ai/dsh-code-runtime-worker/src/worker + */ + +import { parentPort, workerData } from 'node:worker_threads' +import { runWorkerMain } from './bootstrap.ts' +import type { WorkerBootData } from './protocol.ts' + +// A worker always has a parent port; guard loudly rather than run detached. +if (!parentPort) throw new Error('dsh-code-runtime-worker: worker entry loaded outside a worker thread') + +await runWorkerMain(parentPort, workerData as WorkerBootData, { stdout: process.stdout, stderr: process.stderr }) diff --git a/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts b/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts new file mode 100644 index 0000000000..e41f4455bb --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts @@ -0,0 +1,273 @@ +import { describe, expect, it } from 'vitest' +import { EventEmitter } from 'node:events' +import { LogBuffer, makeConsoleShim, makeNamespaces, captureStreamWrites, prepareValue, runWorkerMain, truncateUtf8Bytes, wireReplies } from '@deepseek-ai/dsh-code-runtime-worker/src/bootstrap.ts' +import type { BootstrapPort, PatchableStream, PendingCall } from '@deepseek-ai/dsh-code-runtime-worker/src/bootstrap.ts' +import type { ReplyMessage, WorkerToHost } from '@deepseek-ai/dsh-code-runtime-worker/src/protocol.ts' +import type { CodeLogEntry } from '@deepseek-ai/dsh-code-runtime' + +/** + * An in-process stand-in for the worker's parentPort: the test plays the + * HOST side — inspect what the bootstrap posted, feed replies back — so + * every line of worker-side logic runs under coverage without spawning an + * isolate (real-worker behavior is pinned by runtime.spec.ts). + */ +class FakePort implements BootstrapPort { + sent: WorkerToHost[] = [] + private readonly emitter = new EventEmitter() + /** Host-scripted responder; return undefined to leave the call pending. */ + respond: (message: WorkerToHost) => ReplyMessage | undefined = () => undefined + + postMessage(message: WorkerToHost): void { + this.sent.push(message) + const reply = this.respond(message) + if (reply) queueMicrotask(() => this.emitter.emit('message', reply)) + } + + on(event: 'message', listener: (message: ReplyMessage) => void): void { + this.emitter.on(event, listener) + } + + deliver(message: ReplyMessage): void { + this.emitter.emit('message', message) + } + + logs(): CodeLogEntry[] { + return this.sent.filter(message => message.type === 'log').map(message => message.entry) + } + + done(): WorkerToHost | undefined { + return this.sent.find(message => message.type === 'done') + } +} + +function fakeStreams(): { stdout: PatchableStream; stderr: PatchableStream } { + return { stdout: { write: () => true }, stderr: { write: () => true } } +} + +const BOOT = { maxLogBytes: 65_536, maxValueBytes: 32_768 } + +describe('LogBuffer', () => { + it('streams entries to the sink until the byte budget, then emits one marker and drops the rest', () => { + const seen: CodeLogEntry[] = [] + const buffer = new LogBuffer(10, entry => seen.push(entry)) + buffer.push({ source: 'console', level: 'log', text: '12345' }) + buffer.push({ source: 'console', level: 'log', text: '123456' }) + buffer.push({ source: 'console', level: 'log', text: 'dropped' }) + expect(seen.map(entry => entry.text)).toEqual([ + '12345', + '[dsh-code-runtime-worker] log capture truncated at 10 bytes', + ]) + }) +}) + +describe('makeConsoleShim', () => { + it('captures the five levels and renders non-strings inspect-style', () => { + const seen: CodeLogEntry[] = [] + const shim = makeConsoleShim(new LogBuffer(1_000, entry => seen.push(entry))) + shim.log('plain', { a: 1 }) + shim.info('i') + shim.warn('w') + shim.error('e') + shim.debug('d') + expect(seen.map(entry => entry.level)).toEqual(['log', 'info', 'warn', 'error', 'debug']) + expect(seen[0]?.text).toBe('plain { a: 1 }') + expect(seen.every(entry => entry.source === 'console')).toBe(true) + }) +}) + +describe('captureStreamWrites', () => { + it('redirects writes into the buffer and restores on request', () => { + const seen: CodeLogEntry[] = [] + const buffer = new LogBuffer(1_000, entry => seen.push(entry)) + let underlying = '' + const stream: PatchableStream = { write: (chunk: unknown) => { underlying += String(chunk); return true } } + const restore = captureStreamWrites(buffer, stream, 'stdout') + stream.write('captured', 'utf8') + stream.write(Buffer.from('bytes')) + restore() + stream.write('after') + expect(seen.map(entry => entry.text)).toEqual(['captured', 'bytes']) + expect(seen[0]).toMatchObject({ source: 'stdout' }) + expect(underlying).toBe('after') + }) + + it('invokes the write callback asynchronously, in both optional-encoding shapes', async () => { + const buffer = new LogBuffer(1_000, () => {}) + const stream: PatchableStream = { write: () => true } + captureStreamWrites(buffer, stream, 'stdout') + const calls: (Error | null | undefined)[] = [] + stream.write('two-arg', (error?: Error | null) => calls.push(error)) + stream.write('three-arg', 'utf8', (error?: Error | null) => calls.push(error)) + // Node's contract: the callback fires after the write call returns. + expect(calls).toEqual([]) + await new Promise(resolve => stream.write('awaited flush', resolve)) + expect(calls).toEqual([null, null]) + }) + + it('still fires the callback for a write the exhausted budget drops', async () => { + const buffer = new LogBuffer(4, () => {}) + const stream: PatchableStream = { write: () => true } + captureStreamWrites(buffer, stream, 'stdout') + stream.write('this write overflows the budget and is dropped') + await new Promise(resolve => stream.write('also dropped', resolve)) + }) +}) + +describe('prepareValue', () => { + it('omits undefined, passes small cloneable values raw', () => { + expect(prepareValue(undefined, 100)).toEqual({}) + expect(prepareValue({ a: [1, 'two'] }, 100)).toEqual({ value: { a: [1, 'two'] } }) + }) + + it('replaces a non-cloneable value with its rendering', () => { + const { value } = prepareValue({ fn: () => 1 }, 1_000) + expect(typeof value).toBe('string') + expect(value).toContain('fn') + }) + + it('replaces an oversized value with a truncation-marked capped rendering', () => { + const { value } = prepareValue('x'.repeat(50), 10) + expect(value).toBe(`${'x'.repeat(10)}… [truncated]`) + }) + + it('measures a container by its structured-clone wire size, not its bounded rendering', () => { + // The bounded inspect rendering of a huge array is tiny ("... N more + // items"), but its real cross-boundary size is not — the cap must catch + // it, replacing the value with that bounded rendering. + const huge = new Array(50_000).fill(7) + const { value } = prepareValue(huge, 1_000) + expect(typeof value).toBe('string') + expect(value).toContain('more items') + }) + + it('caps a multibyte string by UTF-8 bytes, not UTF-16 length', () => { + // 4 code units but 12 UTF-8 bytes: a length-counting cap would pass the + // full string through untruncated. + expect(prepareValue('€€€€', 4)).toEqual({ value: '€… [truncated]' }) + }) + + it('caps a multibyte rendering by UTF-8 bytes too', () => { + // Wire size (24-byte string inside an array) exceeds the cap, so the + // value crosses as its rendering — whose truncation must also be + // byte-exact: "[ '" (3 bytes) + two € (6 bytes) = 9; a third € would + // overflow the 10-byte budget. + expect(prepareValue(['€€€€€€€€'], 10)).toEqual({ value: "[ '€€… [truncated]" }) + }) +}) + +describe('truncateUtf8Bytes', () => { + it('returns a fitting string whole', () => { + expect(truncateUtf8Bytes('fits', 4)).toBe('fits') + }) + + it('cuts at a code-point boundary, never mid-surrogate-pair', () => { + // Each 😀 is one code point, two code units, four UTF-8 bytes: a 5-byte + // budget fits exactly one — and never leaves a lone surrogate behind. + const cut = truncateUtf8Bytes('😀😀', 5) + expect(cut).toBe('😀') + expect(Buffer.byteLength(truncateUtf8Bytes('😀😀', 3), 'utf8')).toBe(0) + }) +}) + +describe('makeNamespaces', () => { + it('exposes prototype-colliding names as ordinary own properties', async () => { + const port = new FakePort() + port.respond = message => message.type === 'call' ? { type: 'reply', id: message.id, ok: true, value: `${message.name}-ok` } : undefined + const pending = new Map() + wireReplies(port, pending) + const [tools] = makeNamespaces({ namespaces: [{ global: 'tools', names: ['__proto__', 'constructor', 'toString'] }] }, port, pending, { value: 1 }) as [Record Promise>] + expect(Object.getPrototypeOf(tools)).toBeNull() + await expect(tools['__proto__']?.({})).resolves.toBe('__proto__-ok') + await expect(tools['constructor']?.({})).resolves.toBe('constructor-ok') + await expect(tools['toString']?.({})).resolves.toBe('toString-ok') + }) + + it('rejects a non-cloneable argument without leaking the pending entry', async () => { + let firstCall = true + const throwingPort: BootstrapPort = { + // First call throws an Error (the real DataCloneError shape), the + // second a bare string — the rejection renders both. + postMessage: () => { + if (firstCall) { firstCall = false; throw new Error('DataCloneError-ish') } + throw 'raw-clone-failure' + }, + on: () => {}, + } + const pending = new Map() + const [tools] = makeNamespaces({ namespaces: [{ global: 'tools', names: ['x'] }] }, throwingPort, pending, { value: 1 }) as [Record Promise>] + await expect(tools.x?.(() => 1)).rejects.toThrow(/structured-cloneable: DataCloneError-ish/) + await expect(tools.x?.(() => 1)).rejects.toThrow(/structured-cloneable: raw-clone-failure/) + expect(pending.size).toBe(0) + }) +}) + +describe('runWorkerMain', () => { + it('runs a program end-to-end: bindings, console, return value', async () => { + const port = new FakePort() + port.respond = message => message.type === 'call' ? { type: 'reply', id: message.id, ok: true, value: (message.args as { n: number }).n * 2 } : undefined + await runWorkerMain(port, { + ...BOOT, + code: 'const doubled = await tools.double({ n: 21 }); console.log("got", doubled); return { doubled };', + namespaces: [{ global: 'tools', names: ['double'] }], + }, fakeStreams()) + expect(port.logs()).toEqual([{ source: 'console', level: 'log', text: 'got 42' }]) + expect(port.done()).toEqual({ type: 'done', value: { doubled: 42 } }) + }) + + it('reports a thrown program error on the done message', async () => { + const port = new FakePort() + await runWorkerMain(port, { ...BOOT, code: 'throw new Error("boom")', namespaces: [] }, fakeStreams()) + const done = port.done() + expect(done?.type).toBe('done') + expect(done?.type === 'done' ? done.error?.message : undefined).toContain('boom') + expect(done?.type === 'done' ? done.value : undefined).toBeUndefined() + }) + + it('renders non-Error throws and stack-less Errors on the done message', async () => { + const rawPort = new FakePort() + await runWorkerMain(rawPort, { ...BOOT, code: 'throw "raw-throw"', namespaces: [] }, fakeStreams()) + expect(rawPort.done()).toEqual({ type: 'done', error: { message: 'raw-throw' } }) + + const barePort = new FakePort() + await runWorkerMain(barePort, { ...BOOT, code: 'const e = new Error("bare"); e.stack = undefined; throw e', namespaces: [] }, fakeStreams()) + expect(barePort.done()).toEqual({ type: 'done', error: { message: 'bare' } }) + }) + + it('surfaces a host failure reply as a program-side rejection it can catch', async () => { + const port = new FakePort() + port.respond = message => message.type === 'call' ? { type: 'reply', id: message.id, ok: false, message: 'denied by host' } : undefined + await runWorkerMain(port, { + ...BOOT, + code: 'try { await tools.x({}) } catch (error) { return `caught: ${error.message}` }', + namespaces: [{ global: 'tools', names: ['x'] }], + }, fakeStreams()) + expect(port.done()).toEqual({ type: 'done', value: 'caught: denied by host' }) + }) + + it('ignores replies for unknown pending ids', async () => { + const port = new FakePort() + port.respond = (message) => { + if (message.type !== 'call') return undefined + // Deliver a stray reply first; the real one follows. + port.deliver({ type: 'reply', id: 9_999, ok: true, value: 'stray' }) + return { type: 'reply', id: message.id, ok: true, value: 'real' } + } + await runWorkerMain(port, { + ...BOOT, + code: 'return await tools.x({})', + namespaces: [{ global: 'tools', names: ['x'] }], + }, fakeStreams()) + expect(port.done()).toEqual({ type: 'done', value: 'real' }) + }) + + it('captures raw stream writes through the patched process streams', async () => { + const port = new FakePort() + const streams = fakeStreams() + await runWorkerMain(port, { ...BOOT, code: 'return 1', namespaces: [] }, streams) + streams.stdout.write('never seen — already restored? no: patch persists in worker') + // The patch stays installed for the worker's lifetime; writes during the + // program landed in order. Here the program wrote nothing via streams, so + // only the post-run write above went through the patched slot. + expect(port.logs().at(-1)).toMatchObject({ source: 'stdout' }) + }) +}) diff --git a/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts b/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts new file mode 100644 index 0000000000..66ce1830b6 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts @@ -0,0 +1,55 @@ +import { spawn } from 'node:child_process' +import { existsSync } from 'node:fs' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { describe, expect, it } from 'vitest' + +/** + * BUILT-ARTIFACT smoke for the published package (the real-load-path guard + * from docs/testing.md): the unit suite runs `src/` under vitest, where the + * worker entry resolves to `src/worker.ts` — a consumer runs `lib/index.js` + * under plain `node`, where it must resolve the sibling `lib/worker.js` + * bundle instead. This spawns plain `node` (NOT tsx) from inside the package + * directory and imports the package BY NAME, so resolution flows through the + * real `exports` map exactly as it would from a downstream install; the + * program exercises the type-strip, the worker spawn, the binding bridge, + * and log capture end-to-end through the built bundles. + * + * It build-gates: SKIPS when the built artifacts are absent (suite run + * without `pnpm run build`); CI runs it after the build step. KEYLESS — no + * model is involved. + */ + +const pkgDir = fileURLToPath(new URL('..', import.meta.url)) +const built = ['lib/index.js', 'lib/worker.js'].every(file => existsSync(join(pkgDir, file))) + && existsSync(join(pkgDir, '../code-runtime/lib/index.js')) + +describe.skipIf(!built)('built lib real load path (plain node)', () => { + it('runs a TypeScript program with a binding through lib/index.js and its lib/worker.js entry', async () => { + const script = ` + const { Context } = await import('cordis') + const { WorkerCodeRuntime } = await import('@deepseek-ai/dsh-code-runtime-worker') + const ctx = new Context() + await ctx.plugin(WorkerCodeRuntime, {}) + const result = await ctx.codeRuntime.run({ + program: 'const doubled: number = await tools.double({ n: 21 }); console.log("halfway", doubled); return doubled;', + bindings: [{ global: 'tools', functions: { double: async args => args.n * 2 } }], + }) + console.log(JSON.stringify(result)) + process.exit(0) + ` + const child = spawn(process.execPath, ['--input-type=module', '-e', script], { cwd: pkgDir, stdio: ['ignore', 'pipe', 'pipe'] }) + let stdout = '' + let stderr = '' + child.stdout.on('data', (chunk: Buffer) => { stdout += chunk.toString('utf8') }) + child.stderr.on('data', (chunk: Buffer) => { stderr += chunk.toString('utf8') }) + const exitCode = await new Promise(resolve => child.on('close', resolve)) + + expect(exitCode, `stderr:\n${stderr}`).toBe(0) + const lastLine = stdout.trim().split('\n').at(-1) ?? '' + const result = JSON.parse(lastLine) as { value?: unknown; logs: { source: string; level?: string; text: string }[]; error?: unknown } + expect(result.error).toBeUndefined() + expect(result.value).toBe(42) + expect(result.logs).toContainEqual({ source: 'console', level: 'log', text: 'halfway 42' }) + }) +}) diff --git a/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts b/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts new file mode 100644 index 0000000000..edc2bd1271 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts @@ -0,0 +1,451 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import { WorkerCodeRuntime } from '@deepseek-ai/dsh-code-runtime-worker' +import type { Config } from '@deepseek-ai/dsh-code-runtime-worker' +import type { CodeRunResult } from '@deepseek-ai/dsh-code-runtime' + +/** + * Integration suite over REAL worker threads (no mocks — workers are cheap + * and local, per docs/testing.md's real-over-mock policy). Each test builds + * a fresh context so budgets can be tuned per case. + */ +async function setup(config: Config = {}) { + const ctx = new Context() + await ctx.plugin(WorkerCodeRuntime, config) + const runtime = ctx.codeRuntime as WorkerCodeRuntime + return { ctx, runtime } +} + +/** Convenience: one namespace `tools` with the given functions. */ +function tools(functions: Record Promise>) { + return [{ global: 'tools', functions }] +} + +describe('WorkerCodeRuntime — programs and bindings (real workers)', () => { + it('registers with the seam descriptors', async () => { + const { runtime } = await setup() + expect(runtime.language).toBe('typescript') + expect(runtime.isolation).toBe('worker-thread') + }) + + it('runs TypeScript (erasable syntax), captures console/stdout in order, returns the value', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + interface Point { x: number; y: number } + const p: Point = { x: 1, y: 2 } as Point; + console.log('point', p); + process.stdout.write('raw-out\\n'); + console.warn('careful'); + return p.x + p.y; + `, + bindings: [], + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe(3) + expect(result.logs.map(entry => [entry.source, entry.level ?? null])).toEqual([ + ['console', 'log'], + ['stdout', null], + ['console', 'warn'], + ]) + expect(result.logs[0]?.text).toBe('point { x: 1, y: 2 }') + }) + + it('bridges binding calls both ways and rejects the program-side call on a host rejection', async () => { + const { runtime } = await setup() + const calls: unknown[] = [] + const result = await runtime.run({ + program: ` + const first = await tools.echo({ n: 1 }); + let caught = ''; + try { await tools.fail({}) } catch (error) { caught = error.message } + let caughtRaw = ''; + try { await tools.failRaw({}) } catch (error) { caughtRaw = error.message } + return { first, caught, caughtRaw }; + `, + bindings: tools({ + echo: async (args) => { calls.push(args); return { echoed: args } }, + fail: async () => { throw new Error('nope') }, + // A non-Error throw: the host renders it, the program still catches. + failRaw: async () => { throw 'raw-nope' }, + }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toEqual({ first: { echoed: { n: 1 } }, caught: 'nope', caughtRaw: 'raw-nope' }) + expect(calls).toEqual([{ n: 1 }]) + }) + + it('reports non-erasable syntax as an exception without spawning a worker', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'enum E { A }\nreturn 1', bindings: [] }) + expect(result.error?.kind).toBe('exception') + expect(result.error?.message).toMatch(/enum|strip/i) + }) + + it('reports a runtime throw as an exception with the message', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'throw new Error("kaboom")', bindings: [] }) + expect(result.error?.kind).toBe('exception') + expect(result.error?.message).toContain('kaboom') + }) + + it('gives the program an EMPTY environment', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return JSON.stringify(process.env)', bindings: [] }) + expect(result.value).toBe('{}') + }) + + it('replaces a non-cloneable return value with a string rendering', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return { f: () => 1 }', bindings: [] }) + expect(typeof result.value).toBe('string') + }) + + it('completes a program that returns nothing with no value at all', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'const x = 1', bindings: [] }) + expect(result.error).toBeUndefined() + expect('value' in result).toBe(false) + }) + + it('keeps logs streamed before a failure', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'console.log("before"); throw new Error("after-log")', + bindings: [], + }) + expect(result.error?.kind).toBe('exception') + expect(result.logs.map(entry => entry.text)).toContain('before') + }) +}) + +describe('WorkerCodeRuntime — budgets and containment (real workers)', () => { + it('ends a hot loop at the compute budget — including behind a pending decoy dispatch', async () => { + const { runtime } = await setup({ computeMs: 300, maxWallMs: 30_000 }) + const result = await runtime.run({ + // The decoy: fire a call at a never-resolving binding WITHOUT awaiting, + // then spin. Host-side pending-call bookkeeping would pause a naive + // budget here; measured busy time cannot be fooled. + program: 'void tools.slow({}); for (;;) {}', + bindings: tools({ slow: () => new Promise(() => {}) }), + }) + expect(result.error?.kind).toBe('timeout') + expect(result.error?.message).toContain('compute budget') + }, 15_000) + + it('does not charge time spent awaiting a slow binding against the compute budget', async () => { + const { runtime } = await setup({ computeMs: 250, maxWallMs: 30_000 }) + const result = await runtime.run({ + program: 'return await tools.slow({})', + bindings: tools({ slow: () => new Promise(resolve => setTimeout(() => { resolve('slow-done') }, 700)) }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('slow-done') + }, 15_000) + + it('ends an idle-forever run at the wall-clock ceiling', async () => { + const { runtime } = await setup({ computeMs: 30_000, maxWallMs: 400 }) + const result = await runtime.run({ + program: 'await tools.never({}); return 1', + bindings: tools({ never: () => new Promise(() => {}) }), + }) + expect(result.error?.kind).toBe('timeout') + expect(result.error?.message).toContain('wall-clock ceiling') + }, 15_000) + + it('reports an abort mid-run and stops the worker', async () => { + const { runtime } = await setup() + const controller = new AbortController() + setTimeout(() => { controller.abort('user-cancel') }, 150) + const result = await runtime.run({ program: 'for (;;) {}', bindings: [], signal: controller.signal }) + expect(result.error).toEqual({ kind: 'abort', message: 'user-cancel' }) + }, 15_000) + + it('reports a pre-aborted signal without spawning', async () => { + const { runtime } = await setup() + const controller = new AbortController() + controller.abort('too-late') + const result = await runtime.run({ program: 'return 1', bindings: [], signal: controller.signal }) + expect(result.error).toEqual({ kind: 'abort', message: 'too-late' }) + }) + + it('drops a binding resolution that lands after the run settled', async () => { + const { runtime } = await setup() + const controller = new AbortController() + let replyDelivered!: Promise + const result = await runtime.run({ + program: 'void tools.late({}); for (;;) {}', + bindings: tools({ + // Anchored on invocation: abort 100ms after the call reaches the + // host, resolve 400ms after — by then the run has settled, so the + // resolution's reply hits the post-settlement drop. + late: () => new Promise((resolve) => { + setTimeout(() => { controller.abort('cancel-now') }, 100) + replyDelivered = new Promise(done => setTimeout(() => { resolve('too-late'); done() }, 400)) + }), + }), + signal: controller.signal, + }) + expect(result.error).toEqual({ kind: 'abort', message: 'cancel-now' }) + // Let the late resolution actually fire so its reply executes instead of + // being cancelled with the test. + await replyDelivered + }, 15_000) + + it('contains an OOM under resourceLimits as worker-exit, host process healthy', async () => { + const { runtime } = await setup({ maxOldGenerationSizeMb: 32 }) + const result = await runtime.run({ + program: 'const hog = []; for (;;) hog.push(new Array(1e6).fill(1));', + bindings: [], + }) + expect(result.error?.kind).toBe('worker-exit') + // And the host is fine: run something else. + const after = await runtime.run({ program: 'return "alive"', bindings: [] }) + expect(after.value).toBe('alive') + }, 30_000) + + it('truncates runaway log output at the byte budget with an in-band marker', async () => { + const { runtime } = await setup({ maxLogBytes: 300 }) + const result = await runtime.run({ + program: 'for (let i = 0; i < 1000; i++) console.log("spam line", i); return 1', + bindings: [], + }) + expect(result.logs.at(-1)?.text).toContain('truncated at 300 bytes') + const total = result.logs.reduce((sum, entry) => sum + Buffer.byteLength(entry.text, 'utf8'), 0) + expect(total).toBeLessThan(1_000) + }) + + it('caps an oversized return value with a truncation marker', async () => { + const { runtime } = await setup({ maxValueBytes: 64 }) + const result = await runtime.run({ program: 'return "y".repeat(10_000)', bindings: [] }) + expect(result.value).toBe(`${'y'.repeat(64)}… [truncated]`) + }) + + it('caps a multibyte return value by UTF-8 bytes, not string length', async () => { + // 4 code units, 12 UTF-8 bytes: a length-counting cap would let the full + // string cross. The worker's byte-exact capped rendering then passes the + // host re-cap unchanged (cap + marker is exactly the granted slack). + const { runtime } = await setup({ maxValueBytes: 4 }) + const result = await runtime.run({ program: 'return "€€€€"', bindings: [] }) + expect(result.value).toBe('€… [truncated]') + }) + + it('completes a program that awaits its write callback, capturing the chunk', async () => { + // Node's write(chunk[, encoding][, callback]) contract: dropping the + // callback would leave this promise pending until the wall ceiling and + // misreport a completed program as a timeout. + const { runtime } = await setup({ maxWallMs: 2_000 }) + const result = await runtime.run({ + program: 'await new Promise(resolve => process.stdout.write("flushed", resolve)); return "done"', + bindings: [], + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('done') + expect(result.logs).toContainEqual({ source: 'stdout', text: 'flushed' }) + }) + + it('caps a huge container whose bounded rendering is small (wire size, not rendering, is what counts)', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return new Array(50_000).fill(7)', bindings: [] }) + expect(result.error).toBeUndefined() + expect(typeof result.value).toBe('string') + expect(result.value).toContain('more items') + }) + + it('captures pipe writes that bypass the patched write slot as stray logs, capped by the same budget', async () => { + const { runtime } = await setup({ maxLogBytes: 4 }) + const result = await runtime.run({ + // The bootstrap patches the stream instance's own `write`; going + // through the prototype's slot reaches the real pipe underneath, so + // the bytes arrive host-side as stray data. The pauses keep the two + // writes in separate pipe chunks and let them land before settlement. + program: ` + const write = (text) => Object.getPrototypeOf(process.stdout).write.call(process.stdout, text); + write('abcd'); + await new Promise(resolve => setTimeout(resolve, 150)); + write('ef'); + await new Promise(resolve => setTimeout(resolve, 100)); + return 1; + `, + bindings: [], + }) + expect(result.error).toBeUndefined() + expect(result.logs).toContainEqual({ source: 'stdout', text: 'abcd' }) + expect(result.logs.map(entry => entry.text)).not.toContain('ef') + }, 15_000) +}) + +describe('WorkerCodeRuntime — hostile programs (real workers)', () => { + it('survives forged port traffic: unknown binding names, duplicate ids, junk shapes', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} }); + parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} }); + parentPort.postMessage({ type: 'call', id: 7778, global: 'tools', name: 'constructor', args: {} }); + parentPort.postMessage({ type: 'junk' }); + return await tools.real({}); + `, + bindings: tools({ real: async () => 'still-works' }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('still-works') + }) + + it('survives arbitrary junk on the port: non-objects, junk types, malformed calls, logs, and dones', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + for (const junk of [ + null, 42, 'junk', [], + { type: 'nope' }, + { type: 'call' }, + { type: 'call', id: 'x', global: 'tools', name: 'real', args: {} }, + { type: 'call', id: 1e9, global: 7, name: 'real', args: {} }, + { type: 'call', id: 1e9, global: 'tools', name: 7, args: {} }, + { type: 'log' }, + { type: 'log', entry: null }, + { type: 'log', entry: { source: 'stdout', text: 7 } }, + { type: 'log', entry: { source: 'nope', text: 'x' } }, + { type: 'log', entry: { source: 'console', level: 'nope', text: 'x' } }, + { type: 'log', entry: { source: 'console', level: 7, text: 'x' } }, + { type: 'done', error: 5 }, + { type: 'done', error: { message: 5 } }, + ]) parentPort.postMessage(junk); + return await tools.real({}); + `, + bindings: tools({ real: async () => 'still-works' }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('still-works') + expect(result.logs).toEqual([]) + }) + + it('caps forged log floods and forged done values at the configured budgets, dropping forged extra fields', async () => { + const { runtime } = await setup({ maxLogBytes: 200, maxValueBytes: 64 }) + const result = await runtime.run({ + // Forged messages bypass the worker-side LogBuffer and prepareValue + // entirely — only the host-side ledger and re-cap stand between model + // code and an unbounded result. + program: ` + const { parentPort } = await import('node:worker_threads'); + for (let i = 0; i < 50; i++) parentPort.postMessage({ type: 'log', entry: { source: 'stdout', text: 'F'.repeat(100), forged: true } }); + parentPort.postMessage({ type: 'done', value: 'V'.repeat(100000) }); + for (;;) {} + `, + bindings: [], + }) + expect(typeof result.value).toBe('string') + const value = result.value as string + expect(value.startsWith('V'.repeat(64))).toBe(true) + expect(value.endsWith('… [truncated]')).toBe(true) + expect(value.length).toBeLessThan(120) + const marker = '[dsh-code-runtime-worker] log capture truncated at 200 bytes' + const total = result.logs.reduce((sum, entry) => sum + Buffer.byteLength(entry.text, 'utf8'), 0) + expect(total).toBeLessThanOrEqual(200 + Buffer.byteLength(marker, 'utf8')) + expect(result.logs.at(-1)?.text).toBe(marker) + expect(result.logs.every(entry => !('forged' in entry))).toBe(true) + }) + + it('accepts a forged done carrying both value and error (self-sabotage, contained)', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'done', value: 'lied', error: { message: 'fake failure' } }); + for (;;) {} + `, + bindings: [], + }) + expect(result.value).toBe('lied') + expect(result.error).toEqual({ kind: 'exception', message: 'fake failure' }) + }) + + it('byte-bounds forged multibyte error text at the host', async () => { + // Forged error text bypasses the worker entirely; the host bound is a + // BYTE bound (two € = 6 bytes fit an 8-byte cap, a third would not). + const { runtime } = await setup({ maxValueBytes: 8 }) + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'done', error: { message: '€'.repeat(1000) } }); + for (;;) {} + `, + bindings: [], + }) + expect(result.error).toEqual({ kind: 'exception', message: '€€' }) + }) + + it('answers a binding whose resolution cannot be cloned with a failure reply', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'try { await tools.bad({}) } catch (error) { return error.message }', + bindings: tools({ bad: async () => (() => 1) }), + }) + expect(result.value).toContain('not structured-cloneable') + }) + + it('exposes binding names that collide with Object.prototype as ordinary functions', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'return [await tools["__proto__"]({}), await tools["constructor"]({}), typeof tools["hasOwnProperty"]]', + // Computed keys: a literal `'__proto__': …` entry would SET the record's + // prototype instead of declaring a binding of that name. + bindings: tools({ ['__proto__']: async () => 'proto-ok', ['constructor']: async () => 'ctor-ok' }), + }) + expect(result.value).toEqual(['proto-ok', 'ctor-ok', 'undefined']) + }) +}) + +describe('WorkerCodeRuntime — seam misuse and lifecycle', () => { + it('rejects invalid binding globals loudly (identifier, reserved word, duplicate, console)', async () => { + const { runtime } = await setup() + const cases: [string, RegExp][] = [ + ['not valid!', /not a usable identifier/], + ['await', /not a usable identifier/], + ['console', /duplicate binding global/], + ] + for (const [global, message] of cases) { + await expect(runtime.run({ program: 'return 1', bindings: [{ global, functions: {} }] })).rejects.toThrow(message) + } + await expect(runtime.run({ + program: 'return 1', + bindings: [{ global: 'tools', functions: {} }, { global: 'tools', functions: {} }], + })).rejects.toThrow(/duplicate binding global/) + }) + + it('rejects config values that are not positive numbers', async () => { + const ctx = new Context() + await expect(ctx.plugin(WorkerCodeRuntime, { computeMs: -1 })).rejects.toThrow(/positive number/) + }) + + it('keeps runs isolated: no state survives from one run to the next', async () => { + const { runtime } = await setup() + await runtime.run({ program: 'globalThis.leak = "value"; return 1', bindings: [] }) + const second = await runtime.run({ program: 'return typeof globalThis.leak', bindings: [] }) + expect(second.value).toBe('undefined') + }) + + it('disposal aborts in-flight runs, awaits worker exit, and rejects later runs', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(WorkerCodeRuntime) + const runtime = ctx.codeRuntime as WorkerCodeRuntime + const inflight: Promise = runtime.run({ program: 'for (;;) {}', bindings: [] }) + // Give the worker a moment to actually start spinning. + await new Promise(resolve => setTimeout(resolve, 200)) + await fiber.dispose() + const result = await inflight + expect(result.error).toEqual({ kind: 'abort', message: 'runtime disposed' }) + await expect(runtime.run({ program: 'return 1', bindings: [] })).rejects.toThrow(/after disposal/) + }, 15_000) + + it('removes ctx.codeRuntime when the providing fiber disposes (HMR safety)', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(WorkerCodeRuntime) + expect(ctx.get('codeRuntime')).toBeInstanceOf(WorkerCodeRuntime) + await fiber.dispose() + expect(ctx.get('codeRuntime')).toBeUndefined() + }) +}) diff --git a/packages/code-runtime/code-runtime-worker/tsconfig.json b/packages/code-runtime/code-runtime-worker/tsconfig.json new file mode 100644 index 0000000000..af962eda4f --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../code-runtime" + } + ] +} diff --git a/packages/code-runtime/code-runtime-worker/tsdown.config.ts b/packages/code-runtime/code-runtime-worker/tsdown.config.ts new file mode 100644 index 0000000000..5af39d936a --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/tsdown.config.ts @@ -0,0 +1,35 @@ +import { defineConfig } from 'tsdown' + +/** + * Package-shape override (see the root tsdown.config.ts): besides the + * default lib/index.js bundle, the worker BOOTSTRAP ships as its own + * sibling entry — `new Worker(new URL('./worker.js', import.meta.url))` + * loads it as a file, so it cannot be part of the index bundle. TWO + * single-entry builds, not one two-entry build: a multi-entry build emits + * the shared bootstrap module as a `lib/bootstrap-*.js` chunk both bundles + * import, which the package.json `files` whitelist (deliberately exact) + * would omit from the packed artifact — each single-entry build inlines its + * own bootstrap copy instead, keeping every shipped file self-contained. + */ +export default defineConfig([ + { + entry: ['lib/types/index.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, + }, + { + entry: ['lib/types/worker.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, + }, +]) diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md new file mode 100644 index 0000000000..2d7b12add1 --- /dev/null +++ b/packages/code-runtime/code-runtime/README.md @@ -0,0 +1,19 @@ +# @deepseek-ai/dsh-code-runtime + +The **code-execution seam**: an abstract `CodeRuntime` service (`ctx.codeRuntime`) defining WHAT a code runtime does — run one model-written program against a set of host-provided async bindings and report `{ value, logs, error? }` — without saying HOW. + +This package is the interface third of the capability (the bash trio is the template — see [capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): implementations subclass `CodeRuntime` and register the service; the consumer is the tool registry's Code Mode, which generates the model-facing SDK and bridges tool dispatch — both specified in the [Code Mode RFC](../../../docs/rfc/proposed/feature/2026-06-15-code-mode.md), whose first implementation is a Node worker-thread backend. The runtime knows nothing about tools or sessions: it is handed named async functions and a program string, and everything tool-shaped stays with the consumer. + +## Service API (`ctx.codeRuntime`) + +| Member | Semantics | +|---|---| +| `run(request)` | Execute one program against the request's bindings. **Resolves with an error FIELD for every program outcome** — parse/transform failure, thrown exception, budget expiry, abort, substrate death (`CodeRunFailure`'s orthogonal `kind` taxonomy); it rejects only for caller misuse of the seam itself (e.g. a run submitted after disposal). The program runs as the body of an async function: top-level `await`/`return` work, and the completion value becomes `result.value` when it survives the serialization boundary. | +| `language` | Readonly descriptor: the source language `run` expects (`'typescript'` is the well-known value). Informational, not gating — a consumer that generates language-specific presentation switches on it and fails loud on a language it cannot present. | +| `isolation` | Readonly descriptor: the execution substrate (`'worker-thread'`, `'process'`, `'container'`). A label for deployments and diagnostics, **not a security claim**. | + +Semantics every implementation must honor (contract details in the class JSDoc): binding calls bridge to the caller's functions verbatim with structured-cloneable arguments/resolutions; the program is treated as a hostile peer (arbitrary binding names are own properties, malformed traffic never crashes the host); no state survives between runs; disposal terminates in-flight runs AND awaits their exit before completing. + +## Vocabulary + +`CodeRunRequest` (`program`, `bindings`, `signal?`) carries everything the runtime acts on — defaulting (time budgets, output caps) is the implementation's validated config, never a hidden `??` inside `run()`. `bindings` is a list of `CodeBindingNamespace`s (`global` + `functions`), each exposed to the program as one global object of async callables. `CodeRunResult` reports the completion `value?`, the ordered `logs` (`CodeLogEntry`: `console`/`stdout`/`stderr` source, console `level`, capped text), and the `error?` (`CodeRunFailure`: `kind` + model-feedable `message`). See `src/types.ts` for the full contracts. diff --git a/packages/code-runtime/code-runtime/package.json b/packages/code-runtime/code-runtime/package.json new file mode 100644 index 0000000000..0fe24bb15c --- /dev/null +++ b/packages/code-runtime/code-runtime/package.json @@ -0,0 +1,30 @@ +{ + "name": "@deepseek-ai/dsh-code-runtime", + "description": "Abstract code-execution seam (ctx.codeRuntime) for the DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/code-runtime/code-runtime/src/index.ts b/packages/code-runtime/code-runtime/src/index.ts new file mode 100644 index 0000000000..af967da61d --- /dev/null +++ b/packages/code-runtime/code-runtime/src/index.ts @@ -0,0 +1,93 @@ +/** + * The code-execution seam (`ctx.codeRuntime`): an abstract service defining + * WHAT a code runtime does — run one model-written program against a set of + * host-provided async bindings and report `{ value, logs, error? }` — without + * saying HOW. Implementations subclass {@link CodeRuntime} and register + * themselves as the `codeRuntime` service; backends may differ by execution + * substrate (worker thread, separate process, container) and by source + * language, both declared as readonly descriptors. The design and its + * consumer (the tool registry's Code Mode) are specified in the Code Mode RFC + * (docs/rfc/proposed/feature/2026-06-15-code-mode.md). + * + * The split mirrors the bash seam (`BashExecutor`): the runtime knows nothing + * about tools or sessions — it is handed named async functions and a program, + * and everything tool-shaped stays with the consumer. + * + * @module @deepseek-ai/dsh-code-runtime + */ + +import { Context, Service } from 'cordis' +import type { CodeRunRequest, CodeRunResult } from './types.ts' + +export type { + CodeBindingFunction, + CodeBindingNamespace, + CodeLogEntry, + CodeRunFailure, + CodeRunRequest, + CodeRunResult, +} from './types.ts' + +declare module 'cordis' { + interface Context { + codeRuntime: CodeRuntime + } +} + +/** + * Abstract code-execution service. Subclass, implement {@link run} and the + * two descriptors, and load the subclass as a plugin — it registers as + * `ctx.codeRuntime` (one implementation per context; loading a second throws, + * cordis' standard duplicate-service behavior). + * + * Semantics every implementation must honor: + * - {@link run} resolves with an error FIELD for every program outcome — + * parse/transform failures, thrown exceptions, budget expiry, abort, + * substrate death ({@link CodeRunFailure}'s taxonomy). It REJECTS only for + * caller misuse of the seam itself (e.g. a run submitted after disposal). + * - Binding calls bridge to the caller's {@link CodeBindingFunction}s + * verbatim; arguments and resolutions must be structured-cloneable, and the + * runtime treats the program as a hostile peer (arbitrary binding names are + * own properties, malformed traffic is rejected or ignored, never crashes + * the host). + * - Runs are isolated from each other: no state survives from one run to the + * next through the runtime. + * - Disposal reaches quiescence: in-flight runs are terminated AND awaited + * before the service's own teardown completes (no orphan substrate survives + * `fiber.dispose()`). + */ +export abstract class CodeRuntime extends Service { + /** + * The source language {@link run} expects `program` to be written in, as a + * lowercase identifier. Informational, not gating — a consumer that + * generates language-specific presentation (typed SDK stubs, usage + * instructions) switches on it and fails loud on a language it cannot + * present. Well-known value: `'typescript'`. + */ + abstract readonly language: string + + /** + * The execution substrate, as a lowercase identifier. Informational, not + * gating — a descriptor so deployments and diagnostics can tell backends + * apart, not a security claim. Well-known values: `'worker-thread'`, + * `'process'`, `'container'`. + */ + abstract readonly isolation: string + + constructor(ctx: Context) { + super(ctx, 'codeRuntime') + } + + /** + * Execute one program against the request's bindings and capture what it + * emitted. See the class doc for the resolution contract (error is a result + * field; rejection means seam misuse only). + * @param request - the program, its bindings, and the abort signal; the + * request carries everything the runtime acts on, with no hidden defaults. + * @returns the run's outcome: completion value (when transferable), the + * ordered log capture, and the failure (if any). + */ + abstract run(request: CodeRunRequest): Promise +} + +export default CodeRuntime diff --git a/packages/code-runtime/code-runtime/src/types.ts b/packages/code-runtime/code-runtime/src/types.ts new file mode 100644 index 0000000000..8278f33a39 --- /dev/null +++ b/packages/code-runtime/code-runtime/src/types.ts @@ -0,0 +1,105 @@ +/** + * Vocabulary types for the code-execution seam: what a caller hands a + * {@link ../index.ts | CodeRuntime} and what it gets back. Pure types — no + * runtime code lives here. + * + * @module @deepseek-ai/dsh-code-runtime/src/types + */ + +/** + * One host-side function exposed to the program as an async callable. The + * runtime bridges calls to it (possibly across a serialization boundary), so + * `args` and the resolution value MUST be structured-cloneable; a runtime + * rejects a non-cloneable value with a descriptive error rather than + * corrupting the run. A rejection of this function surfaces inside the + * program as a rejection of the corresponding call. + */ +export type CodeBindingFunction = (args: unknown) => Promise + +/** + * A named group of {@link CodeBindingFunction}s the runtime exposes to the + * program as one global object (e.g. `tools`). Function names are arbitrary + * strings — a runtime must treat names like `__proto__` or `constructor` as + * ordinary own properties (null-prototype construction), never as prototype + * collisions. + */ +export interface CodeBindingNamespace { + /** The global identifier the program sees (must be a valid JS identifier). */ + global: string + /** The callable members, keyed by the exact name the program calls. */ + functions: Record +} + +/** + * One run: the program source plus everything the runtime acts on. Per the + * explicit-over-implicit convention, defaulting (time budgets, output caps) + * is the implementation's validated config — a request carries no optional + * tuning knobs for a hidden `??` to fill in. + */ +export interface CodeRunRequest { + /** + * The program source, in the runtime's {@link ../index.ts | language}. It + * runs as the body of an async function: top-level `await` and `return` + * are available, and the completion value becomes + * {@link CodeRunResult.value}. + */ + program: string + /** Host functions exposed to the program, one global object per namespace. */ + bindings: CodeBindingNamespace[] + /** + * Abort the run: the runtime stops the program (hard, even mid-loop) and + * resolves with a {@link CodeRunFailure} of kind `'abort'`. In-flight + * binding calls are the CALLER's to settle — the runtime only stops asking. + */ + signal?: AbortSignal +} + +/** + * One captured output entry, in emission order. `source` says which channel + * produced it: the program's `console` (shimmed by the runtime), or a stray + * write to the underlying stdout/stderr streams. + */ +export interface CodeLogEntry { + /** Which channel produced the text. */ + source: 'console' | 'stdout' | 'stderr' + /** The console method used; present only when `source` is `'console'`. */ + level?: 'log' | 'info' | 'warn' | 'error' | 'debug' + /** The captured text (possibly truncated by the implementation's caps, marked in-band). */ + text: string +} + +/** + * Why a run failed. The kinds are orthogonal outcomes reported independently + * (per docs/defensive-patterns.md): a budget expiry is not an exception, an + * abort is not a timeout, and a substrate death is neither. + * + * - `'exception'` — the program threw or failed to parse/transform. + * - `'timeout'` — an implementation-owned budget expired; the message says which. + * - `'abort'` — {@link CodeRunRequest.signal} fired. + * - `'worker-exit'` — the execution substrate died without settling (e.g. OOM). + */ +export interface CodeRunFailure { + /** The failure class (see the interface doc for each kind's meaning). */ + kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' + /** Human-readable detail, suitable for feeding back to a model to self-correct. */ + message: string +} + +/** + * The outcome of one run. An error is a FIELD on a resolved result, never a + * rejection of `run()` — reporting a failed program is the caller's job, not + * an exception path. + */ +export interface CodeRunResult { + /** + * The program's completion value (its top-level `return`), when it ran to + * completion and the value survived the runtime's serialization boundary; + * a non-transferable value is replaced by a string rendering, and a failed + * or value-less run leaves this absent. + */ + value?: unknown + /** Everything the program emitted, in order (capped by the implementation). */ + logs: CodeLogEntry[] + /** Present iff the run failed; see {@link CodeRunFailure} for the taxonomy. */ + error?: CodeRunFailure +} diff --git a/packages/code-runtime/code-runtime/tests/service.spec.ts b/packages/code-runtime/code-runtime/tests/service.spec.ts new file mode 100644 index 0000000000..4ff6d8f313 --- /dev/null +++ b/packages/code-runtime/code-runtime/tests/service.spec.ts @@ -0,0 +1,87 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime' +import type { CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime' + +/** + * Minimal concrete runtime: records requests, "executes" by invoking every + * binding once in declaration order, and lets tests script the outcome. The + * seam package ships no implementation, so the contract is exercised through + * the smallest subclass that honors it. + */ +class StubRuntime extends CodeRuntime { + readonly language = 'typescript' + readonly isolation = 'in-process-stub' + requests: CodeRunRequest[] = [] + nextResult: CodeRunResult = { logs: [] } + + async run(request: CodeRunRequest): Promise { + this.requests.push(request) + if (request.signal?.aborted) { + return { logs: [], error: { kind: 'abort', message: String(request.signal.reason) } } + } + for (const namespace of request.bindings) { + for (const fn of Object.values(namespace.functions)) { + await fn({ from: 'stub' }) + } + } + return this.nextResult + } +} + +async function setup() { + const ctx = new Context() + await ctx.plugin(StubRuntime) + const runtime = ctx.codeRuntime as StubRuntime + return { ctx, runtime } +} + +describe('CodeRuntime service seam', () => { + it('registers as ctx.codeRuntime and serves the abstract API', async () => { + const { runtime } = await setup() + expect(runtime.language).toBe('typescript') + expect(runtime.isolation).toBe('in-process-stub') + + const calls: unknown[] = [] + const result = await runtime.run({ + program: 'return 1', + bindings: [{ global: 'tools', functions: { probe: async args => void calls.push(args) } }], + }) + expect(result).toEqual({ logs: [] }) + expect(calls).toEqual([{ from: 'stub' }]) + expect(runtime.requests).toHaveLength(1) + }) + + it('reports a failed run as an error field on a resolved result, never a rejection', async () => { + const { runtime } = await setup() + runtime.nextResult = { + logs: [{ source: 'console', level: 'error', text: 'boom' }], + error: { kind: 'exception', message: 'boom' }, + } + const result = await runtime.run({ program: 'throw new Error("boom")', bindings: [] }) + expect(result.error).toEqual({ kind: 'exception', message: 'boom' }) + expect(result.value).toBeUndefined() + }) + + it('reports a pre-aborted signal as an abort failure', async () => { + const { runtime } = await setup() + const controller = new AbortController() + controller.abort('cancelled') + const result = await runtime.run({ program: 'return 1', bindings: [], signal: controller.signal }) + expect(result.error).toEqual({ kind: 'abort', message: 'cancelled' }) + }) + + it('is removed from the context when the providing fiber disposes (HMR safety)', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(StubRuntime) + expect(ctx.get('codeRuntime')).toBeInstanceOf(StubRuntime) + + await fiber.dispose() + expect(ctx.get('codeRuntime')).toBeUndefined() + }) + + it('rejects a second implementation in the same context (duplicate service)', async () => { + const { ctx } = await setup() + await expect(ctx.plugin(StubRuntime)).rejects.toThrow(/registered/) + }) +}) diff --git a/packages/code-runtime/code-runtime/tsconfig.json b/packages/code-runtime/code-runtime/tsconfig.json new file mode 100644 index 0000000000..754725418e --- /dev/null +++ b/packages/code-runtime/code-runtime/tsconfig.json @@ -0,0 +1,18 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + } + ] +} diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index 01fcadbaea..03f7c9ab4b 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -216,6 +216,11 @@ export class BasicCompactService extends CompactService { * Estimate the token count of content blocks — chars divided by the * `charsPerToken` config, with per-block overhead. Override in a subclass to * plug in a real tokenizer. + * + * @param blocks - the blocks to estimate; `tool-result` blocks recurse into + * their nested content, and unknown (merge-extended) types fall back to + * their JSON-stringified length. + * @returns the estimated token count. */ estimateContentTokens(blocks: readonly ContentBlock[]): number { const { charsPerToken } = this.config @@ -246,6 +251,11 @@ export class BasicCompactService extends CompactService { /** * Estimate token count for a single session event. Returns 0 for non-message * event types (boundaries, chunks, usage, errors, compact markers). + * + * @param event - any session event; only the message-bearing types carry + * content to count. + * @returns the estimated token count of the event's content, or 0 for a + * non-message event. */ estimateEventTokens(event: SessionEvent): number { switch (event.type) { @@ -260,7 +270,14 @@ export class BasicCompactService extends CompactService { } } - /** Estimate total tokens across a list of messages plus optional system prompt. */ + /** + * Estimate total tokens across a list of messages plus optional system prompt. + * + * @param messages - the derived conversation messages; each adds a fixed + * role-framing overhead on top of its content estimate. + * @param systemPrompt - counted at chars / `charsPerToken` when provided. + * @returns the estimated token footprint of the whole request. + */ estimateTokens(messages: readonly Message[], systemPrompt?: string): number { let total = 0 for (const msg of messages) { @@ -293,6 +310,13 @@ export class BasicCompactService extends CompactService { * used (`model`, `maxTokens`) — the caller logs the envelope on the * `compact/summary` provenance event, so an overriding subclass (template * or remote summarizer) reports its own envelope honestly. + * + * @param text - plain-text rendering of the conversation region to condense. + * @param agent - supplies the fallback model and the session id stamped on + * the call; throws when neither it nor the config names a model. + * @param signal - optional abort signal, forwarded into the model call. + * @returns the text-only summary blocks plus the call envelope used + * (`model`, and `maxTokens` when the summarizer has a cap). */ async summarize( text: string, agent: Agent, signal?: AbortSignal, diff --git a/packages/compact/compact-basic/src/types.ts b/packages/compact/compact-basic/src/types.ts index 2f10084ac3..a590c01431 100644 --- a/packages/compact/compact-basic/src/types.ts +++ b/packages/compact/compact-basic/src/types.ts @@ -54,6 +54,9 @@ export type ResolvedConfig = Required * each committed summary must be smaller than the content it shadows, and * `compactIfNeeded` may re-compact up to `compactionRetries` extra times before * throwing if the surface still exceeds the threshold. + * + * @param config - the raw, unresolved backend config. + * @returns the validated config with `auto` and `charsPerToken` defaulted. */ export function resolveConfig(config: BasicCompactConfig): ResolvedConfig { const resolved: ResolvedConfig = { auto: true, charsPerToken: 4, ...config } diff --git a/packages/core/agent-core/README.md b/packages/core/agent-core/README.md index 353e68332d..3f2ff08c0e 100644 --- a/packages/core/agent-core/README.md +++ b/packages/core/agent-core/README.md @@ -35,11 +35,11 @@ This is the [interface/implementation/consumer seam](../../../docs/rfc/implement ```ts import type { Config } from '@deepseek-ai/dsh-agent-core' -// { agents?, persona? } — the schema is z.intersect([AgentLoop.Config, SystemPrompt.Config]), +// { agents?, persona?, toolOrder? } — the schema is z.intersect([AgentLoop.Config, SystemPrompt.Config]), // so validation and defaulting can never drift from the owners'. ``` -The bundle FORWARDS each field to the child that owns it: `agents` to `agent-loop` (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`) — and `persona` to `dsh-system-prompt` (default `''`), the deployment's persona section. Forwarding is exactly why the owners can live in the shared spine even though the apps disagree on what to configure. +The bundle FORWARDS each field to the child that owns it: `agents` to `agent-loop` (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`) — `persona` to `dsh-system-prompt` (default `''`), the deployment's persona section — and `toolOrder` to `dsh-system-prompt` (absent — lexicographic), the explicit model-facing tool order. Forwarding is exactly why the owners can live in the shared spine even though the apps disagree on what to configure. ## Why a code bundle, not a shared YAML include diff --git a/packages/core/agent-core/src/index.ts b/packages/core/agent-core/src/index.ts index 0831ae929e..785c8d1ff2 100644 --- a/packages/core/agent-core/src/index.ts +++ b/packages/core/agent-core/src/index.ts @@ -59,17 +59,20 @@ export const name = 'agent-core' /** * Bundle config: each field forwarded verbatim to the child that owns it — * `agents` to the agent loop (an app that pre-creates no agents, like the ACP - * bridge, simply omits it), `persona` to the system-prompt plugin (the - * deployment's persona section). Both are optional INPUT here because each - * owner's schema supplies the default (`[]` / `''`); the schema is the - * INTERSECTION of the owners' own schemas, so validation and defaulting can - * never drift from them. + * bridge, simply omits it), `persona` and `toolOrder` to the system-prompt + * plugin (the deployment's persona section and the explicit model-facing tool + * order). Every field is optional INPUT here because each owner's schema + * supplies the default (`[]` / `''` / absent — lexicographic); the schema is + * the INTERSECTION of the owners' own schemas, so validation and defaulting + * can never drift from them. */ export interface Config { /** The agent-loop `agents` list (see dsh-agent-loop's `Config`). */ agents?: AgentLoopConfig['agents'] /** The deployment persona (see dsh-system-prompt's `Config`). */ persona?: SystemPromptConfig['persona'] + /** The explicit model-facing tool order (see dsh-system-prompt's `Config`). */ + toolOrder?: SystemPromptConfig['toolOrder'] } /** Intersect the owners' schemas so validation + defaulting stay identical. */ @@ -78,11 +81,11 @@ export const Config = z.intersect([AgentLoop.Config, SystemPrompt.Config]) as un /** * Load the spine. Each `ctx.plugin(...)` mounts one child of the bundle fiber; * `agent-loop` receives the forwarded `agents` list and `system-prompt` the - * forwarded `persona`. Load order is irrelevant (cordis pends each fiber on - * its `inject` until the services it needs exist), but the listing mirrors the - * dependency layering for readability: the LLM vocabulary and core registries - * first, then the dev tripwire and the bash tool consumer, then the loop that - * drives them. + * forwarded `persona` and `toolOrder`. Load order is irrelevant (cordis pends + * each fiber on its `inject` until the services it needs exist), but the + * listing mirrors the dependency layering for readability: the LLM vocabulary + * and core registries first, then the dev tripwire and the bash tool consumer, + * then the loop that drives them. */ export function apply(ctx: Context, config: Config): void { ctx.plugin(Timer) @@ -91,8 +94,13 @@ export function apply(ctx: Context, config: Config): void { // The forwarded fields are validated + defaulted by this bundle's intersected // schema before apply runs, so the ?? fallbacks only narrow the // optional-input TYPES — they mirror the owners' schema defaults, never - // introduce different ones. - ctx.plugin(SystemPrompt, { persona: config.persona ?? '' }) + // introduce different ones. toolOrder has no owner-supplied default value — + // ABSENT means "lexicographic order" — so it is forwarded conditionally + // rather than via ??. + ctx.plugin(SystemPrompt, { + persona: config.persona ?? '', + ...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {}, + }) ctx.plugin(ToolRegistry) ctx.plugin(AgentRegistry) ctx.plugin(invariants) diff --git a/packages/core/agent-core/tests/agent-core.spec.ts b/packages/core/agent-core/tests/agent-core.spec.ts index 4a4c5587ed..818da77311 100644 --- a/packages/core/agent-core/tests/agent-core.spec.ts +++ b/packages/core/agent-core/tests/agent-core.spec.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' +import { TOOL_ORDER_REST } from '@deepseek-ai/dsh-system-prompt' import * as agentCore from '../src/index.ts' import { AgentId } from '@deepseek-ai/dsh-agent' @@ -67,6 +68,23 @@ describe('dsh-agent-core bundle', () => { await ctx.fiber.dispose() }) + it('forwards toolOrder to the system-prompt assembly', async () => { + const ctx = await mount({ toolOrder: ['zulu', TOOL_ORDER_REST] }) + // The bundle's own bash tools pend on the absent `ctx.bash` executor in + // this providerless mount, so register two plain tools to order. + for (const name of ['alpha', 'zulu']) { + ctx.get('tools')!.register({ + name, + description: name, + parameters: {}, + execute: async () => [], + }) + } + const assembly = await ctx.get('systemPrompt')!.assemble() + expect(assembly.tools.map(tool => tool.name)).toEqual(['zulu', 'alpha']) + await ctx.fiber.dispose() + }) + it('re-exports the loop config schema as its own', () => { expect(agentCore.Config).toBeDefined() expect(agentCore.name).toBe('agent-core') diff --git a/packages/core/agent-loop/src/agent.ts b/packages/core/agent-loop/src/agent.ts index 433c28326b..49de0f77c4 100644 --- a/packages/core/agent-loop/src/agent.ts +++ b/packages/core/agent-loop/src/agent.ts @@ -22,6 +22,10 @@ import { isTurnOpen, lastTurnNumber, runLoop } from './loop.ts' * the agent/* event taxonomy — plugins never need this class. */ export class ReactLoopAgent implements Agent { + /** + * The queued + steering FIFOs behind {@link send}/{@link steer}. Public so + * the driver loop can drain it; {@link cancel} clears it wholesale. + */ readonly inbox = new Inbox() private _status: AgentStatus = 'idle' @@ -256,6 +260,8 @@ export class ReactLoopAgent implements Agent { * promise (unblocking the idle wait), releases any `whenIdle` waiters, and * aborts the current request if any. The returned `agent.done` promise * resolves once the loop exits. + * @returns the disposer — idempotent and infallible (it runs inside the + * fiber's LIFO disposal chain, where a throw would skip later disposers). */ start(): () => void { this.done = runLoop(this.ctx, this, { diff --git a/packages/core/agent-loop/src/inbox.ts b/packages/core/agent-loop/src/inbox.ts index a7b2e64e2c..abb588b919 100644 --- a/packages/core/agent-loop/src/inbox.ts +++ b/packages/core/agent-loop/src/inbox.ts @@ -24,30 +24,47 @@ export class Inbox { private steeringMessages: InboxMessage[] = [] private wakeup: (() => void) | undefined - /** Resolves when a queued message arrives (used by the idle loop). */ + /** True while queued messages are pending — read by the idle wait's fast path and the loop's turn-start checks. */ get hasQueued(): boolean { return this.queuedMessages.length > 0 } + /** True while steering messages are pending — read by `cancel()`'s arm gate and the loop's stop-override check. */ get hasSteering(): boolean { return this.steeringMessages.length > 0 } + /** + * Add a message to the queued FIFO and wake a parked {@link waitForQueued}. + * @param message - the message to queue for the next turn start. + */ enqueue(message: InboxMessage): void { this.queuedMessages.push(message) this.wakeup?.() } + /** + * Add a message to the steering FIFO. Deliberately no wakeup: steering is + * drained between steps of a running turn, never by the idle wait — + * `Agent.steer()` on an idle agent falls back to `send()` instead. + * @param message - the message to inject between steps of the running turn. + */ steer(message: InboxMessage): void { this.steeringMessages.push(message) } - /** Drain all queued messages (turn start). */ + /** + * Drain all queued messages (turn start). + * @returns the drained messages in arrival order; the queued FIFO is left empty. + */ drainQueued(): InboxMessage[] { return this.queuedMessages.splice(0) } - /** Drain all steering messages (between steps). */ + /** + * Drain all steering messages (between steps). + * @returns the drained messages in arrival order; the steering FIFO is left empty. + */ drainSteering(): InboxMessage[] { return this.steeringMessages.splice(0) } @@ -62,7 +79,12 @@ export class Inbox { this.steeringMessages.length = 0 } - /** Wait until a queued message arrives or `cancel` resolves. */ + /** + * Wait until a queued message arrives or `cancel` resolves. + * @param cancel - a promise whose resolution abandons the wait without a + * message (the driver loop passes the agent's disposed promise so a parked + * loop can exit). + */ waitForQueued(cancel: Promise): Promise { if (this.hasQueued) return Promise.resolve() const { promise, resolve } = Promise.withResolvers() diff --git a/packages/core/agent-loop/src/index.ts b/packages/core/agent-loop/src/index.ts index 92bbe7b8dc..e7e0e562d9 100644 --- a/packages/core/agent-loop/src/index.ts +++ b/packages/core/agent-loop/src/index.ts @@ -29,6 +29,10 @@ declare module 'cordis' { } } +/** + * Plugin config: the agents to create — or resume, via `resumeSessionId` — + * declaratively at startup, so a cordis.yml deployment needs no code. + */ export interface Config { /** Agents created from configuration at startup. */ agents: (AgentOptions & { diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 9484f0ad5f..eb5b07ba23 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -185,6 +185,9 @@ export interface LoopHandle { * re-enqueue leftover steering as queued ⟵ steering is never stranded * idle (emit agent/status) unless more queued * ``` + * @param ctx - the plugin context the loop reaches events (agent/…, session/flush) and services (systemPrompt, llm, tools) through. + * @param agent - the agent this invocation drives for its whole lifetime (its inbox, session, and options). + * @param handle - the bridge to the agent's mutable state: status/abort setters plus the disposal and cancel-marker reads. */ export async function runLoop(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle): Promise { // Per-instance transmission bookkeeping: whether THIS loop instance has @@ -875,7 +878,11 @@ function withoutToolCalls(message: Message): Message { return { ...message, content: message.content.filter(block => block.type !== 'tool-call') } } -/** The last turn number in a (possibly seeded) session log, or 0. */ +/** + * The last turn number in a (possibly seeded) session log, or 0. + * @param session - the session whose log is scanned for the latest `turn/start`. + * @returns the latest `turn/start`'s turn number, or 0 when the log has none (the next turn is this plus one). + */ export function lastTurnNumber(session: Session): number { const lastStart = session.events.findLast(event => event.type === 'turn/start') return lastStart?.data.turn ?? 0 @@ -889,6 +896,8 @@ export function lastTurnNumber(session: Session): number { * returns to idle), so status is not a reliable open-turn signal. Used by * `inject()` to choose between appending into an open turn vs. wrapping the * injection in its own one-shot turn (the turn-enclosure RFC). + * @param session - the session whose log is inspected. + * @returns true when the log's last turn boundary is a `turn/start` with no matching `turn/end` yet. */ export function isTurnOpen(session: Session): boolean { const last = session.events.findLast(e => e.type === 'turn/start' || e.type === 'turn/end') diff --git a/packages/core/agent-loop/src/request-log.ts b/packages/core/agent-loop/src/request-log.ts index 07c57b9ff9..90f47068fa 100644 --- a/packages/core/agent-loop/src/request-log.ts +++ b/packages/core/agent-loop/src/request-log.ts @@ -19,7 +19,10 @@ export interface TransmissionLog { loggedHeader: boolean } -/** Fresh bookkeeping for a newly-started loop instance. */ +/** + * Fresh bookkeeping for a newly-started loop instance. + * @returns state with `loggedHeader` false, so the instance's first request appends an anchoring snapshot. + */ export function createTransmissionLog(): TransmissionLog { return { loggedHeader: false } } diff --git a/packages/core/agent-loop/tests/tool-order.spec.ts b/packages/core/agent-loop/tests/tool-order.spec.ts new file mode 100644 index 0000000000..35329ce679 --- /dev/null +++ b/packages/core/agent-loop/tests/tool-order.spec.ts @@ -0,0 +1,117 @@ +/** + * Loop-level tool-order determinism: the request/header event — and therefore + * the frozen request the adapter receives — carries the assembly's canonical + * tool order (system-prompt's `toolOrder` config, or lexicographic name + * order), regardless of the order tool plugins happened to register in. + * Registration order is a plugin-load artifact (concurrent dynamic imports + * race), so nothing downstream of the registry may depend on it. + */ + +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import LlmService from '@deepseek-ai/dsh-llm' +import SessionStore, { foldRequestHeader } from '@deepseek-ai/dsh-session' +import SystemPrompt, { TOOL_ORDER_REST } from '@deepseek-ai/dsh-system-prompt' +import type { Config as SystemPromptConfig } from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' +import { MockAdapter, textResponse } from './mock-adapter.ts' + +async function harness(adapter: MockAdapter, toolOrder?: SystemPromptConfig['toolOrder']) { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt, { persona: 'stable base', ...toolOrder !== undefined ? { toolOrder } : {} }) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + ctx.llm.registerAdapter(['mock'], adapter) + return ctx +} + +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { + return new Promise((resolve) => { + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'idle') { + dispose() + resolve() + } + }) + }) +} + +function registerNamed(ctx: Context, name: string) { + ctx.tools.register(defineTool({ + name, + description: `the ${name} tool`, + parameters: {}, + async execute() { + return [{ type: 'text', text: name }] + }, + })) +} + +/** Run one text-only turn and return the harness context + agent. */ +async function runTurn(registrationOrder: string[], toolOrder?: SystemPromptConfig['toolOrder']) { + const adapter = new MockAdapter([textResponse('done')]) + const ctx = await harness(adapter, toolOrder) + for (const name of registrationOrder) registerNamed(ctx, name) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + return { ctx, agent, adapter } +} + +describe('loop-level canonical tool order', () => { + it('logs the request/header with tools in canonical order, not registration order', async () => { + const { agent, adapter } = await runTurn(['zulu', 'alpha', 'mike']) + const header = foldRequestHeader(agent.session.events) + expect(header?.tools?.map(tool => tool.name)).toEqual(['alpha', 'mike', 'zulu']) + // The dispatched request is built FROM the logged header (whose tools the + // assembly already canonicalized) and reaches the adapter deep-frozen — + // the marker the reconstruction invariant keys on. + expect(adapter.requests[0]?.tools?.map(tool => tool.name)).toEqual(['alpha', 'mike', 'zulu']) + expect(Object.isFrozen(adapter.requests[0])).toBe(true) + expect(adapter.requests[0]?.sessionId).toBe(agent.session.id) + }) + + it('produces the same header order for any registration order', async () => { + const first = await runTurn(['alpha', 'mike', 'zulu']) + const second = await runTurn(['zulu', 'mike', 'alpha']) + const names = (run: typeof first) => foldRequestHeader(run.agent.session.events)?.tools?.map(tool => tool.name) + expect(names(first)).toEqual(['alpha', 'mike', 'zulu']) + expect(names(second)).toEqual(names(first)) + }) + + it('honors a configured toolOrder in the logged header and the dispatched request', async () => { + const { agent, adapter } = await runTurn(['alpha', 'zulu', 'mike'], ['zulu', TOOL_ORDER_REST]) + const header = foldRequestHeader(agent.session.events) + expect(header?.tools?.map(tool => tool.name)).toEqual(['zulu', 'alpha', 'mike']) + expect(adapter.requests[0]?.tools?.map(tool => tool.name)).toEqual(['zulu', 'alpha', 'mike']) + expect(Object.isFrozen(adapter.requests[0])).toBe(true) + }) + + it('fails the turn — no model request — when toolOrder names an unregistered tool', async () => { + // The assemble rejection escapes to runTurn's outer catch: the open turn + // closes with an `error` reason (agent/error mirrors it), no step opens, + // no request/header is logged, the adapter never sees a request, and the + // agent returns to idle — a misconfigured deployment fails every turn + // deterministically instead of silently reordering nothing. + const adapter = new MockAdapter([textResponse('never sent')]) + const ctx = await harness(adapter, ['ghost', TOOL_ORDER_REST]) + registerNamed(ctx, 'alpha') + const errors: Error[] = [] + ctx.on('agent/error', (_agent, _turn, _step, error) => void errors.push(error)) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + expect(adapter.requests).toHaveLength(0) + expect(errors.map(e => e.message)).toEqual(['toolOrder lists unregistered tool "ghost"; registered tools: alpha']) + expect(foldRequestHeader(agent.session.events)).toBeUndefined() + const end = agent.session.events.find(e => e.type === 'turn/end') + expect(end?.type === 'turn/end' && end.data.reason).toMatchObject({ kind: 'error', step: 1 }) + // The turn is balanced (turn/start → turn/end) with no step events inside. + expect(agent.session.events.some(e => e.type === 'step/start')).toBe(false) + }) +}) diff --git a/packages/core/agent/src/types.ts b/packages/core/agent/src/types.ts index 1fd6a57867..46bc52128a 100644 --- a/packages/core/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -50,7 +50,11 @@ import type {} from '@deepseek-ai/dsh-system-prompt' /** Identifies one live agent in the registry. */ export type AgentId = Branded<'AgentId'> -/** Brand a string as an {@link AgentId}. */ +/** + * Brand a string as an {@link AgentId}. + * @param id - the raw agent id string. + * @returns the same string, branded (a compile-time cast — no runtime cost). + */ export function AgentId(id: string): AgentId { return id as AgentId } @@ -80,10 +84,22 @@ export interface AgentOptions { model?: string } +/** + * Options for {@link Agent.send}/{@link Agent.steer}/{@link Agent.inject}. An + * absent `source` resolves to `{ kind: 'user' }`, so a plugin supplying content + * must label itself here or its message is recorded as a user prompt (see + * {@link HookContext} on why that label is load-bearing). + */ export interface SendOptions { source?: MessageSource } +/** + * An agent's lifecycle state, emitted on every transition as `agent/status`: + * `idle` (parked, waiting for queued work), `running` (a turn is in progress), + * `disposed` (terminal — no transition leaves it, and `send`/`steer`/`inject` + * throw). + */ export type AgentStatus = 'idle' | 'running' | 'disposed' /** diff --git a/packages/core/agent/tests/verify-export-jsdoc.spec.ts b/packages/core/agent/tests/verify-export-jsdoc.spec.ts new file mode 100644 index 0000000000..b699a72765 --- /dev/null +++ b/packages/core/agent/tests/verify-export-jsdoc.spec.ts @@ -0,0 +1,555 @@ +/** + * Negative-path tests for the export-surface JSDoc gate + * (`scripts/verify-export-jsdoc.ts`). + * + * The gate's positive half runs against the real tree in CI (`pnpm run + * verify-export-jsdoc`, part of doc-sync). What that run cannot prove is that + * the walk REJECTS an undocumented surface the way it promises to — and that + * every deliberate exemption (heritage members, plugin-protocol slots, + * constructors, overload implementations, augmentation bodies, re-exports) + * actually holds. These tests drive `collectExportJsdocViolations()` against + * synthetic fixture packages, mirroring the gen-cordis-catalog negative + * tests. + */ + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { collectExportJsdocViolations } from '../../../../scripts/verify-export-jsdoc.ts' + +const roots: string[] = [] + +afterEach(() => { + while (roots.length) rmSync(roots.pop()!, { recursive: true, force: true }) +}) + +/** Write fixture files under `packages/group/fix/src/` and return the scan root. */ +function fixture(files: Record): string { + const root = mkdtempSync(join(tmpdir(), 'export-jsdoc-')) + roots.push(root) + for (const [rel, content] of Object.entries(files)) { + const abs = join(root, 'packages', 'group', 'fix', 'src', rel) + mkdirSync(dirname(abs), { recursive: true }) + writeFileSync(abs, content) + } + return root +} + +/** Single-file fixture shorthand: the content becomes `src/index.ts`. */ +const make = (content: string): string => fixture({ 'index.ts': content }) + +describe('verify-export-jsdoc functions and consts', () => { + it('accepts a fully documented surface', () => { + expect(collectExportJsdocViolations(make(` +/** + * Add one to a count. + * @param n - the count to bump. + * @returns the count plus one. + */ +export function bump(n: number): number { return n + 1 } + +/** + * Fire-and-forget (void needs no @returns). + * @param flag - whether to arm. + */ +export function poke(flag: boolean): void { void flag } + +/** The default retry budget. */ +export const RETRIES = 3 + +/** + * Halve a count. + * @param n - the count to halve. + * @returns the count halved. + */ +export const halve = (n: number): number => n / 2 +`))).toEqual([]) + }) + + it('flags an exported function with no JSDoc at all', () => { + expect(collectExportJsdocViolations(make( + 'export function bare(): void {}\n', + ))).toEqual([expect.stringMatching(/exported function 'bare' .* has no JSDoc\./)]) + }) + + it('flags a missing @param and a missing @returns', () => { + const violations = collectExportJsdocViolations(make( + '/** Docs without tags. */\nexport function f(x: number): number { return x }\n', + )) + expect(violations).toEqual([ + expect.stringMatching(/exported function 'f' .* is missing @param x\./), + expect.stringMatching(/exported function 'f' .* is missing @returns \(return type: number\)\./), + ]) + }) + + it('flags an unannotated (inferred) return type', () => { + expect(collectExportJsdocViolations(make( + '/**\n * Docs.\n * @param x - value.\n */\nexport function f(x: number) { return x }\n', + ))).toEqual([expect.stringMatching(/no return type annotation/)]) + }) + + it('flags tags-only JSDoc with no description prose', () => { + expect(collectExportJsdocViolations(make( + '/**\n * @param x - value.\n */\nexport function f(x: number): void {}\n', + ))).toEqual([expect.stringMatching(/no description prose above its block tags/)]) + }) + + it('flags a stale @param and a binding-pattern parameter', () => { + const violations = collectExportJsdocViolations(make( + '/**\n * Docs.\n * @param ghost - not real.\n */\nexport function f({ a }: { a: number }): void {}\n', + )) + expect(violations).toEqual([ + expect.stringMatching(/parameter '\{ a \}' is a binding pattern; the export surface needs simple identifier parameters/), + expect.stringMatching(/@param ghost does not match any parameter \(stale tag\?\)/), + ]) + }) + + it('exempts a `this` receiver annotation from @param', () => { + expect(collectExportJsdocViolations(make( + '/**\n * Docs.\n * @param x - value.\n */\nexport function f(this: object, x: number): void {}\n', + ))).toEqual([]) + }) + + it('waives @returns for a declarator-annotated const but not an unannotated one', () => { + expect(collectExportJsdocViolations(make(` +type Fn = (x: number) => number +/** + * Uses the named signature. + * @param x - value. + */ +export const good: Fn = x => x +/** + * No signature anywhere. + * @param x - value. + */ +export const bad = (x: number) => x +`))).toEqual([expect.stringMatching(/exported const 'bad' .* has no return type annotation/)]) + }) + + it('requires description prose on a non-function const', () => { + expect(collectExportJsdocViolations(make( + 'export const LIMIT = 10\n', + ))).toEqual([expect.stringMatching(/exported const 'LIMIT' .* has no JSDoc\./)]) + }) +}) + +describe('verify-export-jsdoc type-level exports', () => { + it('requires description prose on interfaces, type aliases, and enums', () => { + const violations = collectExportJsdocViolations(make( + 'export interface I { a: number }\nexport type T = number\nexport enum E { A }\n', + )) + expect(violations).toEqual([ + expect.stringMatching(/exported interface 'I' .* has no JSDoc\./), + expect.stringMatching(/exported type 'T' .* has no JSDoc\./), + expect.stringMatching(/exported enum 'E' .* has no JSDoc\./), + ]) + }) + + it('skips `declare module` augmentation bodies (the cordis gate owns them)', () => { + expect(collectExportJsdocViolations(make( + "declare module 'cordis' {\n interface Events {\n 'fix/x'(): void\n }\n}\nexport {}\n", + ))).toEqual([]) + }) +}) + +describe('verify-export-jsdoc export forms', () => { + it('resolves an `export { … }` list to the local declaration', () => { + expect(collectExportJsdocViolations(make( + 'function f(): void {}\nexport { f }\n', + ))).toEqual([expect.stringMatching(/exported function 'f' .* has no JSDoc\./)]) + }) + + it('does not treat a never-exported sibling declarator as surface (review round 2)', () => { + // `export { publicValue }` resolves to the whole variable statement; only + // the named declarator is surface — the gate must not demand JSDoc for + // the private sibling sharing the statement. + expect(collectExportJsdocViolations(make( + '/** The public knob. */\nconst publicValue = 1, privateHelper = 2\nexport { publicValue }\nvoid privateHelper\n', + ))).toEqual([]) + }) + + it('unions declarators across multiple export lists over one statement (review round 2)', () => { + // Two lists each name one declarator of the same undocumented statement: + // both are surface (deduplicating on first resolution would drop `b`), + // while the never-exported `c` stays out. + const violations = collectExportJsdocViolations(make( + 'const a = 1, b = 2, c = 3\nexport { a }\nexport { b }\nvoid c\n', + )) + expect(violations).toEqual([ + expect.stringMatching(/exported const 'a' .* has no JSDoc\./), + expect.stringMatching(/exported const 'b' .* has no JSDoc\./), + ]) + }) + + it('scopes a default-export identifier to its own declarator (review round 2)', () => { + // `export default` of an identifier reaches the statement through the + // same name lookup as an export list; the sibling stays private. + expect(collectExportJsdocViolations(make( + '/** The app entry. */\nconst app = 1, scratch = 2\nexport default app\nvoid scratch\n', + ))).toEqual([]) + }) + + it('reports a re-exported module once, at its defining file', () => { + const violations = collectExportJsdocViolations(fixture({ + 'index.ts': "export * from './other.ts'\n", + 'other.ts': 'export function f(): void {}\n', + })) + expect(violations).toEqual([expect.stringMatching(/other\.ts:1\) has no JSDoc\./)]) + }) + + it('exempts overload implementations when the signatures are documented', () => { + expect(collectExportJsdocViolations(make(` +/** + * From a number. + * @param x - the number. + * @returns its text. + */ +export function f(x: number): string +/** + * From a flag. + * @param x - the flag. + * @returns its text. + */ +export function f(x: boolean): string +export function f(x: number | boolean): string { return String(x) } +`))).toEqual([]) + }) +}) + +describe('verify-export-jsdoc classes', () => { + it('flags an undocumented class, method, property, and accessor', () => { + const violations = collectExportJsdocViolations(make(` +export class C { + state = 1 + get view(): number { return this.state } + run(x: number): number { return x } +} +`)) + expect(violations).toEqual([ + expect.stringMatching(/exported class 'C' .* has no JSDoc\./), + expect.stringMatching(/exported class property 'C.state' .* has no JSDoc\./), + expect.stringMatching(/exported class accessor 'C.view' .* has no JSDoc\./), + expect.stringMatching(/exported class method 'C.run' .* has no JSDoc\./), + ]) + }) + + it('exempts members declared by an extends/implements heritage type', () => { + expect(collectExportJsdocViolations(make(` +/** Seam. */ +export abstract class Base { + /** + * Do it. + * @param x - input. + * @returns output. + */ + abstract run(x: number): number +} +/** Iface. */ +export interface Sized { + /** Byte size. */ + size: number +} +/** Impl. */ +export class Impl extends Base implements Sized { + size = 0 + run(x: number): number { return x } +} +`))).toEqual([]) + }) + + it('skips private/protected/#private members and constructors', () => { + expect(collectExportJsdocViolations(make(` +/** Documented. */ +export class C { + #secret = 1 + private hidden(): void {} + protected hook(): void {} + constructor(x: number) { void x } +} +`))).toEqual([]) + }) + + it('exempts plugin-protocol statics but checks other statics', () => { + const violations = collectExportJsdocViolations(make(` +/** Plugin. */ +export class C { + static Config = { a: 1 } + static inject = ['bash'] + static reusable = true + static other = 1 +} +`)) + expect(violations).toEqual([expect.stringMatching(/exported class property 'C.other' .* has no JSDoc\./)]) + }) + + it("covers a set accessor by the getter's doc", () => { + expect(collectExportJsdocViolations(make(` +/** Documented. */ +export class C { + /** The current width. */ + get width(): number { return 1 } + set width(_v: number) {} +} +`))).toEqual([]) + }) +}) + +describe('verify-export-jsdoc plugin protocol and namespaces', () => { + it('exempts top-level plugin-protocol exports', () => { + expect(collectExportJsdocViolations(make(` +export const name = 'fix' +export const inject = ['bash'] +export const reusable = true +export const Config = { parse: true } +export function apply(): void {} +`))).toEqual([]) + }) + + it('recurses into namespaces with qualified names and honors the merge idiom', () => { + const violations = collectExportJsdocViolations(make(` +/** The plugin class. */ +export class Fix {} +export namespace Fix { + export interface Config { a: number } +} +export namespace Loose { + export const x = 1 +} +`)) + expect(violations).toEqual([ + expect.stringMatching(/exported interface 'Fix.Config' .* has no JSDoc\./), + expect.stringMatching(/exported namespace 'Loose' .* has no JSDoc\./), + expect.stringMatching(/exported const 'Loose.x' .* has no JSDoc\./), + ]) + }) +}) + +describe('verify-export-jsdoc fail-closed forms (review round 1)', () => { + it('checks the function contract on a non-identifier default export', () => { + expect(collectExportJsdocViolations(make( + '/** Doubles. */\nexport default (x: number): number => x * 2\n', + ))).toEqual([ + expect.stringMatching(/default export .* is missing @param x\./), + expect.stringMatching(/default export .* is missing @returns \(return type: number\)\./), + ]) + expect(collectExportJsdocViolations(make( + '/**\n * Doubles.\n * @param x - the input.\n * @returns twice the input.\n */\nexport default (x: number): number => x * 2\n', + ))).toEqual([]) + }) + + it('treats an inline function-type annotation as the surface signature', () => { + expect(collectExportJsdocViolations(make( + '/** Maps a number. */\nexport declare const f: (x: number) => number\n', + ))).toEqual([ + expect.stringMatching(/exported const 'f' .* is missing @param x\./), + expect.stringMatching(/exported const 'f' .* is missing @returns \(return type: number\)\./), + ]) + expect(collectExportJsdocViolations(make( + '/**\n * Maps a number.\n * @param x - the input.\n * @returns the mapped value.\n */\nexport const f: (x: number) => number = v => v\n', + ))).toEqual([]) + }) + + it('recurses into an ambient declare namespace where members export implicitly', () => { + expect(collectExportJsdocViolations(make( + 'export declare namespace N {\n function f(x: number): number\n}\n', + ))).toEqual([ + expect.stringMatching(/exported namespace 'N' .* has no JSDoc\./), + expect.stringMatching(/exported function 'N.f' .* has no JSDoc\./), + ]) + }) + + it('requires an export-import alias to document itself (its target may be unwalked)', () => { + expect(collectExportJsdocViolations(make( + '/** Holder. */\nexport namespace N {\n /** The value. */\n export const x = 1\n}\nexport import y = N.x\n', + ))).toEqual([expect.stringMatching(/exported alias 'y' .* has no JSDoc\./)]) + expect(collectExportJsdocViolations(make( + 'namespace N {\n export const x = 1\n}\n/** Alias surfacing the internal counter. */\nexport import y = N.x\n', + ))).toEqual([]) + }) + + it('refuses an export-import alias to a callable, class, or namespace target', () => { + const refusal = /exported alias 'g' .* aliases a callable, class, or namespace target/ + expect(collectExportJsdocViolations(make( + 'namespace N {\n export function f(x: number): number { return x }\n}\n/** Alias. */\nexport import g = N.f\n', + ))).toEqual([expect.stringMatching(refusal)]) + expect(collectExportJsdocViolations(make( + 'namespace N {\n export class C {\n run(x: number): number { return x }\n }\n}\n/** Alias. */\nexport import g = N.C\n', + ))).toEqual([expect.stringMatching(refusal)]) + expect(collectExportJsdocViolations(make( + 'namespace N {\n export namespace Sub {\n export function f(x: number): number { return x }\n }\n}\n/** Alias. */\nexport import g = N.Sub\n', + ))).toEqual([expect.stringMatching(refusal)]) + }) + + it('classifies wrapped function initializers and default exports (parens, satisfies)', () => { + expect(collectExportJsdocViolations(make( + 'type Fn = (x: number) => number\n/** Wrapped. */\nexport const f = (((x: number): number => x)) satisfies Fn\n', + ))).toEqual([ + expect.stringMatching(/exported const 'f' .* is missing @param x\./), + expect.stringMatching(/exported const 'f' .* is missing @returns \(return type: number\)\./), + ]) + expect(collectExportJsdocViolations(make( + 'type Fn = (x: number) => number\n/** Wrapped. */\nexport default (((x: number): number => x * 2) satisfies Fn)\n', + ))).toEqual([ + expect.stringMatching(/default export .* is missing @param x\./), + expect.stringMatching(/default export .* is missing @returns \(return type: number\)\./), + ]) + }) + + it('treats a single-call-signature type literal as the surface signature', () => { + expect(collectExportJsdocViolations(make( + '/** Maps. */\nexport declare const f: { (x: number): number }\n', + ))).toEqual([ + expect.stringMatching(/exported const 'f' .* is missing @param x\./), + expect.stringMatching(/exported const 'f' .* is missing @returns \(return type: number\)\./), + ]) + }) + + it('refuses a hybrid callable type literal instead of narrowing the check', () => { + expect(collectExportJsdocViolations(make( + '/** Hybrid. */\nexport declare const f: { (x: number): number; flush: () => void }\n', + ))).toEqual([expect.stringMatching(/exported const 'f'.*callable type literal is not gate-classifiable; extract a named type/)]) + }) + + it('refuses an export-equals assignment instead of failing open', () => { + expect(collectExportJsdocViolations(make( + 'const x = 1\nexport = x\n', + ))).toEqual([expect.stringMatching(/export-equals assignment .* is not a gate-supported export form/)]) + }) +}) + +describe('verify-export-jsdoc heritage refinement (review round 1)', () => { + it('requires @param for parameters the base member never names', () => { + const violations = collectExportJsdocViolations(make(` +/** Seam. */ +export abstract class Base { + /** + * Do it. + * @param x - input. + * @returns output. + */ + abstract run(x: number): number +} +/** Impl. */ +export class Impl extends Base { + override run(x: number, verbose?: boolean): number { return verbose ? x : -x } +} +`)) + expect(violations).toEqual([expect.stringMatching(/exported class method 'Impl.run' .* is missing @param verbose\./)]) + }) + + it('does not exempt a public override of a protected-only base member', () => { + expect(collectExportJsdocViolations(make(` +/** Seam. */ +export abstract class Base { + /** Subclass hook. */ + protected hook(): void {} +} +/** Impl. */ +export class Impl extends Base { + override hook(): void {} +} +`))).toEqual([expect.stringMatching(/exported class method 'Impl.hook' .* has no JSDoc\./)]) + }) + + it('treats an underscore-prefixed rename of a base parameter as the same parameter', () => { + expect(collectExportJsdocViolations(make(` +/** Seam. */ +export abstract class Base { + /** + * Load it. + * @param cwd - the working directory to scope the lookup. + * @returns the loaded value. + */ + abstract load(cwd: string): number +} +/** Impl (ignores cwd). */ +export class Impl extends Base { + load(_cwd: string): number { return 1 } +} +`))).toEqual([]) + }) + + it('flags a binding-pattern parameter an override adds beyond the base', () => { + expect(collectExportJsdocViolations(make(` +/** Seam. */ +export abstract class Base { + /** + * Do it. + * @param x - input. + * @returns output. + */ + abstract run(x: number): number +} +/** Impl. */ +export class Impl extends Base { + override run(x: number, { verbose }: { verbose?: boolean } = {}): number { return verbose ? x : -x } +} +`))).toEqual([expect.stringMatching(/exported class method 'Impl.run' .* is a binding pattern/)]) + }) + + it('revives the @returns duty when an override grows a concrete result over a void base', () => { + const voidBase = ` +/** Seam. */ +export abstract class Base { + /** Do it (fire-and-forget). */ + abstract run(): void +} +` + expect(collectExportJsdocViolations(make(`${voidBase} +/** Impl. */ +export class Impl extends Base { + override run(): number { return 1 } +} +`))).toEqual([expect.stringMatching(/exported class method 'Impl.run' .* is missing @returns \(return type: number\)\./)]) + expect(collectExportJsdocViolations(make(`${voidBase} +/** Impl. */ +export class Impl extends Base { + /** + * Do it and count. + * @returns how many were done. + */ + override run(): number { return 1 } +} +`))).toEqual([]) + }) + + it('classifies an unannotated override return over a void base via the checker', () => { + const voidBase = ` +/** Seam. */ +export abstract class Base { + /** Do it (fire-and-forget). */ + abstract run(): void +} +` + expect(collectExportJsdocViolations(make(`${voidBase} +/** Impl. */ +export class Impl extends Base { + override run() { return 1 } +} +`))).toEqual([expect.stringMatching(/exported class method 'Impl.run' .* non-void result its heritage declaration does not document/)]) + expect(collectExportJsdocViolations(make(`${voidBase} +/** Impl (faithful void, no annotation needed). */ +export class Impl extends Base { + override run() {} +} +`))).toEqual([]) + }) + + it('keeps the full exemption when the base return already carries the @returns duty', () => { + expect(collectExportJsdocViolations(make(` +/** Seam. */ +export abstract class Base { + /** + * Count things. + * @returns the count. + */ + abstract run(): number +} +/** Impl. */ +export class Impl extends Base { + override run(): number { return 1 } +} +`))).toEqual([]) + }) +}) diff --git a/packages/core/session/README.md b/packages/core/session/README.md index 1a08f200b8..0ae5bbf39e 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -9,6 +9,7 @@ Creates and holds event-sourced `Session` instances. Persistence is intentionall ### Public API - `ctx.sessions.create(id?: SessionId, options?: { seed?: SessionEvent[]; meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number; seedLength?: number } }): Session` — Create a session. `options.seed` replays/forks an existing event log; `options.meta` attaches creation metadata (validated absolute `cwd`, `parentSession` lineage, seed boundary) as the immutable `SessionHeader`. The store fills `version`/`id` and defaults `createdAt` to now; a caller reconstructing a persisted session passes the original `createdAt` and persisted `seedLength` to preserve them. Disposed with the calling fiber. +- `ctx.sessions.fork(source, boundary?, childSessionId?): Session` — Resolve a live session object or id, select a seed through the inclusive `boundary` event seq (default: current last event), require that boundary to be `turn/end`, and create a live child session with lineage metadata. - `ctx.sessions.get(id: SessionId): Session | undefined` - `ctx.sessions.list(): Session[]` @@ -72,9 +73,9 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata) ### Extension points - Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log. -- Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. The surface rebuilds deterministically from `surfaceOp` markers in the seeded events. The seed is validated to the SAME invariants `append` enforces — including that every surface-eligible event (`SurfaceEventType`) carries a `surfaceOp` marker — so a marker-less message event is rejected at construction rather than silently vanishing from `deriveMessages()` (the surface is the sole derivation path) on resume. +- Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. The surface rebuilds deterministically from `surfaceOp` markers in the seeded events. The seed is validated to the SAME always-on invariants `append` enforces — contiguous seqs, JSON-serializable data, and required `surfaceOp` markers on surface-eligible events — so marker-less message events are rejected at construction rather than silently vanishing from `deriveMessages()`. Broader turn-enclosure checks stay in `dsh-invariants` and persistence repair. Ordinary live-session forks use `ctx.sessions.fork(source, boundary?, childSessionId?)`, where `boundary` is the inclusive source event seq to fork through. - Compaction: the `dsh-compact-basic` plugin appends a `user/message` with `surfaceOp: { op: 'replace', start, end }` to shadow old surface nodes behind a summary checkpoint. ### What is NOT here (TODO) -- **Session branching/tree** (pi-style entry tree) — deferred unless needed beyond seed-based forking. +- **Session branching/tree** (pi-style entry tree) — deferred unless needed beyond boundary-based `fork()`. diff --git a/packages/core/session/src/index.ts b/packages/core/session/src/index.ts index 18c7b4a8a2..cd9e1828b2 100644 --- a/packages/core/session/src/index.ts +++ b/packages/core/session/src/index.ts @@ -153,10 +153,15 @@ export class Session { this.header = header ?? { version: SESSION_FORMAT_VERSION, id, createdAt: Date.now() } } + /** + * The append-only event log, exposed live by reference (readonly-typed, not + * a snapshot): later appends are visible through the same array. + */ get events(): readonly SessionEvent[] { return this.log } + /** The next event's sequence number — always the log length (the `seq = log.length` contiguity contract). */ get seq(): number { return this.log.length } @@ -175,6 +180,9 @@ export class Session { * declare how it joins the surface, the sole source of derived history) and * rejected by the compiler for non-surface types like `turn/start` or * `assistant/chunk`. + * @returns the logged event — its assigned `seq`/`time` plus the SNAPSHOT of + * `data` that entered the log, so reading `event.data` back sees the logged + * value, never the caller's still-mutable input. * @throws if `data` is not losslessly JSON-serializable (BigInt, function, * symbol, undefined, non-finite number, circular ref, or an exotic object * like Map/Set/Date). The event log is the durable source of truth, so this @@ -362,6 +370,32 @@ export class Session { } } +/** A fork source: either the live session object or its live store id. */ +export type SessionForkSource = Session | SessionId + +/** + * Rejection codes for session forking: the fork source id is unknown to the + * live store (`SESSION_NOT_FOUND`) or names a session object that is not the + * store's live instance (`SESSION_NOT_LIVE`); the requested child id is + * already taken (`SESSION_ALREADY_EXISTS`); the boundary is not a contiguous + * existing seq (`INVALID_BOUNDARY`); or the boundary event is not a + * `turn/end` — a fork must cut on a closed turn (`OPEN_TURN`). + */ +export type SessionForkErrorCode = + | 'SESSION_NOT_FOUND' + | 'SESSION_NOT_LIVE' + | 'SESSION_ALREADY_EXISTS' + | 'INVALID_BOUNDARY' + | 'OPEN_TURN' + +/** Typed error for session fork rejections. */ +export class SessionForkError extends Error { + constructor(message: string, public readonly code: SessionForkErrorCode) { + super(message) + this.name = 'SessionForkError' + } +} + /** * In-memory session store (`ctx.sessions`). * @@ -496,6 +530,92 @@ export class SessionStore extends Service { list(): Session[] { return [...this.store.values()] } + + /** + * Create a live child session from a turn-enclosed prefix of a live source. + * `boundary` is an inclusive source event seq; omitted means the source's + * current last event. A non-empty selected slice must end at `turn/end`. + * + * @param source - Live source session object or id. + * @param boundary - Inclusive source event seq to fork through; omitted means + * the source's current last event, and omitted on an empty source forks an + * empty child. + * @param childSessionId - Optional child session id; omitted delegates to + * `SessionStore`'s id policy. + * @returns The created live child session. + */ + fork(source: SessionForkSource, boundary?: number, childSessionId?: SessionId): Session { + if (childSessionId !== undefined && this.get(childSessionId) !== undefined) { + throw new SessionForkError(`session "${childSessionId}" already exists`, 'SESSION_ALREADY_EXISTS') + } + const liveSource = this._resolveForkSource(source) + const seed = this._forkSeed(liveSource, boundary) + return this.create(childSessionId, { + seed, + meta: { + ...liveSource.header.cwd !== undefined ? { cwd: liveSource.header.cwd } : {}, + parentSession: liveSource.id, + seedLength: seed.length, + }, + }) + } + + private _forkSeed(session: Session, requestedBoundary: number | undefined): SessionEvent[] { + const events = session.events + const lastEvent = events.at(-1) + let boundary: number + if (requestedBoundary !== undefined) { + boundary = requestedBoundary + } else { + if (lastEvent === undefined) return [] + boundary = lastEvent.seq + } + if (!Number.isSafeInteger(boundary) || boundary < 0) { + throw new SessionForkError( + `fork boundary for session "${session.id}" must be a non-negative safe integer, got ${String(boundary)}`, + 'INVALID_BOUNDARY', + ) + } + if (boundary >= events.length) { + const lastSeq = events.at(-1)?.seq + throw new SessionForkError( + `fork boundary ${boundary} does not exist in session "${session.id}" (last seq: ${lastSeq ?? 'none'})`, + 'INVALID_BOUNDARY', + ) + } + + const boundaryEvent = events[boundary] + if (boundaryEvent === undefined || boundaryEvent.seq !== boundary) { + throw new SessionForkError( + `fork boundary ${boundary} does not match a contiguous event seq in session "${session.id}"`, + 'INVALID_BOUNDARY', + ) + } + if (boundaryEvent.type !== 'turn/end') { + throw new SessionForkError( + `fork boundary ${boundary} in session "${session.id}" must be turn/end, got ${boundaryEvent.type}`, + 'OPEN_TURN', + ) + } + + return events.slice(0, boundary + 1).map(event => structuredClone(event)) + } + + private _resolveForkSource(source: SessionForkSource): Session { + if (typeof source === 'string') { + const session = this.get(source) + if (session === undefined) throw new SessionForkError(`session "${source}" not found`, 'SESSION_NOT_FOUND') + return session + } + + const live = this.get(source.id) + if (live === undefined) { + throw new SessionForkError(`session "${source.id}" not found`, 'SESSION_NOT_FOUND') + } + if (live !== source) throw new SessionForkError(`session "${source.id}" is not the live store instance`, 'SESSION_NOT_LIVE') + return source + } + } export default SessionStore diff --git a/packages/core/session/src/json.ts b/packages/core/session/src/json.ts index 47197b7b90..22303c6c61 100644 --- a/packages/core/session/src/json.ts +++ b/packages/core/session/src/json.ts @@ -40,6 +40,10 @@ export type JsonValue = null | boolean | number | string | JsonValue[] | { [key: * hiding under a symbol/non-enumerable key cannot make the round-trip lossy. * Getters are invoked during the check (again as `JSON.stringify` would), so the * contract is for plain data records, not objects with side-effecting accessors. + * @param value - the candidate event data to test. + * @param seen - objects on the current descent path, for circular-reference + * detection; the recursion threads it — callers omit it. + * @returns true when `value` survives a JSON round-trip losslessly. */ export function isJsonValue(value: unknown, seen: Set = new Set()): boolean { if (value === null) return true diff --git a/packages/core/session/src/repair.ts b/packages/core/session/src/repair.ts index b894ef26cb..cb780da013 100644 --- a/packages/core/session/src/repair.ts +++ b/packages/core/session/src/repair.ts @@ -54,6 +54,8 @@ import type { SessionEvent } from './types.ts' * Only the LAST turn can be open: the invariants plugin guarantees a `turn/end` * before any later `turn/start`, so an interior open turn is impossible in a * valid committed log. Likewise at most one step is open within that turn. + * @param events - the loaded durable log to scan (a valid committed prefix, possibly with a crash tail). + * @returns the synthetic closer events to append after `events`, in order; empty when the log is already balanced. */ export function interruptedTurnClosers(events: readonly SessionEvent[]): SessionEvent[] { let openTurn: number | null = null diff --git a/packages/core/session/src/surface.ts b/packages/core/session/src/surface.ts index 37843970a2..7219856bdb 100644 --- a/packages/core/session/src/surface.ts +++ b/packages/core/session/src/surface.ts @@ -29,6 +29,8 @@ const SURFACE_EVENT_TYPES = new Set([ * surface-eligible event that is MISSING its mandatory marker (e.g. validating * a seed/load log); use {@link isSurfaceEvent} to narrow to a fully-formed * {@link SurfaceEvent} with `surfaceOp` present. + * @param type - the event type string to test. + * @returns true when the type is one of the five message-producing types. */ export function isSurfaceEligibleType(type: string): boolean { return SURFACE_EVENT_TYPES.has(type) @@ -38,6 +40,8 @@ export function isSurfaceEligibleType(type: string): boolean { * Narrow a {@link SessionEvent} to {@link SurfaceEvent}: checks that the * event's `type` is surface-eligible AND that `surfaceOp` is present. * The narrowed type has mandatory {@link SurfaceOp}. + * @param event - the event to narrow. + * @returns true when the event is surface-eligible and carries its `surfaceOp` marker. */ export function isSurfaceEvent(event: SessionEvent): event is SurfaceEvent { if (!SURFACE_EVENT_TYPES.has(event.type)) return false diff --git a/packages/core/session/src/tool-pairing.ts b/packages/core/session/src/tool-pairing.ts index 638daaf654..6ea3042bd7 100644 --- a/packages/core/session/src/tool-pairing.ts +++ b/packages/core/session/src/tool-pairing.ts @@ -74,6 +74,12 @@ function nodeDelta(event: SessionEvent): number { * surface successor (`SurfaceNode.next`), or `null` when `end` is the tail — * for the cut after `end`. * + * @param nodes - the surface linked list in head→tail order. + * @param events - the session log each node's `seq` indexes into. + * @param beforeSeq - names the cut (the node it sits immediately before); + * `null` — or any seq not on the surface — means the after-tail cut. + * @returns true when every `tool-call` before the cut is answered before it + * (the unanswered-call depth at the cut is zero). * @throws if the surface prefix drives the unanswered-call depth negative — a * `tool/result` with no preceding open `tool-call` on the surface. That is a * corrupt surface (a structural invariant violation), surfaced loudly here diff --git a/packages/core/session/src/types.ts b/packages/core/session/src/types.ts index ec227e9884..e2e0e1b216 100644 --- a/packages/core/session/src/types.ts +++ b/packages/core/session/src/types.ts @@ -4,7 +4,11 @@ import type { CallId, ContentBlock, LlmCallConfig, MessageSource, StreamChunk, T /** Identifies one session in the store (and its persistence artifacts). */ export type SessionId = Branded<'SessionId'> -/** Brand a string as a {@link SessionId}. */ +/** + * Brand a string as a {@link SessionId}. + * @param id - the raw session id string. + * @returns the same string, branded (a compile-time cast — no runtime cost). + */ export function SessionId(id: string): SessionId { return id as SessionId } @@ -102,6 +106,7 @@ export interface TurnTriggerMap { injection: { kind: 'injection'; source: MessageSource } } +/** The union over {@link TurnTriggerMap} — what started a turn; plugins extend it by merging variants into the map. */ export type TurnTrigger = TurnTriggerMap[keyof TurnTriggerMap] /** @@ -156,6 +161,7 @@ export interface TurnEndReasonMap { interrupted: { kind: 'interrupted' } } +/** The union over {@link TurnEndReasonMap} — why a turn ended; plugins extend it by merging variants into the map. */ export type TurnEndReason = TurnEndReasonMap[keyof TurnEndReasonMap] /** @@ -361,6 +367,7 @@ export interface SessionEventMap { 'request/header-delta': { system?: SystemDelta; tools?: ToolsDelta; config?: LlmCallConfig } } +/** The appendable event-type keys of {@link SessionEventMap}, plugin-merged extensions included. */ export type SessionEventType = keyof SessionEventMap /** diff --git a/packages/core/session/tests/fork.spec.ts b/packages/core/session/tests/fork.spec.ts new file mode 100644 index 0000000000..25328294cf --- /dev/null +++ b/packages/core/session/tests/fork.spec.ts @@ -0,0 +1,240 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import { CallId } from '@deepseek-ai/dsh-llm' +import SessionStore, { Session, SessionForkError, SessionId } from '@deepseek-ai/dsh-session' +import type { SessionEvent, TurnEndReason } from '@deepseek-ai/dsh-session' + +async function setup(): Promise<{ ctx: Context; sessions: SessionStore }> { + const ctx = new Context() + await ctx.plugin(SessionStore) + return { ctx, sessions: ctx.sessions } +} + +function appendClosedTurn( + session: Session, + turn: number, + text = `hello ${turn}`, + reason: TurnEndReason = { kind: 'completed' }, +): void { + session.append('turn/start', { turn, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { + content: [{ type: 'text', text }], + source: { kind: 'user' }, + }, { surfaceOp: 'append' }) + session.append('turn/end', { turn, reason }) +} + +function appendOpenTurn(session: Session, turn: number): void { + session.append('turn/start', { turn, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { + content: [{ type: 'text', text: `open ${turn}` }], + source: { kind: 'user' }, + }, { surfaceOp: 'append' }) +} + +function firstUserMessage(events: readonly SessionEvent[]): SessionEvent<'user/message'> { + const event = events.find((e): e is SessionEvent<'user/message'> => e.type === 'user/message') + if (event === undefined) throw new Error('missing user/message') + return event +} + +function lastSeq(session: Session): number { + const event = session.events.at(-1) + if (event === undefined) throw new Error('missing last event') + return event.seq +} + +describe('SessionStore.fork', () => { + it('forks an empty live session as an empty child with lineage metadata', async () => { + const { ctx, sessions } = await setup() + const source = ctx.sessions.create(SessionId('empty-parent'), { meta: { cwd: '/workspace' } }) + + const child = sessions.fork(source, undefined, SessionId('empty-child')) + + expect(child.events).toEqual([]) + expect(child.header).toMatchObject({ + id: SessionId('empty-child'), + cwd: '/workspace', + parentSession: SessionId('empty-parent'), + seedLength: 0, + }) + }) + + it('forks the latest completed boundary by default and deep-clones seed events', async () => { + const { ctx, sessions } = await setup() + const source = ctx.sessions.create(SessionId('parent'), { meta: { cwd: '/workspace' } }) + appendClosedTurn(source, 1, 'hello') + + const child = sessions.fork(SessionId('parent'), undefined, SessionId('child')) + + expect(child.events).toEqual(source.events) + expect(child.events).not.toBe(source.events) + expect(child.events[1]).not.toBe(source.events[1]) + firstUserMessage(child.events).data.content[0] = { type: 'text', text: 'child mutation' } + expect(firstUserMessage(source.events).data.content).toEqual([{ type: 'text', text: 'hello' }]) + expect(child.header).toMatchObject({ + id: SessionId('child'), + cwd: '/workspace', + parentSession: SessionId('parent'), + seedLength: source.events.length, + }) + }) + + it('forks from an earlier turn boundary even when the source currently has an open tail', async () => { + const { ctx, sessions } = await setup() + const source = ctx.sessions.create(SessionId('parent'), { meta: { cwd: '/workspace' } }) + appendClosedTurn(source, 1, 'first') + const firstBoundary = lastSeq(source) + appendClosedTurn(source, 2, 'second') + appendOpenTurn(source, 3) + + const child = sessions.fork(source, firstBoundary, SessionId('child-from-first')) + + expect(child.events).toEqual(source.events.slice(0, firstBoundary + 1)) + expect(child.header.seedLength).toBe(firstBoundary + 1) + expect(child.deriveMessages()).toEqual([{ role: 'user', content: [{ type: 'text', text: 'first' }] }]) + }) + + it('accepts every turn/end reason as an explicit fork boundary', async () => { + const { ctx, sessions } = await setup() + const reasons: TurnEndReason[] = [ + { kind: 'completed' }, + { kind: 'aborted', reason: 'cancelled by user' }, + { kind: 'error', step: 1, message: 'model failed', code: 'MODEL' }, + { kind: 'disposed' }, + { kind: 'max-tokens' }, + { kind: 'interrupted' }, + ] + + for (const reason of reasons) { + const source = ctx.sessions.create(SessionId(`parent-${reason.kind}`)) + appendClosedTurn(source, 1, reason.kind, reason) + + const child = sessions.fork(source, lastSeq(source), SessionId(`child-${reason.kind}`)) + + expect(child.events.at(-1)?.type).toBe('turn/end') + expect(child.header.seedLength).toBe(source.events.length) + } + }) + + it('rejects invalid boundaries before creating a child', async () => { + const { ctx, sessions } = await setup() + const empty = ctx.sessions.create(SessionId('empty')) + expect(() => sessions.fork(empty, 0, SessionId('empty-child'))) + .toThrow(new SessionForkError('fork boundary 0 does not exist in session "empty" (last seq: none)', 'INVALID_BOUNDARY')) + expect(ctx.sessions.get(SessionId('empty-child'))).toBeUndefined() + + const source = ctx.sessions.create(SessionId('parent')) + appendClosedTurn(source, 1) + expect(() => sessions.fork(source, -1, SessionId('negative'))) + .toThrow(/non-negative safe integer/) + expect(() => sessions.fork(source, 0.5, SessionId('fraction'))) + .toThrow(/non-negative safe integer/) + expect(() => sessions.fork(source, Number.MAX_SAFE_INTEGER + 1, SessionId('unsafe'))) + .toThrow(/non-negative safe integer/) + expect(() => sessions.fork(source, source.seq, SessionId('past-end'))) + .toThrow(new SessionForkError(`fork boundary ${source.seq} does not exist in session "parent" (last seq: ${source.seq - 1})`, 'INVALID_BOUNDARY')) + }) + + it('rejects a corrupted live source whose array index no longer matches event seq', async () => { + const { ctx, sessions } = await setup() + const source = ctx.sessions.create(SessionId('corrupt-parent')) + appendClosedTurn(source, 1) + const mutableLog = (source as unknown as { log: SessionEvent[] }).log + mutableLog[2] = { ...mutableLog[2]!, seq: 99 } + + expect(() => sessions.fork(source, 2, SessionId('corrupt-child'))) + .toThrow(new SessionForkError('fork boundary 2 does not match a contiguous event seq in session "corrupt-parent"', 'INVALID_BOUNDARY')) + expect(ctx.sessions.get(SessionId('corrupt-child'))).toBeUndefined() + }) + + it('rejects an unknown live session id', async () => { + const { sessions } = await setup() + + expect(() => sessions.fork(SessionId('missing'))) + .toThrow(new SessionForkError('session "missing" not found', 'SESSION_NOT_FOUND')) + }) + + it('rejects a detached Session object that is not live in ctx.sessions', async () => { + const { sessions } = await setup() + const detached = new Session(SessionId('detached')) + + expect(() => sessions.fork(detached)) + .toThrow(new SessionForkError('session "detached" not found', 'SESSION_NOT_FOUND')) + }) + + it('rejects a stale Session object whose id is live on a different instance', async () => { + const { ctx, sessions } = await setup() + ctx.sessions.create(SessionId('same-id')) + const stale = new Session(SessionId('same-id')) + + expect(() => sessions.fork(stale)) + .toThrow(new SessionForkError('session "same-id" is not the live store instance', 'SESSION_NOT_LIVE')) + }) + + it('rejects selected slices whose boundary is inside an open turn', async () => { + const { ctx, sessions } = await setup() + const cases: [string, (session: Session) => number][] = [ + ['turn/start', (session) => { + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + return lastSeq(session) + }], + ['step/start', (session) => { + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + return lastSeq(session) + }], + ['user/message', (session) => { + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { content: [{ type: 'text', text: 'open' }], source: { kind: 'user' } }, { surfaceOp: 'append' }) + return lastSeq(session) + }], + ['assistant/message', (session) => { + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + session.append('assistant/message', { turn: 1, step: 1, content: [{ type: 'text', text: 'partial' }] }, { surfaceOp: 'append' }) + return lastSeq(session) + }], + ['tool/call', (session) => { + const callId = CallId('call-open') + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + session.append('assistant/message', { + turn: 1, + step: 1, + content: [{ type: 'tool-call', id: callId, name: 'bash', arguments: '{}' }], + }, { surfaceOp: 'append' }) + session.append('tool/call', { turn: 1, step: 1, callId, name: 'bash', arguments: '{}' }) + return lastSeq(session) + }], + ] + + for (const [lastType, build] of cases) { + const source = ctx.sessions.create(SessionId(`open-${lastType}`)) + const boundary = build(source) + + expect(() => sessions.fork(source, boundary)) + .toThrow(new SessionForkError(`fork boundary ${boundary} in session "open-${lastType}" must be turn/end, got ${lastType}`, 'OPEN_TURN')) + } + }) + + it('rejects a child session id that is already live with a typed fork error', async () => { + const { ctx, sessions } = await setup() + const source = ctx.sessions.create(SessionId('parent')) + appendClosedTurn(source, 1) + ctx.sessions.create(SessionId('child')) + + expect(() => sessions.fork(source, undefined, SessionId('child'))) + .toThrow(new SessionForkError('session "child" already exists', 'SESSION_ALREADY_EXISTS')) + }) + + it('rejects a duplicate child session id before validating the boundary', async () => { + const { ctx, sessions } = await setup() + const source = ctx.sessions.create(SessionId('open-parent')) + source.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + ctx.sessions.create(SessionId('child')) + + expect(() => sessions.fork(source, undefined, SessionId('child'))) + .toThrow(new SessionForkError('session "child" already exists', 'SESSION_ALREADY_EXISTS')) + }) +}) diff --git a/packages/core/session/tests/session.spec.ts b/packages/core/session/tests/session.spec.ts index 4ff13940bc..f338b635f3 100644 --- a/packages/core/session/tests/session.spec.ts +++ b/packages/core/session/tests/session.spec.ts @@ -61,8 +61,10 @@ describe('Session', () => { it('replays identically from a seeded event log', () => { const original = new Session(SessionId('s3')) + original.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) original.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }, { surfaceOp: 'append' }) original.append('assistant/message', { turn: 1, step: 1, content: [{ type: 'text', text: 'a' }] }, { surfaceOp: 'append' }) + original.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) const replayed = new Session(SessionId('s3-replay'), [...original.events]) expect(replayed.deriveMessages()).toEqual(original.deriveMessages()) @@ -423,7 +425,9 @@ describe('todo/write event', () => { it('round-trips through a seeded replay identically (durable, no surfaceOp needed)', () => { const original = new Session(SessionId('t4')) + original.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) original.append('todo/write', { todos: [{ content: 'only', status: 'completed' }] }) + original.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) // Seeding a non-surface event with no surfaceOp must not throw. const replayed = new Session(SessionId('t4-replay'), [...original.events]) expect(replayed.events.findLast(e => e.type === 'todo/write')!.data.todos) diff --git a/packages/core/system-prompt/README.md b/packages/core/system-prompt/README.md index be705de03e..704cd8e80f 100644 --- a/packages/core/system-prompt/README.md +++ b/packages/core/system-prompt/README.md @@ -7,15 +7,16 @@ System prompt assembly registry. Plugins contribute ordered text sections, tool- | Key | Default | Meaning | |---|---|---| | `persona` | `''` | The deployment persona: the ONE deployment-authored prompt fragment, rendered as the order-0 `deployment:persona` section and shared by every agent in the context (subagents included). A template — complete `{{…}}` groups are interpreted strictly against the registered variables (the shipped loop registers `{{model}}`/`{{cwd}}`), with no escape syntax for literal braces yet. Empty ⇒ the section is dropped at render. | +| `toolOrder` | — | Explicit model-facing tool order, as a list of `ToolSchema.name`s with one `''` rest entry (`TOOL_ORDER_REST`): listed tools take their listed position, unlisted tools land at the rest entry in lexicographic name order. Absent ⇒ plain lexicographic name order. Applied to the collected tools BEFORE the `system-prompt/assemble` waterfall — like the sections' `order` sort, it canonicalizes what the registry contributed (registration order is a plugin-load artifact), and a waterfall listener that mutates the list owns the determinism of what it emits. Misconfiguration fails loud: a list without exactly one rest entry, or with duplicates, throws at load; a listed name with no registered tool rejects every `assemble()`; a tool provider returning the reserved rest-entry name also rejects. Under the shipped loop the turn fails before any model request. Why a central list and not per-plugin weights: [Explicit model-facing tool order](../../../docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md). | ## Service: `SystemPrompt` (ctx key: `systemPrompt`) ### Public API - `ctx.systemPrompt.section(section: PromptSection): () => void` Contribute a section. Duplicate names throw. Disposed with the calling fiber. -- `ctx.systemPrompt.tools(provider: () => ToolSchema[]): () => void` Contribute tool schemas (evaluated at each assembly). Disposed with the calling fiber. +- `ctx.systemPrompt.tools(provider: () => ToolSchema[]): () => void` Contribute tool schemas (evaluated at each assembly). A provider must not return a schema named `TOOL_ORDER_REST`; that name is reserved for `toolOrder`'s rest entry. Disposed with the calling fiber. - `ctx.systemPrompt.variable(name: string, provider: (context) => string | undefined): () => void` Contribute a prompt variable, referenced from section text as `{{name}}`. Duplicate or unreferenceable names throw; `undefined` means "no value for this assembly". Disposed with the calling fiber. -- `ctx.systemPrompt.assemble(context?: AssembleContext): Promise` Assemble the prompt for one caller. Runs through the `system-prompt/assemble` waterfall. +- `ctx.systemPrompt.assemble(context?: AssembleContext): Promise` Assemble the prompt for one caller. Runs through the `system-prompt/assemble` waterfall. Rejects when a configured `toolOrder` names a tool no provider contributed, or when a provider returns the reserved rest-entry name. ### Events diff --git a/packages/core/system-prompt/src/index.ts b/packages/core/system-prompt/src/index.ts index c970c66b30..81ca8087bb 100644 --- a/packages/core/system-prompt/src/index.ts +++ b/packages/core/system-prompt/src/index.ts @@ -88,7 +88,8 @@ export interface AssembledSection { * * Tool schemas are part of the assembly by design: "what the model is told it * can do" is one coherent thing managed here, even though adapters transmit - * `tools` as a separate wire field rather than prompt text. + * `tools` as a separate wire field rather than prompt text. They arrive in + * the canonical model-facing order (see {@link Config.toolOrder}). * * `variables` carries every registered prompt variable resolved against this * assembly's context — key present means registered, `undefined` value means @@ -110,6 +111,71 @@ const VARIABLE_NAME = /^[a-z][a-z0-9_]*$/ /** A complete `{{...}}` reference group at the scan position (validated after). */ const GROUP_AT = /^\{\{([^{}]*)\}\}/ +/** + * The rest entry for {@link Config.toolOrder}: the position where registered + * tools not named in the list are inserted (in lexicographic name order). + * Reserved: collected tool schemas using this name are rejected before + * ordering, so the marker can never collide with a real model-facing tool. + */ +export const TOOL_ORDER_REST = '' + +/** + * Validate a configured tool-order list's shape at service construction: + * the {@link TOOL_ORDER_REST} rest entry exactly once, no duplicate names. + * Returns the list (or undefined when unconfigured); throws otherwise, + * failing the service at load — a bad order config must never reach an + * assembly. Whether every listed name matches a registered tool is checked + * at each assembly instead ({@link orderTools}): tool plugins register after + * this service constructs, so the tool set does not exist yet here. + */ +function validateToolOrder(toolOrder: string[] | undefined): string[] | undefined { + if (toolOrder === undefined) return undefined + const seen = new Set() + for (const name of toolOrder) { + if (seen.has(name)) throw new Error(`toolOrder lists "${name}" more than once`) + seen.add(name) + } + if (!seen.has(TOOL_ORDER_REST)) { + throw new Error(`toolOrder must contain the "${TOOL_ORDER_REST}" rest entry (where unlisted tools are inserted)`) + } + return toolOrder +} + +/** + * Order collected tool schemas by the validated policy: with no configured + * list, plain lexicographic name order; with one, listed names take their + * listed position and every unlisted tool lands at the + * {@link TOOL_ORDER_REST} rest entry in lexicographic name order. A listed + * name with no collected tool throws — misconfiguration fails loud, and this + * is the earliest moment the registered tool set exists to check against + * (tool plugins register after the service constructs, so load time is too + * early): the assembly rejects, failing the caller's turn before any model + * request. Never drops a tool, and both sorts are stable, so tools sharing a + * name keep their collection order. + */ +function orderTools(tools: ToolSchema[], toolOrder: string[] | undefined): ToolSchema[] { + const reserved = tools.find(tool => tool.name === TOOL_ORDER_REST) + if (reserved !== undefined) { + throw new Error(`tool provider returned reserved tool name "${TOOL_ORDER_REST}" (reserved for toolOrder's rest entry)`) + } + if (toolOrder === undefined) return tools.sort(compareToolNames) + const registered = new Set(tools.map(tool => tool.name)) + const unknown = toolOrder.filter(name => name !== TOOL_ORDER_REST && !registered.has(name)) + if (unknown.length > 0) { + throw new Error(`toolOrder lists unregistered tool${unknown.length > 1 ? 's' : ''} ${unknown.map(name => `"${name}"`).join(', ')}; registered tools: ${[...registered].sort().join(', ') || '(none)'}`) + } + const listed = new Set(toolOrder) + const rest = tools.filter(tool => !listed.has(tool.name)).sort(compareToolNames) + return toolOrder.flatMap(name => + name === TOOL_ORDER_REST ? rest : tools.filter(tool => tool.name === name)) +} + +/** Lexicographic (code-unit) name comparison — locale-independent, so the order is identical on every machine. */ +function compareToolNames(a: ToolSchema, b: ToolSchema): number { + return a.name < b.name ? -1 : a.name > b.name ? 1 : 0 +} + +/** Plugin config: the deployment-authored fragment of the system prompt (see {@link Config.persona} for its contract). */ export interface Config { /** * The deployment's persona — the ONE deployment-authored fragment of the @@ -124,6 +190,29 @@ export interface Config { * deployment opens with the harness identity alone. */ persona?: string + /** + * Explicit model-facing tool order, as a list of `ToolSchema.name`s: listed + * tools take their listed position, and tools absent from the list are + * inserted at the {@link TOOL_ORDER_REST} (`''`) entry in + * lexicographic name order. A configured list must contain the rest entry + * exactly once, no duplicate names, and no name without a registered tool — + * a misconfigured order blocks work instead of silently reaching a model + * request: shape violations throw at load, and an unregistered name rejects + * every assembly. `TOOL_ORDER_REST` is reserved for the list marker and may + * not be a collected tool name; such a provider output also rejects the + * assembly. The single assembly-time validation rejects either failure + * before any model request — the earliest moment the registered tool set + * exists to check against, since tool plugins register after this service + * constructs. When omitted, tools are ordered lexicographically by name. + * Applied to the tools + * {@link SystemPrompt.assemble} collects, BEFORE the + * `system-prompt/assemble` waterfall — like the sections' `order` sort, it + * canonicalizes what the registry contributed (registration order is a + * plugin-load artifact); a waterfall listener that mutates the tool list + * owns the determinism of what it emits. Rationale (and why not per-plugin + * weights): docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md. + */ + toolOrder?: string[] } /** @@ -138,6 +227,10 @@ export interface Config { * while a `}}` still follows (e.g. `{{{model}}}`, `{{a{b}}`) all throw. A * lone `{{` with no `}}` anywhere after it is ordinary prose and passes * through verbatim. Substituted values are never re-scanned. + * @param assembly - the assembly to render (typically the awaited result of + * {@link SystemPrompt.assemble}); only `sections` and `variables` are read. + * @returns the full system prompt text; `''` when every section renders empty + * (the caller then sends no system prompt at all). */ export function renderPrompt(assembly: PromptAssembly): string { return assembly.sections @@ -198,14 +291,23 @@ function interpolate(section: AssembledSection, variables: Record = z.object({ persona: z.string().default(''), + // A schemastery array defaults to [] when omitted, but an omitted + // toolOrder must stay absent ("lexicographic order"), not become an + // explicitly-configured empty list (which is invalid — it lacks the + // rest entry). Forcing the default to undefined keeps the key out of the + // validated config; the cast is needed because .default() expects the + // array type. + toolOrder: z.array(z.string()).default(undefined as unknown as string[]), }) private sections: PromptSection[] = [] private toolProviders: (() => ToolSchema[])[] = [] private variableProviders = new Map string | undefined>() + private readonly toolOrder: string[] | undefined constructor(ctx: Context, public config: Config) { super(ctx, 'systemPrompt') + this.toolOrder = validateToolOrder(config.toolOrder) // The harness-owned openers. They live HERE (not on the loop plugin) so a // deployment that swaps in a different loop keeps them: the identity is a // harness fact stated ahead of everything, and the persona is the @@ -261,7 +363,10 @@ export class SystemPrompt extends Service { /** * Contribute a tool-schema provider that is evaluated at each assembly * call (so it can reflect the live registry state). The provider is - * removed when the calling fiber is disposed. Emits `system-prompt/change`. + * removed when the calling fiber is disposed. A provider must not return a + * schema named {@link TOOL_ORDER_REST}; that name is reserved for + * {@link Config.toolOrder}'s rest entry and rejects the assembly. Emits + * `system-prompt/change`. * @param provider - evaluated at every {@link assemble} for fresh schemas. * @returns the disposer that removes the provider. */ @@ -318,19 +423,28 @@ export class SystemPrompt extends Service { /** * Assemble the current prompt for one caller: section texts are resolved - * against `context` and sorted by order, tools collected from all - * providers, and every registered variable resolved against `context` into - * `assembly.variables`. Tool schemas are deep-cloned because adapters and - * request waterfalls may mutate schema objects. Runs through the - * `system-prompt/assemble` waterfall, giving listeners the opportunity to - * mutate or replace the assembly before it reaches the model. Await the - * result before reading the assembly values — waterfall listeners may be - * async. Interpolation happens later, in {@link renderPrompt}. + * against `context` and sorted by order, tools collected from all providers + * and put in the canonical model-facing order ({@link Config.toolOrder}, or + * lexicographic name order when unconfigured — provider registration order + * is a plugin-load artifact and never reaches the assembly; a configured + * order naming a tool no provider contributed rejects the assembly), and every + * registered variable resolved against `context` into `assembly.variables`. + * Tool schemas are deep-cloned because adapters and request waterfalls may + * mutate schema objects. Runs through the `system-prompt/assemble` + * waterfall, giving listeners the opportunity to mutate or replace the + * assembly before it reaches the model — like the sections' `order` sort, + * tool canonicalization happens on the initial assembly, and a listener + * owns the determinism of whatever it emits. Await the result before + * reading the assembly values — waterfall listeners may be async. + * Interpolation happens later, in {@link renderPrompt}. * @param context - what this assembly is for (defaults to an empty context; * see {@link AssembleContext}). * @returns the assembly after the waterfall has run. */ - assemble(context: AssembleContext = {}): Promise { + // async so the misconfigured-toolOrder throw in orderTools surfaces as a + // rejection: a Promise-returning method must not throw synchronously + // (`assemble().catch(...)` would miss it). + async assemble(context: AssembleContext = {}): Promise { const variables: Record = {} for (const [name, provider] of this.variableProviders) { variables[name] = provider(context) @@ -343,8 +457,10 @@ export class SystemPrompt extends Service { text: typeof section.text === 'function' ? section.text(context) : section.text, })) .sort((a, b) => a.order - b.order), - tools: this.toolProviders.flatMap(provider => - provider().map(tool => ({ ...tool, parameters: structuredClone(tool.parameters) }))), + tools: orderTools( + this.toolProviders.flatMap(provider => + provider().map(tool => ({ ...tool, parameters: structuredClone(tool.parameters) }))), + this.toolOrder), variables, } return this.ctx.waterfall(this, 'system-prompt/assemble', assembly, context, () => Promise.resolve(assembly)) diff --git a/packages/core/system-prompt/tests/tool-order.spec.ts b/packages/core/system-prompt/tests/tool-order.spec.ts new file mode 100644 index 0000000000..02cc99b2d7 --- /dev/null +++ b/packages/core/system-prompt/tests/tool-order.spec.ts @@ -0,0 +1,115 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import SystemPrompt, { PromptAssembly, TOOL_ORDER_REST } from '@deepseek-ai/dsh-system-prompt' +import type { ToolSchema } from '@deepseek-ai/dsh-llm' + +function tool(name: string, description = name): ToolSchema { + return { name, description, parameters: { type: 'object', properties: {} } } +} + +async function mount(config: { persona?: string; toolOrder?: string[] } = {}): Promise { + const ctx = new Context() + await ctx.plugin(SystemPrompt, config) + return ctx +} + +function names(assembly: PromptAssembly): string[] { + return assembly.tools.map(t => t.name) +} + +describe('SystemPrompt tool order', () => { + // The ONE place the public constant's value is pinned; everything else + // (tests and deployment configs alike) references TOOL_ORDER_REST. + it('exports the rest entry as ""', () => { + expect(TOOL_ORDER_REST).toBe('') + }) + + it('assembles tools in lexicographic name order when no toolOrder is configured', async () => { + const ctx = await mount() + ctx.systemPrompt.tools(() => [tool('charlie'), tool('alpha')]) + ctx.systemPrompt.tools(() => [tool('bravo')]) + expect(names(await ctx.systemPrompt.assemble())).toEqual(['alpha', 'bravo', 'charlie']) + }) + + it('assembles the same order regardless of provider registration order', async () => { + const forward = await mount() + forward.systemPrompt.tools(() => [tool('alpha')]) + forward.systemPrompt.tools(() => [tool('zulu')]) + const backward = await mount() + backward.systemPrompt.tools(() => [tool('zulu')]) + backward.systemPrompt.tools(() => [tool('alpha')]) + expect(names(await forward.systemPrompt.assemble())).toEqual(['alpha', 'zulu']) + expect(names(await backward.systemPrompt.assemble())).toEqual(['alpha', 'zulu']) + }) + + it('applies a configured toolOrder: listed positions, rest at the rest entry lexicographically', async () => { + const ctx = await mount({ toolOrder: ['todo_write', TOOL_ORDER_REST, 'bash'] }) + ctx.systemPrompt.tools(() => [tool('bash'), tool('echo_b'), tool('todo_write'), tool('echo_a')]) + expect(names(await ctx.systemPrompt.assemble())).toEqual(['todo_write', 'echo_a', 'echo_b', 'bash']) + }) + + it('rejects the assembly when toolOrder names a tool that is not registered (misconfiguration blocks work)', async () => { + const ctx = await mount({ toolOrder: ['todo_write', 'ghost', TOOL_ORDER_REST, 'wraith'] }) + ctx.systemPrompt.tools(() => [tool('bash'), tool('todo_write')]) + await expect(ctx.systemPrompt.assemble()).rejects.toThrow( + 'toolOrder lists unregistered tools "ghost", "wraith"; registered tools: bash, todo_write') + }) + + it('names the single unregistered tool when no tools are registered at all', async () => { + const ctx = await mount({ toolOrder: ['ghost', TOOL_ORDER_REST] }) + await expect(ctx.systemPrompt.assemble()).rejects.toThrow( + 'toolOrder lists unregistered tool "ghost"; registered tools: (none)') + }) + + it.each([ + ['without an explicit toolOrder', undefined], + ['with only the rest entry configured', [TOOL_ORDER_REST]], + ])('rejects a provider tool named like the reserved rest entry %s', async (_case, toolOrder) => { + const ctx = await mount(toolOrder === undefined ? {} : { toolOrder }) + ctx.systemPrompt.tools(() => [tool(TOOL_ORDER_REST)]) + await expect(ctx.systemPrompt.assemble()).rejects.toThrow( + `tool provider returned reserved tool name "${TOOL_ORDER_REST}"`) + }) + + it('keeps collection order between tools that share a name (stable sort)', async () => { + const ctx = await mount() + ctx.systemPrompt.tools(() => [tool('dup', 'first'), tool('anchor'), tool('dup', 'second')]) + const assembly = await ctx.systemPrompt.assemble() + expect(assembly.tools.map(t => t.description)).toEqual(['anchor', 'first', 'second']) + }) + + it('canonicalizes BEFORE the assemble waterfall: listeners see the ordered list and own their own edits', async () => { + const ctx = await mount() + ctx.systemPrompt.tools(() => [tool('zulu'), tool('alpha')]) + let seen: string[] | undefined + ctx.on('system-prompt/assemble', function (assembly, _context, next) { + seen = assembly.tools.map(t => t.name) + // A listener-appended tool is NOT re-sorted — same contract as sections: + // canonicalization applies to what the registry contributed, and a + // listener owns the determinism of what it emits. + assembly.tools.push(tool('aardvark')) + return next() + }) + const assembly = await ctx.systemPrompt.assemble() + expect(seen).toEqual(['alpha', 'zulu']) + expect(names(assembly)).toEqual(['alpha', 'zulu', 'aardvark']) + }) + + it.each([ + ['an empty list', []], + ['a list without the rest entry', ['bash', 'todo_write']], + ])('rejects %s at load (the rest entry is required)', async (_case, toolOrder) => { + await expect(new Context().plugin(SystemPrompt, { toolOrder })).rejects.toThrow(`must contain the "${TOOL_ORDER_REST}" rest entry`) + }) + + it.each([ + ['a duplicate tool name', ['bash', 'bash', TOOL_ORDER_REST]], + ['a duplicate rest entry', [TOOL_ORDER_REST, 'bash', TOOL_ORDER_REST]], + ])('rejects %s at load', async (_case, toolOrder) => { + await expect(new Context().plugin(SystemPrompt, { toolOrder })).rejects.toThrow('more than once') + }) + + it('throws from direct construction too', () => { + expect(() => new SystemPrompt(new Context(), { toolOrder: ['bash'] })).toThrow('rest entry') + }) +}) diff --git a/packages/core/tools/README.md b/packages/core/tools/README.md index 627e23f096..bb1603f68a 100644 --- a/packages/core/tools/README.md +++ b/packages/core/tools/README.md @@ -1,6 +1,6 @@ # dsh-tools -Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the allow/deny gate) → core dispatch → `tools/post-execute` (inspect/replace the result, attach context). +Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the allow/deny gate) → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context). ## Service: `ToolRegistry` (ctx key: `tools`) @@ -9,7 +9,7 @@ Tool registry and execution pipeline. Tool plugins register their schemas and ex - `ctx.tools.register(definition: ToolDefinition): () => void` Register a tool. Disposed with the calling fiber. - `ctx.tools.get(name: string): ToolDefinition | undefined` - `ctx.tools.schemas(): ToolSchema[]` Schemas of all registered tools (without the `execute` functions). The shipped tools' schemas are catalogued in [docs/tool-catalog.md](../../../docs/tool-catalog.md), generated by booting each tool plugin and harvesting this method (see [the tool-schema-catalog RFC](../../../docs/rfc/implemented/process/2026-07-02-tool-schema-catalog.md)). -- `ctx.tools.execute(exec: ToolExecution): Promise` Execute one tool call through the `tools/pre-execute` → dispatch → `tools/post-execute` pipeline. +- `ctx.tools.execute(exec: ToolExecution): Promise` Execute one tool call through the `tools/pre-execute` → `tools/execute` → `tools/post-execute` pipeline. ### Injected services @@ -20,12 +20,13 @@ Tool registry and execution pipeline. Tool plugins register their schemas and ex | Event | Mode | Purpose | |---|---|---| | `tools/pre-execute` | waterfall | Allow/deny gate BEFORE a tool runs (sandbox, permission, hooks); returns `PreToolDecision` | +| `tools/execute` | waterfall | Around-dispatch wrapper (timeout, retry, metrics): `(exec, next)` → the dispatched `ToolExecutionResult`; `next()` is dispatch-with-normalization | | `tools/post-execute` | waterfall | Inspect/replace the result AFTER a tool runs, attach context; returns `PostToolDecision` | | `tools/change` | emit | A tool was registered or unregistered | ### Key types -- `ToolDefinition` — `ToolSchema` + `execute(args, exec): Promise` (the bare array is the model-facing content; the object form additionally attaches an opaque, JSON-serializable `meta` presentation payload persisted on the `tool/result` event and handed back to `presentResult`), plus optional `presentCall(args)` / `presentResult(args, result)` for tool-owned UI presentation (see below). +- `ToolDefinition` — `ToolSchema` + `execute(args, exec): Promise` (the bare array is the model-facing content; the object form additionally attaches an opaque, JSON-serializable `meta` presentation payload persisted on the `tool/result` event and handed back to `presentResult`), plus optional `presentCall(args)` / `presentResult(args, result)` for tool-owned UI presentation (see below). It also carries an optional cooperative timeout budget `timeoutMs?: number` (ms) enforced by `@deepseek-ai/dsh-timeout-policy`, never sent to the model. - `ToolExecution` — one pending tool call: `{ callId, name, arguments, agent?, signal? }`. - `ToolExecutionResult` — outcome: `{ callId, content, isError, error?, additionalContext?, meta? }`. On failure with a `HarnessError`, `error: { name, code }` carries the structured failure class alongside the model-facing text (the loop forwards it onto the `tool/result` session event for retry/sandbox plugins and replay). `additionalContext` (a `HookContext`) ferries any `tools/post-execute` context up to the loop, which buffers it and appends it as a `context/message` after all `tool/result`s in the step. `meta` is the tool's opaque presentation payload from a successful `execute` (the object return form); the loop forwards it onto the `tool/result` session event for result-card rendering. - `PreToolDecision` — `{kind:'allow'}` | `{kind:'deny', reason}` | `{kind:'ask', reason?}`. Input rewrite (changing `arguments`) is deliberately NOT offered (it would desync the pre-execution audit/history/UI from what ran — its own proposed RFC); `ask` degrades to `deny` until the permission system lands. @@ -35,7 +36,7 @@ Tool registry and execution pipeline. Tool plugins register their schemas and ex ### Extension points - Tool plugins call `ctx.tools.register()` — schemas flow into the assembly automatically. -- `tools/pre-execute` is the allow/deny gate (sandbox, permission, hooks): listeners receive `(exec, next)` and call `next()` to delegate to the default (allow) or return a `PreToolDecision` to short-circuit; a `deny`/`ask` skips dispatch and yields an `isError` result. `tools/post-execute` is the inspect/transform seam: `(exec, result, next)` → a `PostToolDecision` that can replace content, block with feedback, or attach `additionalContext`. Core dispatch sits between them as plain code; the tool body keeps its own try/catch so a thrown tool still reaches `post-execute` as an `isError`. Both follow the typed-Decision idiom shared with the `agent/*` interception seams (see [`dsh-agent`](../agent/README.md)). +- `tools/pre-execute` is the allow/deny gate (sandbox, permission, hooks): listeners receive `(exec, next)` and call `next()` to delegate to the default (allow) or return a `PreToolDecision` to short-circuit; a `deny`/`ask` skips dispatch and yields an `isError` result. `tools/execute` is the around-dispatch seam (timeout, retry, metrics): listeners receive `(exec, next)` and call `next()` to delegate to core dispatch (returning its `ToolExecutionResult`, optionally wrapped), or return a replacement result to short-circuit dispatch; the base `next()` IS dispatch-with-normalization, so `await next()` already yields an `isError` result for a thrown/unknown tool (never a raw throw). A wrapper mutates `exec` in place before `next()` — e.g. replacing `exec.signal` with a per-call deadline — because cordis `next()` ignores passed arguments. `tools/post-execute` is the inspect/transform seam: `(exec, result, next)` → a `PostToolDecision` that can replace content, block with feedback, or attach `additionalContext`. Core dispatch is the base of the `tools/execute` waterfall; the tool body keeps its own try/catch so a thrown tool still reaches `post-execute` as an `isError`. All follow the typed-Decision idiom shared with the `agent/*` interception seams (see [`dsh-agent`](../agent/README.md)); `@deepseek-ai/dsh-timeout-policy` is the reference `tools/execute` wrapper. - MCP servers: one plugin per server, discover tools, call `ctx.tools.register()` with the server's schemas. ### Typed tool parameter schemas @@ -71,6 +72,14 @@ A `defineTool` tool also **validates the model-generated arguments against its ` See `defineTool`, `validateArgs`, `ToolArgsError`, `SchemaSpec`, `InferArgs`, and `schemaSpecToJsonSchema` in the public API for details. +`defineTool` also validates an optional `timeoutMs` at definition time when present: it must be a positive finite number, or the helper throws — the budget is attached to the produced `ToolDefinition` (for `@deepseek-ai/dsh-timeout-policy`) and never reaches the model. + +### Structured-output schema subset + +A separate vocabulary for callers that DEMAND a machine-readable value from an agent — the subagent seam's `SubagentStartRequest.outputSchema` (and, by extension, a workflow's `agent({ schema })`). Unlike `SchemaSpec` (the author-facing DSL for tool parameters), a `StructuredOutputSchema` is an object-rooted **raw JSON Schema subset** as data: it travels verbatim to the model as a forced tool's `parameters`, and the produced value is validated against it. + +The subset is deliberately narrow and REJECTS LOUD outside it — accepting a keyword the validator doesn't enforce would validate less than the schema promises (accepted-then-ignored). Supported: single-string `type` (`object`/`array`/`string`/`number`/`integer`/`boolean`/`null`; type arrays rejected), `properties`/`required`/`additionalProperties` (boolean; every `required` key must be declared), `items`, scalar-only `enum`/`const`; annotations (`description`/`title`/`default`/`examples`) are ignored but must still be JSON data. `assertSupportedOutputSchema(schema)` throws `OutputSchemaError` (`code: 'UNSUPPORTED_SCHEMA'`, listing every violation) for anything else; `validateStructuredValue(schema, value)` returns path-qualified violations (empty = valid, total — never throws). + ### Tool-owned UI presentation A tool owns how ITS calls render in a UI (an editor's tool-call card, a CLI log line) — a UI plugin must NOT special-case tool names. A `ToolDefinition` may declare two optional, pure, display-only methods that return a **`card`-tagged render intent** (a discriminated union — a tool declares its card kind once and a UI bridge switches on `card`): diff --git a/packages/core/tools/src/index.ts b/packages/core/tools/src/index.ts index dd0ed918db..2b038f1c94 100644 --- a/packages/core/tools/src/index.ts +++ b/packages/core/tools/src/index.ts @@ -1,9 +1,10 @@ /** * Tool registry and execution pipeline. Plugins register tools; the registry * feeds schemas into the system prompt, and `execute()` dispatches each call - * through `tools/pre-execute` (the allow/deny gate) → core dispatch → - * `tools/post-execute` (inspect/replace the result, attach context) for - * sandbox, permission, and hook plugins to gate or transform a call. + * through `tools/pre-execute` (the allow/deny gate) → `tools/execute` (an + * around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` + * (inspect/replace the result, attach context) for sandbox, permission, and hook + * plugins to gate or transform a call. * * @module @deepseek-ai/dsh-tools */ @@ -28,6 +29,16 @@ export { type JsonSchemaObject, } from './schema.ts' +export { + assertSupportedOutputSchema, + validateStructuredValue, + OutputSchemaError, + type StructuredOutputSchema, + type StructuredSchemaNode, + type StructuredSchemaType, + type StructuredScalar, +} from './json-schema.ts' + // The render-intent vocabulary a tool declares via `presentCall`/`presentResult` // lives in its own UI-facing module; re-export it so `@deepseek-ai/dsh-tools` // stays the single public surface for consumers (producers + the ACP bridge). @@ -64,17 +75,37 @@ declare module 'cordis' { * @mode waterfall */ 'tools/pre-execute'(this: ToolRegistry, exec: ToolExecution, next: () => Promise): Promise + /** + * Around-dispatch waterfall wrapping the registry's core tool dispatch, + * between the `tools/pre-execute` gate and the `tools/post-execute` seam. A + * listener receives `(exec, next)`: call `next()` to delegate to dispatch + * (returning its {@link ToolExecutionResult}, optionally wrapped), or return a + * replacement result without calling `next()` to short-circuit dispatch. The + * base `next()` IS the dispatch-with-normalization thunk — a thrown tool (or + * unknown tool) is already normalized to an `isError` result by the time a + * listener's `await next()` returns, so a wrapper never sees a raw throw from + * the tool body. This is the seam a timeout/retry/metrics plugin wraps: it can + * mutate `exec` (e.g. replace `exec.signal` with a per-call deadline) BEFORE + * `next()` and inspect the result AFTER. (Cordis `next()` ignores any passed + * arguments and re-invokes downstream with the shared payload, so a wrapper + * mutates `exec` in place rather than passing a new object to `next()`.) + * Multiple listeners compose by registration order — an outer one wraps the + * inner ones plus dispatch. + * @param exec - the allowed call about to dispatch (name, parsed arguments, caller agent, signal). + * @mode waterfall + */ + 'tools/execute'(this: ToolRegistry, exec: ToolExecution, next: () => Promise): Promise /** * Waterfall AFTER a tool runs — where hook plugins inspect the result and * accept it (optionally REPLACING the model-facing content, and/or attaching * `additionalContext` for the next request) or block it with corrective * `feedback` (Claude Code's `PostToolUse`). Listeners receive * `(exec, result, next)`: call `next()` to delegate to the default (accept - * unchanged), or return a {@link PostToolDecision} to override. The core tool - * dispatch sits between the two waterfalls as plain code, all inside - * `execute`'s outer try/catch (and the tool body keeps its own inner - * try/catch, so a thrown tool still reaches `post-execute` as an `isError` - * result). + * unchanged), or return a {@link PostToolDecision} to override. Core tool + * dispatch runs earlier as the base `next()` of the `tools/execute` + * waterfall, all inside `execute`'s outer try/catch (and the tool body keeps + * its own inner try/catch, so a thrown tool still reaches `post-execute` as an + * `isError` result). * @param exec - the call that just ran (name, parsed arguments, caller agent). * @param result - the dispatch outcome a listener may accept, replace, or block. * @mode waterfall @@ -107,6 +138,14 @@ export type ToolExecuteReturn = ContentBlock[] | { content: ContentBlock[]; meta /** A registered tool: its schema plus the execution function. */ export interface ToolDefinition extends ToolSchema { execute(args: unknown, exec: ToolExecution): Promise + /** + * Cooperative tool-call timeout budget in milliseconds. Omit for no deadline. + * Enforced by `@deepseek-ai/dsh-timeout-policy` (a `tools/execute` wrapper); it + * is NEVER sent to the model — `schemas()` whitelists only name/description/ + * parameters. Declaring it asserts this tool forwards `exec.signal` to a + * cooperative implementation that can reach quiescence when the signal aborts. + */ + timeoutMs?: number /** * Optional: how to present the PENDING state of one call in a UI, derived from * the call's `args` (parsed arguments, `unknown` — the tool validates/narrows @@ -261,7 +300,7 @@ function errorInfo(error: unknown): ToolErrorInfo | undefined { /** * Tool registry (`ctx.tools`): tool plugins register definitions; the agent - * loop executes calls through the `tools/pre-execute` → dispatch → + * loop executes calls through the `tools/pre-execute` → `tools/execute` → * `tools/post-execute` pipeline. The registry contributes its schemas into the * system-prompt assembly. */ @@ -335,18 +374,20 @@ export class ToolRegistry extends Service { } /** - * Execute one tool call through the `tools/pre-execute` → dispatch → - * `tools/post-execute` pipeline. The two waterfalls are the gate (allow/deny) - * and the inspect/transform seam; core dispatch sits between them as plain - * code. The whole thing is wrapped in one outer try/catch so a throwing - * listener (in either waterfall) becomes an `isError` result instead of - * failing the turn; the tool body ALSO keeps its own inner try/catch, so a - * thrown tool becomes an `isError` result that `post-execute` listeners can - * still inspect. If the tool is not registered, the result is an `isError` - * carrying a `UNKNOWN_TOOL` structured error. A thrown {@link HarnessError} - * surfaces its `{ name, code }` on the result. + * Execute one tool call through the `tools/pre-execute` → `tools/execute` + * (around dispatch) → `tools/post-execute` pipeline. `pre-execute` is the gate + * (allow/deny), `tools/execute` wraps core dispatch (a timeout/retry/metrics + * seam), and `post-execute` is the inspect/transform seam; core dispatch sits + * as the base `next()` of the `tools/execute` waterfall. The whole thing is + * wrapped in one outer try/catch so a throwing listener (in any waterfall) + * becomes an `isError` result instead of failing the turn; the tool body ALSO + * keeps its own inner try/catch, so a thrown tool becomes an `isError` result + * that `tools/execute` and `post-execute` listeners can still inspect. If the + * tool is not registered, the result is an `isError` carrying a `UNKNOWN_TOOL` + * structured error. A thrown {@link HarnessError} surfaces its `{ name, code }` + * on the result. * @param exec - the call to run (name, parsed arguments, caller agent, signal). - * @returns the final result after both waterfalls; failures resolve as + * @returns the final result after every waterfall; failures resolve as * `isError` results, never rejections. */ async execute(exec: ToolExecution): Promise { @@ -372,23 +413,30 @@ export class ToolRegistry extends Service { return await this.postExecute(exec, denied) } - // --- Core dispatch (plain code between the waterfalls). The tool body's - // own try/catch turns a throw into an isError result so post-execute can - // inspect it; an unknown tool routes through the same catch. --- - let result: ToolExecutionResult - try { - const tool = this.store.get(exec.name) - if (!tool) throw new ToolNotFoundError(exec.name) - // Normalize the two `execute` return shapes: a bare ContentBlock[] (no - // meta) or a { content, meta } object (a tool attaching a private - // presentation payload). An array IS the content; the object carries it. - const returned = await tool.execute(exec.arguments, exec) - const content = Array.isArray(returned) ? returned : returned.content - const meta = Array.isArray(returned) ? undefined : returned.meta - result = { callId: exec.callId, content, isError: false, ...meta !== undefined ? { meta } : {} } - } catch (error: unknown) { - result = toolErrorResult(exec.callId, error) - } + // --- Around-dispatch: tools/execute. The base `next` is the dispatch- + // with-normalization thunk — the tool body's own try/catch turns a throw + // into an isError result so a wrapper (and post-execute) can inspect it; + // an unknown tool routes through the same catch. A `tools/execute` listener + // (e.g. a timeout plugin) wraps this thunk: it may mutate `exec` before + // delegating and inspect the normalized result after. --- + const result = await this.ctx.waterfall( + this, 'tools/execute', exec, + async (): Promise => { + try { + const tool = this.store.get(exec.name) + if (!tool) throw new ToolNotFoundError(exec.name) + // Normalize the two `execute` return shapes: a bare ContentBlock[] (no + // meta) or a { content, meta } object (a tool attaching a private + // presentation payload). An array IS the content; the object carries it. + const returned = await tool.execute(exec.arguments, exec) + const content = Array.isArray(returned) ? returned : returned.content + const meta = Array.isArray(returned) ? undefined : returned.meta + return { callId: exec.callId, content, isError: false, ...meta !== undefined ? { meta } : {} } + } catch (error: unknown) { + return toolErrorResult(exec.callId, error) + } + }, + ) return await this.postExecute(exec, result) } catch (error: unknown) { diff --git a/packages/core/tools/src/json-schema.ts b/packages/core/tools/src/json-schema.ts new file mode 100644 index 0000000000..4c1036773b --- /dev/null +++ b/packages/core/tools/src/json-schema.ts @@ -0,0 +1,345 @@ +/** + * Structured-output JSON Schema subset: the vocabulary a caller uses to demand + * a machine-readable result from a subagent (`SubagentStartRequest.outputSchema`) + * or a workflow `agent()` call. + * + * This is deliberately NOT full JSON Schema. The schema travels verbatim to the + * model as a forced tool's `parameters`, and the value the model produces is + * validated here — so every accepted keyword must be one this module actually + * enforces. Accepting a keyword we don't enforce would validate less than the + * schema promises (accepted-then-ignored), so anything outside the subset is + * REJECTED LOUD by {@link assertSupportedOutputSchema} instead. The subset: + * + * - `type` — a single string (`object`/`array`/`string`/`number`/`integer`/ + * `boolean`/`null`); type ARRAYS (`["string","null"]`) are rejected. + * - `properties`/`required`/`additionalProperties` (boolean) on objects; every + * `required` key must be declared in `properties`. `additionalProperties` + * absent keeps standard JSON Schema semantics (extra keys allowed). + * - `items` on arrays (absent ⇒ any JSON items). + * - `enum` (non-empty, scalars only) and `const` (scalar) on scalar types. + * - Annotations `description`/`title`/`default`/`examples` are allowed and + * ignored (they constrain nothing), except that they must still be JSON data + * — the schema is serialized onto the wire, so a non-JSON annotation would be + * silently mangled. + * + * Values checked by {@link validateStructuredValue} are expected to be plain + * host-realm JSON data (model tool-call arguments are parsed wire JSON; a + * caller holding foreign-realm data materializes it first). + * + * @module dsh-tools/json-schema + */ + +import { assertNever, HarnessError } from '@deepseek-ai/dsh-llm' + +/** The scalar values `enum`/`const` may carry (finite numbers only). */ +export type StructuredScalar = string | number | boolean | null + +/** The `type` keywords the subset accepts. */ +export type StructuredSchemaType = 'object' | 'array' | 'string' | 'number' | 'integer' | 'boolean' | 'null' + +/** + * One node of the structured-output schema subset. Recursive via `properties` + * and `items`; see the module doc for the exact keyword semantics. + */ +export interface StructuredSchemaNode { + type: StructuredSchemaType + /** Nested property schemas (`type: 'object'` only). */ + properties?: Record + /** Required property names; each must appear in `properties`. */ + required?: string[] + /** `false` rejects undeclared keys; absent/`true` allows them (JSON Schema default). */ + additionalProperties?: boolean + /** Item schema (`type: 'array'` only); absent ⇒ any JSON items. */ + items?: StructuredSchemaNode + /** Allowed values (scalar types only). */ + enum?: StructuredScalar[] + /** The single allowed value (scalar types only). */ + const?: StructuredScalar + /** Annotation, ignored for validation. */ + description?: string + /** Annotation, ignored for validation. */ + title?: string + /** Annotation, ignored for validation (must still be JSON data). */ + default?: unknown + /** Annotation, ignored for validation (must still be JSON data). */ + examples?: unknown +} + +/** A structured-output schema: an OBJECT-rooted {@link StructuredSchemaNode}. */ +export type StructuredOutputSchema = StructuredSchemaNode & { type: 'object' } + +/** + * Thrown by {@link assertSupportedOutputSchema} when a schema falls outside the + * supported subset. Extends {@link HarnessError} (`code: 'UNSUPPORTED_SCHEMA'`) + * so seam code and tool results can route on it; `violations` lists every + * offending path, not just the first. + */ +export class OutputSchemaError extends HarnessError { + /** The individual violation messages, in walk order. */ + readonly violations: string[] + + constructor(violations: string[]) { + super(`unsupported output schema: ${violations.join('; ')}`, 'UNSUPPORTED_SCHEMA') + this.name = 'OutputSchemaError' + this.violations = violations + } +} + +/** The keywords the subset accepts, checked (`constraint`) or ignored (`annotation`). */ +const CONSTRAINT_KEYWORDS = new Set(['type', 'properties', 'required', 'additionalProperties', 'items', 'enum', 'const']) +const ANNOTATION_KEYWORDS = new Set(['description', 'title', 'default', 'examples']) + +const SCHEMA_TYPES: readonly StructuredSchemaType[] = ['object', 'array', 'string', 'number', 'integer', 'boolean', 'null'] + +/** + * Whether a value is a PLAIN JSON object — non-null, non-array, and with a + * prototype chain of at most one link (`null`-proto, or any realm's + * `Object.prototype`, whose own prototype is `null`). Realm-agnostic on + * purpose: a schema materialized in another realm carries THAT realm's + * `Object.prototype`, which an identity check would wrongly reject. Exotic + * hosts (`Date`, `Map`, class instances) have longer chains and are rejected — + * they would serialize lossily (`Date` → string, `Map` → `{}`) instead of + * failing loud. + */ +function isObjectLike(value: unknown): value is Record { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false + const proto: unknown = Object.getPrototypeOf(value) + return proto === null || Object.getPrototypeOf(proto) === null +} + +/** Whether a value is a supported scalar (`enum`/`const` member): string, finite number, boolean, or null. */ +function isStructuredScalar(value: unknown): value is StructuredScalar { + return value === null || typeof value === 'string' || typeof value === 'boolean' + || (typeof value === 'number' && Number.isFinite(value)) +} + +/** + * Whether a value is JSON data (annotation payloads only): scalars, arrays, and + * object-likes of such values. Realm-agnostic on purpose (no prototype check) — + * the schema may have been materialized from another realm; structural JSON-ness + * is what the wire needs. Cycles are rejected via `seen`. + */ +function isJsonData(value: unknown, seen: Set): boolean { + if (isStructuredScalar(value)) return true + // The scalar check above already returned for null, so `object` here is a real object. + if (typeof value !== 'object') return false + if (seen.has(value)) return false + seen.add(value) + try { + if (Array.isArray(value)) return value.every(entry => isJsonData(entry, seen)) + // A non-plain object (Date, Map, class instance) is NOT JSON data even when + // it has no enumerable values — it would serialize lossily, not loudly. + if (!isObjectLike(value)) return false + return Object.values(value).every(entry => isJsonData(entry, seen)) + } finally { + seen.delete(value) + } +} + +/** Collect subset violations for one schema node (recursive walk). */ +function checkSchemaNode(node: unknown, path: string, violations: string[], seen: Set): void { + if (!isObjectLike(node)) { + violations.push(`${path} must be a schema object`) + return + } + if (seen.has(node)) { + violations.push(`${path} is circular`) + return + } + seen.add(node) + + for (const key of Object.keys(node)) { + if (CONSTRAINT_KEYWORDS.has(key)) continue + if (ANNOTATION_KEYWORDS.has(key)) { + if (!isJsonData(node[key], new Set())) violations.push(`${path}.${key} annotation must be JSON data`) + continue + } + violations.push(`${path}.${key} is not a supported keyword (subset: type/properties/required/additionalProperties/items/enum/const + annotations)`) + } + if (typeof node.description !== 'undefined' && typeof node.description !== 'string') { + violations.push(`${path}.description must be a string`) + } + if (typeof node.title !== 'undefined' && typeof node.title !== 'string') { + violations.push(`${path}.title must be a string`) + } + + const type = node.type + if (typeof type !== 'string' || !(SCHEMA_TYPES as readonly unknown[]).includes(type)) { + violations.push(Array.isArray(type) + ? `${path}.type must be a single type string (type arrays are not supported)` + : `${path}.type must be one of ${SCHEMA_TYPES.join('/')}`) + seen.delete(node) + return + } + const schemaType = type as StructuredSchemaType + + // Keywords that only make sense on one type are rejected elsewhere — an + // `items` on an object (or `properties` on a string) is a schema-author bug + // the subset surfaces rather than ignores. + const allowedFor: Record = { + properties: ['object'], + required: ['object'], + additionalProperties: ['object'], + items: ['array'], + enum: ['string', 'number', 'integer', 'boolean', 'null'], + const: ['string', 'number', 'integer', 'boolean', 'null'], + } + for (const [key, types] of Object.entries(allowedFor)) { + if (key in node && !types.includes(schemaType)) { + violations.push(`${path}.${key} is not supported on type "${schemaType}"`) + } + } + + switch (schemaType) { + case 'object': { + const properties = node.properties + if (properties !== undefined) { + if (!isObjectLike(properties)) { + violations.push(`${path}.properties must be an object of schemas`) + } else { + for (const [key, child] of Object.entries(properties)) { + checkSchemaNode(child, `${path}.properties.${key}`, violations, seen) + } + } + } + const required = node.required + if (required !== undefined) { + if (!Array.isArray(required) || required.some(entry => typeof entry !== 'string')) { + violations.push(`${path}.required must be an array of strings`) + } else { + const declared = isObjectLike(properties) ? properties : {} + // The guard above proved every entry is a string. + for (const key of required as string[]) { + // Own-property check: `in` would let inherited names (`toString`) + // satisfy the declared-in-properties contract via the prototype. + if (!Object.hasOwn(declared, key)) violations.push(`${path}.required names "${key}" which is not in properties`) + } + } + } + if (node.additionalProperties !== undefined && typeof node.additionalProperties !== 'boolean') { + violations.push(`${path}.additionalProperties must be a boolean`) + } + break + } + case 'array': { + if (node.items !== undefined) checkSchemaNode(node.items, `${path}.items`, violations, seen) + break + } + case 'string': + case 'number': + case 'integer': + case 'boolean': + case 'null': { + const allowed = node.enum + if (allowed !== undefined) { + if (!Array.isArray(allowed) || allowed.length === 0 || !allowed.every(entry => isStructuredScalar(entry))) { + violations.push(`${path}.enum must be a non-empty array of scalars`) + } + } + if ('const' in node && !isStructuredScalar(node.const)) { + violations.push(`${path}.const must be a scalar`) + } + break + } + /* v8 ignore start -- defensive: schemaType was membership-checked against SCHEMA_TYPES above, so no runtime value reaches here */ + default: + assertNever(schemaType, 'assertSupportedOutputSchema') + /* v8 ignore stop */ + } + + seen.delete(node) +} + +/** + * Assert `schema` is a supported {@link StructuredOutputSchema} — object-rooted + * and entirely within the enforced subset. Throws {@link OutputSchemaError} + * (`UNSUPPORTED_SCHEMA`) listing EVERY violation; returns (and narrows) on + * success. Call this at the seam boundary, before any child is created. + * @param schema - the caller-supplied schema (unknown until asserted). + * @returns nothing — the assertion signature narrows `schema` to + * {@link StructuredOutputSchema} in the caller's scope on normal return. + */ +export function assertSupportedOutputSchema(schema: unknown): asserts schema is StructuredOutputSchema { + const violations: string[] = [] + checkSchemaNode(schema, 'schema', violations, new Set()) + if (violations.length === 0 && (schema as StructuredSchemaNode).type !== 'object') { + violations.push('schema.type must be "object" (structured output is object-rooted)') + } + if (violations.length > 0) throw new OutputSchemaError(violations) +} + +/** Collect violations for one value against an (already asserted) schema node. */ +function checkValue(node: StructuredSchemaNode, value: unknown, path: string): string[] { + switch (node.type) { + case 'object': { + if (!isObjectLike(value)) return [`"${path}" must be an object`] + const violations: string[] = [] + const properties = node.properties ?? {} + // Own-property discipline throughout: JSON carries own enumerable + // properties only, so an inherited `toString` must not satisfy + // `required`, dodge `additionalProperties: false`, or be validated as if + // the value carried it. + for (const key of node.required ?? []) { + if (!Object.hasOwn(value, key) || value[key] === undefined) violations.push(`missing required property "${path}.${key}"`) + } + for (const [key, child] of Object.entries(properties)) { + if (!Object.hasOwn(value, key) || value[key] === undefined) continue + violations.push(...checkValue(child, value[key], `${path}.${key}`)) + } + if (node.additionalProperties === false) { + for (const key of Object.keys(value)) { + if (!Object.hasOwn(properties, key)) violations.push(`"${path}.${key}" is not a declared property (additionalProperties: false)`) + } + } + return violations + } + case 'array': { + if (!Array.isArray(value)) return [`"${path}" must be an array`] + if (!node.items) return [] + const items = node.items + return value.flatMap((entry, index) => checkValue(items, entry, `${path}[${index}]`)) + } + case 'string': { + if (typeof value !== 'string') return [`"${path}" must be a string`] + break + } + case 'number': { + if (typeof value !== 'number' || !Number.isFinite(value)) return [`"${path}" must be a finite number`] + break + } + case 'integer': { + if (typeof value !== 'number' || !Number.isInteger(value)) return [`"${path}" must be an integer`] + break + } + case 'boolean': { + if (typeof value !== 'boolean') return [`"${path}" must be a boolean`] + break + } + case 'null': { + if (value !== null) return [`"${path}" must be null`] + break + } + default: + return assertNever(node.type, 'validateStructuredValue') + } + // Scalar constraint checks, shared by every scalar branch above. + if (node.enum && !node.enum.includes(value)) { + return [`"${path}" must be one of ${JSON.stringify(node.enum)}`] + } + if ('const' in node && value !== node.const) { + return [`"${path}" must be ${JSON.stringify(node.const)}`] + } + return [] +} + +/** + * Validate a value against an (already {@link assertSupportedOutputSchema}- + * asserted) schema. Returns human-readable, path-qualified violation messages + * — empty means valid. Total: never throws, however malformed the value. + * @param schema - the asserted schema to check against. + * @param value - the candidate value (e.g. parsed tool-call arguments). + * @returns every violation found, in walk order (empty = valid). + */ +export function validateStructuredValue(schema: StructuredOutputSchema, value: unknown): string[] { + return checkValue(schema, value, 'value') +} diff --git a/packages/core/tools/src/schema.ts b/packages/core/tools/src/schema.ts index 9149241225..1a428ffd40 100644 --- a/packages/core/tools/src/schema.ts +++ b/packages/core/tools/src/schema.ts @@ -155,6 +155,9 @@ export interface JsonSchemaObject { * `properties`, `required` array). * * This is a plain function — no schemastery or other framework dependency. + * @param spec - the author-facing per-property schema to convert. + * @returns the wire-format JSON Schema; the top-level `required` array is + * omitted entirely when no property is marked required. */ export function schemaSpecToJsonSchema(spec: SchemaSpec): JsonSchemaObject { const properties: Record = {} @@ -269,6 +272,9 @@ function checkSpec(spec: SchemaSpec, value: unknown, path: string): string[] { * keys are allowed (no `additionalProperties: false`); `default` is not * applied; an `object`/`array` prop without `properties`/`items` only * type-checks; `enum` is membership (strings only). + * @param spec - the declared parameter schema to validate against. + * @param args - the model-generated arguments, however malformed. + * @returns the violation messages in declaration order; empty means valid. */ export function validateArgs(spec: SchemaSpec, args: unknown): string[] { return checkSpec(spec, args, '') @@ -289,6 +295,13 @@ export interface DefineToolOptions { * standard JSON Schema at runtime. */ parameters: S + /** + * Optional cooperative tool-call timeout budget in milliseconds. When given it + * must be a positive finite number; it is attached to the produced + * {@link ToolDefinition} for `@deepseek-ai/dsh-timeout-policy` to enforce and + * is never sent to the model. + */ + timeoutMs?: number /** * Tool execution function. `args` is typed as {@link InferArgs} — zero * casts needed. Returns either a bare {@link ContentBlock}`[]` (model-facing @@ -340,6 +353,13 @@ export interface DefineToolOptions { * Raw JSON-Schema tool definitions (from MCP servers) are still accepted * by `ToolRegistry.register()` directly — `defineTool` is sugar for * first-party plugin authors. + * @param options - the tool's name, description, typed parameter schema, + * execute body, and optional presenters. + * @returns a registry-ready {@link ToolDefinition}: its `execute` validates the + * raw args first (throwing {@link ToolArgsError} on mismatch, which the + * registry turns into an isError result), and its presenters validate softly + * (returning undefined on mismatch, since replay may feed them older-schema + * args). */ export function defineTool(options: DefineToolOptions): ToolDefinition { // Object-literal execute methods don't use `this`; the reference is safe. @@ -349,10 +369,14 @@ export function defineTool(options: DefineToolOptions): const userPresentCall = options.presentCall // eslint-disable-next-line @typescript-eslint/unbound-method const userPresentResult = options.presentResult + if (options.timeoutMs !== undefined && (!Number.isFinite(options.timeoutMs) || options.timeoutMs <= 0)) { + throw new Error(`defineTool(${options.name}): timeoutMs must be a positive finite number`) + } const tool: ToolDefinition = { name: options.name, description: options.description, parameters: schemaSpecToJsonSchema(options.parameters) as unknown as Record, + ...(options.timeoutMs !== undefined ? { timeoutMs: options.timeoutMs } : {}), async execute(args: unknown, exec: ToolExecution): Promise { // Validate the model-generated args before the typed body runs. On // mismatch we throw ToolArgsError; the registry turns it into an diff --git a/packages/core/tools/tests/json-schema.spec.ts b/packages/core/tools/tests/json-schema.spec.ts new file mode 100644 index 0000000000..6fa895288e --- /dev/null +++ b/packages/core/tools/tests/json-schema.spec.ts @@ -0,0 +1,304 @@ +import { describe, expect, it } from 'vitest' +import { + assertSupportedOutputSchema, + OutputSchemaError, + validateStructuredValue, + type StructuredOutputSchema, +} from '../src/json-schema.ts' + +/** Assert-and-narrow helper: the asserted schema, typed. */ +function asserted(schema: unknown): StructuredOutputSchema { + assertSupportedOutputSchema(schema) + return schema +} + +/** The violations OutputSchemaError carries for a bad schema (throws if it passes). */ +function violationsOf(schema: unknown): string[] { + try { + assertSupportedOutputSchema(schema) + } catch (error: unknown) { + if (error instanceof OutputSchemaError) return error.violations + throw error + } + throw new Error('expected the schema to be rejected') +} + +describe('assertSupportedOutputSchema', () => { + it('accepts a representative subset schema (all supported keywords)', () => { + const schema = asserted({ + type: 'object', + description: 'a finding', + title: 'Finding', + properties: { + file: { type: 'string', description: 'path' }, + line: { type: 'integer' }, + severity: { type: 'string', enum: ['low', 'high'] }, + kind: { type: 'string', const: 'bug' }, + score: { type: 'number' }, + confirmed: { type: 'boolean' }, + parent: { type: 'null' }, + tags: { type: 'array', items: { type: 'string' } }, + nested: { + type: 'object', + properties: { x: { type: 'number', default: 3, examples: [1, 2] } }, + additionalProperties: false, + }, + anything: { type: 'array' }, + }, + required: ['file', 'line'], + additionalProperties: true, + }) + expect(schema.type).toBe('object') + }) + + it('rejects a non-object root (scalar/array-rooted schemas)', () => { + expect(violationsOf({ type: 'string' })).toEqual(['schema.type must be "object" (structured output is object-rooted)']) + expect(violationsOf({ type: 'array', items: { type: 'string' } })) + .toContain('schema.type must be "object" (structured output is object-rooted)') + }) + + it('rejects non-object schema nodes and missing/unknown type', () => { + expect(violationsOf('nope')).toEqual(['schema must be a schema object']) + expect(violationsOf(null)).toEqual(['schema must be a schema object']) + expect(violationsOf([])).toEqual(['schema must be a schema object']) + expect(violationsOf({})).toEqual(['schema.type must be one of object/array/string/number/integer/boolean/null']) + expect(violationsOf({ type: 'tuple' })[0]).toMatch(/type must be one of/) + expect(violationsOf({ type: 'object', properties: { a: 'str' } })).toEqual(['schema.properties.a must be a schema object']) + }) + + it('rejects type ARRAYS with a dedicated message', () => { + expect(violationsOf({ type: ['string', 'null'] })) + .toEqual(['schema.type must be a single type string (type arrays are not supported)']) + }) + + it('rejects unsupported constraint keywords loudly (never accepted-then-ignored)', () => { + for (const keyword of ['oneOf', 'anyOf', 'allOf', 'not', 'pattern', 'minimum', 'maxLength', '$ref']) { + const bad = violationsOf({ type: 'object', [keyword]: [] }) + expect(bad.some(v => v.includes(`schema.${keyword} is not a supported keyword`))).toBe(true) + } + }) + + it('reports EVERY violation, not just the first', () => { + const bad = violationsOf({ + type: 'object', + pattern: 'x', + properties: { a: { type: 'weird' }, b: { type: 'string', minimum: 1 } }, + }) + expect(bad.length).toBe(3) + }) + + it('rejects keywords on the wrong type (items on object, properties on string, enum on object)', () => { + expect(violationsOf({ type: 'object', items: { type: 'string' } })) + .toEqual(['schema.items is not supported on type "object"']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'string', properties: {} } } })) + .toEqual(['schema.properties.a.properties is not supported on type "string"']) + expect(violationsOf({ type: 'object', enum: [1] })) + .toEqual(['schema.enum is not supported on type "object"']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'array', const: 1 } } })) + .toEqual(['schema.properties.a.const is not supported on type "array"']) + }) + + it('validates required: must be string[] naming declared properties', () => { + expect(violationsOf({ type: 'object', required: 'file' })) + .toEqual(['schema.required must be an array of strings']) + expect(violationsOf({ type: 'object', required: [1] })) + .toEqual(['schema.required must be an array of strings']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'string' } }, required: ['b'] })) + .toEqual(['schema.required names "b" which is not in properties']) + expect(violationsOf({ type: 'object', required: ['a'] })) + .toEqual(['schema.required names "a" which is not in properties']) + }) + + it('validates additionalProperties must be boolean and enum/const must be scalars', () => { + expect(violationsOf({ type: 'object', additionalProperties: {} })) + .toEqual(['schema.additionalProperties must be a boolean']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'string', enum: [] } } })) + .toEqual(['schema.properties.a.enum must be a non-empty array of scalars']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'string', enum: [{}] } } })) + .toEqual(['schema.properties.a.enum must be a non-empty array of scalars']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'string', enum: 'x' } } })) + .toEqual(['schema.properties.a.enum must be a non-empty array of scalars']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'number', enum: [Number.NaN] } } })) + .toEqual(['schema.properties.a.enum must be a non-empty array of scalars']) + expect(violationsOf({ type: 'object', properties: { a: { type: 'string', const: {} } } })) + .toEqual(['schema.properties.a.const must be a scalar']) + }) + + it('rejects non-string description/title and non-JSON annotation payloads', () => { + expect(violationsOf({ type: 'object', description: 7 })) + .toEqual(['schema.description must be a string']) + expect(violationsOf({ type: 'object', title: 7 })) + .toEqual(['schema.title must be a string']) + expect(violationsOf({ type: 'object', default: () => 1 })) + .toEqual(['schema.default annotation must be JSON data']) + expect(violationsOf({ type: 'object', examples: [undefined] })) + .toEqual(['schema.examples annotation must be JSON data']) + expect(violationsOf({ type: 'object', examples: [Number.POSITIVE_INFINITY] })) + .toEqual(['schema.examples annotation must be JSON data']) + // A cyclic annotation payload is caught by the JSON-data walk. + const cyclicAnnotation: Record = {} + cyclicAnnotation.self = cyclicAnnotation + expect(violationsOf({ type: 'object', default: cyclicAnnotation })) + .toEqual(['schema.default annotation must be JSON data']) + // Object/array annotations that ARE JSON data pass. + asserted({ type: 'object', default: { a: [1, 'x', null, true] } }) + }) + + it('rejects a circular schema instead of recursing forever', () => { + const node: Record = { type: 'object' } + node.properties = { self: node } + expect(violationsOf(node)).toEqual(['schema.properties.self is circular']) + }) + + it('accepts the same subschema object reused in two SIBLING positions (a DAG, not a cycle)', () => { + const leaf = { type: 'string' } + asserted({ type: 'object', properties: { a: leaf, b: leaf } }) + }) + + it('required cannot be satisfied by INHERITED names — `toString` is not a declared property', () => { + // `'toString' in {}` is true via Object.prototype; the declared-property + // contract must be an own-property check. + expect(violationsOf({ type: 'object', properties: {}, required: ['toString'] })) + .toEqual(['schema.required names "toString" which is not in properties']) + }) + + it('rejects exotic host objects where the subset expects plain JSON structure', () => { + // A Map as `properties` has no own enumerable entries: structurally it + // would read as "no properties" and serialize to {} — lossy, not loud. + expect(violationsOf({ type: 'object', properties: new Map() })) + .toEqual(['schema.properties must be an object of schemas']) + // A Date node is not a schema object even though Object.values(date) is []. + expect(violationsOf({ type: 'object', properties: { at: new Date(0) } })) + .toEqual(['schema.properties.at must be a schema object']) + }) + + it('rejects exotic annotation payloads that would serialize lossily', () => { + expect(violationsOf({ type: 'object', default: new Date(0) })) + .toEqual(['schema.default annotation must be JSON data']) + expect(violationsOf({ type: 'object', examples: [new Map()] })) + .toEqual(['schema.examples annotation must be JSON data']) + }) +}) + +describe('validateStructuredValue', () => { + const schema = asserted({ + type: 'object', + properties: { + file: { type: 'string' }, + line: { type: 'integer' }, + score: { type: 'number' }, + confirmed: { type: 'boolean' }, + parent: { type: 'null' }, + severity: { type: 'string', enum: ['low', 'high'] }, + kind: { type: 'string', const: 'bug' }, + tags: { type: 'array', items: { type: 'string' } }, + free: { type: 'array' }, + nested: { type: 'object', properties: { x: { type: 'number' } }, required: ['x'], additionalProperties: false }, + }, + required: ['file'], + }) + + it('accepts a fully valid value (empty violations)', () => { + expect(validateStructuredValue(schema, { + file: 'a.ts', line: 3, score: 0.5, confirmed: true, parent: null, + severity: 'high', kind: 'bug', tags: ['x'], free: [1, { any: true }], nested: { x: 1 }, + })).toEqual([]) + }) + + it('reports missing required and wrong root type', () => { + expect(validateStructuredValue(schema, {})).toEqual(['missing required property "value.file"']) + expect(validateStructuredValue(schema, 'nope')).toEqual(['"value" must be an object']) + expect(validateStructuredValue(schema, [])).toEqual(['"value" must be an object']) + }) + + it('type-checks every scalar branch with path-qualified messages', () => { + expect(validateStructuredValue(schema, { file: 1 })).toEqual(['"value.file" must be a string']) + expect(validateStructuredValue(schema, { file: 'a', line: 1.5 })).toEqual(['"value.line" must be an integer']) + expect(validateStructuredValue(schema, { file: 'a', line: 'x' })).toEqual(['"value.line" must be an integer']) + expect(validateStructuredValue(schema, { file: 'a', score: 'x' })).toEqual(['"value.score" must be a finite number']) + expect(validateStructuredValue(schema, { file: 'a', score: Number.NaN })).toEqual(['"value.score" must be a finite number']) + expect(validateStructuredValue(schema, { file: 'a', confirmed: 'yes' })).toEqual(['"value.confirmed" must be a boolean']) + expect(validateStructuredValue(schema, { file: 'a', parent: 0 })).toEqual(['"value.parent" must be null']) + }) + + it('enforces enum membership and const equality', () => { + expect(validateStructuredValue(schema, { file: 'a', severity: 'mid' })) + .toEqual(['"value.severity" must be one of ["low","high"]']) + expect(validateStructuredValue(schema, { file: 'a', kind: 'feature' })) + .toEqual(['"value.kind" must be "bug"']) + }) + + it('checks arrays per index; an items-less array accepts anything', () => { + expect(validateStructuredValue(schema, { file: 'a', tags: 'x' })).toEqual(['"value.tags" must be an array']) + expect(validateStructuredValue(schema, { file: 'a', tags: ['ok', 2] })).toEqual(['"value.tags[1]" must be a string']) + expect(validateStructuredValue(schema, { file: 'a', free: [{ deep: [1] }, null] })).toEqual([]) + }) + + it('recurses into nested objects: required + additionalProperties: false', () => { + expect(validateStructuredValue(schema, { file: 'a', nested: {} })) + .toEqual(['missing required property "value.nested.x"']) + expect(validateStructuredValue(schema, { file: 'a', nested: { x: 1, y: 2 } })) + .toEqual(['"value.nested.y" is not a declared property (additionalProperties: false)']) + expect(validateStructuredValue(schema, { file: 'a', nested: 3 })) + .toEqual(['"value.nested" must be an object']) + }) + + it('a required key present-but-undefined counts as missing', () => { + expect(validateStructuredValue(schema, { file: undefined })).toEqual(['missing required property "value.file"']) + }) + + it('inherited properties satisfy nothing: required, additionalProperties, and recursion are own-property only', () => { + // required: ['toString'] must NOT be satisfied by Object.prototype.toString. + expect(validateStructuredValue( + asserted({ type: 'object', properties: { toString: { type: 'string' } }, required: ['toString'] }), + {}, + )).toEqual(['missing required property "value.toString"']) + // additionalProperties: false must flag an OWN `toString` key even though + // `'toString' in properties` is true via the prototype. + expect(validateStructuredValue( + asserted({ type: 'object', additionalProperties: false }), + { toString: 1 }, + )).toEqual(['"value.toString" is not a declared property (additionalProperties: false)']) + // A declared property the value does NOT carry must not be validated + // against the value's INHERITED member (constructor is a function on + // every plain object's prototype, not a carried property). + expect(validateStructuredValue( + asserted({ type: 'object', properties: { constructor: { type: 'string' } } }), + {}, + )).toEqual([]) + }) + + it('a non-plain object value is not an object in the JSON sense', () => { + expect(validateStructuredValue(asserted({ type: 'object' }), new Date(0))) + .toEqual(['"value" must be an object']) + }) + + it('collects multiple violations across branches in one pass', () => { + expect(validateStructuredValue(schema, { line: 'x', severity: 'mid' })).toEqual([ + 'missing required property "value.file"', + '"value.line" must be an integer', + '"value.severity" must be one of ["low","high"]', + ]) + }) + + it('null-typed const/enum work through the scalar path', () => { + const nullish = asserted({ type: 'object', properties: { a: { type: 'null', const: null } } }) + expect(validateStructuredValue(nullish, { a: null })).toEqual([]) + }) + + it('rejects a non-object properties value in the schema walk', () => { + expect(violationsOf({ type: 'object', properties: [] })) + .toEqual(['schema.properties must be an object of schemas']) + }) + + it('an object schema without properties/required only type-checks its value', () => { + const bare = asserted({ type: 'object' }) + expect(validateStructuredValue(bare, { any: ['thing'] })).toEqual([]) + expect(validateStructuredValue(bare, 7)).toEqual(['"value" must be an object']) + }) + + it('validateStructuredValue throws on a type the assert would never let through (assertNever backstop)', () => { + const forged = { type: 'tuple' } as unknown as StructuredOutputSchema + expect(() => validateStructuredValue(forged, 1)).toThrow(/tuple/) + }) +}) diff --git a/packages/core/tools/tests/tools.spec.ts b/packages/core/tools/tests/tools.spec.ts index 09158b8398..98be207159 100644 --- a/packages/core/tools/tests/tools.spec.ts +++ b/packages/core/tools/tests/tools.spec.ts @@ -62,6 +62,17 @@ describe('ToolRegistry', () => { expect(schema.execute).toBeUndefined() }) + it('schemas() excludes timeoutMs — the budget must never reach the model', async () => { + const ctx = await setup() + ctx.tools.register(defineTool({ + name: 'budgeted', description: 'has a budget', parameters: {}, timeoutMs: 5_000, + async execute() { return [{ type: 'text' as const, text: 'ok' }] }, + })) + const schema = ctx.tools.schemas().find(s => s.name === 'budgeted') + expect(schema).toBeDefined() + expect('timeoutMs' in (schema as object)).toBe(false) + }) + it('executes a tool and returns its content', async () => { const ctx = await setup() ctx.tools.register(echoTool) @@ -272,6 +283,148 @@ describe('ToolRegistry', () => { expect(order).toEqual(['pre:before', 'pre:after', 'post:before', 'post:after']) }) + it('runs tools/execute after an allowed pre-execute, around dispatch, and before post-execute', async () => { + const ctx = await setup() + const order: string[] = [] + ctx.tools.register(defineTool({ + name: 'traced', + description: 'echo', + parameters: { text: { type: 'string' } }, + async execute(args) { + order.push('dispatch') + return [{ type: 'text' as const, text: args.text ?? '' }] + }, + })) + + ctx.on('tools/pre-execute', async (_exec, next) => { order.push('pre'); return next() }) + ctx.on('tools/execute', async (_exec, next) => { + order.push('execute:before') + const result = await next() + order.push('execute:after') + return result + }) + ctx.on('tools/post-execute', async (_exec, _result, next) => { order.push('post'); return next() }) + + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'traced', arguments: { text: 'hi' } }) + expect(result).toEqual({ callId: CallId('c1'), content: [{ type: 'text', text: 'hi' }], isError: false }) + // The around seam wraps dispatch; pre gates before it, post runs over its result. + expect(order).toEqual(['pre', 'execute:before', 'dispatch', 'execute:after', 'post']) + }) + + it('a pre-execute deny short-circuits before tools/execute (the seam never runs)', async () => { + const ctx = await setup() + ctx.tools.register(echoTool) + + let entered = false + ctx.on('tools/pre-execute', async (_exec, _next): Promise => ({ kind: 'deny', reason: 'nope' })) + ctx.on('tools/execute', async (_exec, next) => { entered = true; return next() }) + + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'echo', arguments: { text: 'hi' } }) + expect(result.isError).toBe(true) + expect(result.content[0]).toMatchObject({ text: 'Error: nope' }) + expect(entered).toBe(false) // a denied call never enters the around-dispatch seam + }) + + it('a thrown tool is normalized to an isError result BEFORE a tools/execute listener sees next()', async () => { + const ctx = await setup() + ctx.tools.register({ + ...echoTool, + name: 'boom', + async execute() { throw new HarnessError('kaboom', 'BOOM') }, + }) + + let seen: { isError: boolean; error?: unknown } | undefined + ctx.on('tools/execute', async (_exec, next) => { + const result = await next() + // The base next() IS dispatch-with-normalization: the wrapper sees the + // normalized isError result, never a raw throw from the tool body. + seen = { isError: result.isError, error: result.error } + return result + }) + + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'boom', arguments: {} }) + expect(seen).toEqual({ isError: true, error: { name: 'HarnessError', code: 'BOOM' } }) + expect(result.isError).toBe(true) + expect(result.content[0]).toMatchObject({ text: 'Error: kaboom' }) + }) + + it('a thrown tool normalized inside tools/execute still reaches post-execute', async () => { + const ctx = await setup() + ctx.tools.register({ + ...echoTool, + name: 'boom', + async execute() { throw new Error('exploded') }, + }) + + let postSaw: boolean | undefined + ctx.on('tools/execute', async (_exec, next) => next()) + ctx.on('tools/post-execute', async (_exec, result, next) => { + postSaw = result.isError + return next() + }) + + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'boom', arguments: {} }) + expect(postSaw).toBe(true) // the normalized isError still flows through post-execute + expect(result.isError).toBe(true) + expect(result.content[0]).toMatchObject({ text: 'Error: exploded' }) + }) + + it('a tools/execute listener can replace exec.signal for the dispatched tool (deadline pattern)', async () => { + const ctx = await setup() + let seenSignal: AbortSignal | undefined + ctx.tools.register({ + ...echoTool, + name: 'signal-probe', + async execute(_args, exec) { + seenSignal = exec.signal + return [{ type: 'text' as const, text: 'ok' }] + }, + }) + + const upstream = new AbortController().signal + const replacement = new AbortController().signal + ctx.on('tools/execute', async (exec, next) => { + expect(exec.signal).toBe(upstream) + // Cordis next() ignores passed arguments, so a wrapper mutates exec in + // place (the documented "mutate the shared object, then delegate" idiom). + exec.signal = replacement + return next() + }) + + await ctx.tools.execute({ callId: CallId('c1'), name: 'signal-probe', arguments: {}, signal: upstream }) + expect(seenSignal).toBe(replacement) // dispatch saw the wrapper's replacement, not the upstream + }) + + it('a tools/execute listener can short-circuit dispatch by returning a result without next()', async () => { + const ctx = await setup() + let dispatched = false + ctx.tools.register({ + ...echoTool, + name: 'never-runs', + async execute() { dispatched = true; return [] }, + }) + + ctx.on('tools/execute', async (exec, _next): Promise => + ({ callId: exec.callId, content: [{ type: 'text', text: 'short-circuited' }], isError: false })) + + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'never-runs', arguments: {} }) + expect(dispatched).toBe(false) // returning without next() skips core dispatch + expect(result.content[0]).toMatchObject({ text: 'short-circuited' }) + }) + + it('returns an isError result when a tools/execute listener throws', async () => { + const ctx = await setup() + ctx.tools.register(echoTool) + ctx.on('tools/execute', async () => { throw new Error('wrapper broke') }) + + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'echo', arguments: { text: 'hi' } }) + expect(result).toEqual({ + callId: CallId('c1'), + content: [{ type: 'text', text: 'Error: wrapper broke' }], + isError: true, + }) + }) + it('returns an isError result when a tools/pre-execute listener throws', async () => { const ctx = await setup() ctx.tools.register(echoTool) @@ -989,6 +1142,38 @@ describe('defineTool validation (the runtime-validation RFC, part 1)', () => { const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'raw', arguments: {} }) expect(result.isError).toBe(false) }) + + it('attaches a positive-finite timeoutMs to the definition', () => { + const tool = defineTool({ + name: 'x', description: 'd', parameters: {}, timeoutMs: 30_000, + async execute() { return [{ type: 'text' as const, text: 'ok' }] }, + }) + expect(tool.timeoutMs).toBe(30_000) + }) + + it('omits timeoutMs when not declared', () => { + const tool = defineTool({ + name: 'x', description: 'd', parameters: {}, + async execute() { return [{ type: 'text' as const, text: 'ok' }] }, + }) + expect(tool.timeoutMs).toBeUndefined() + }) + + it('throws when timeoutMs is zero or negative', () => { + const make = (ms: number) => defineTool({ + name: 'x', description: 'd', parameters: {}, timeoutMs: ms, + async execute() { return [{ type: 'text' as const, text: 'ok' }] }, + }) + expect(() => make(0)).toThrow('timeoutMs must be a positive finite number') + expect(() => make(-5)).toThrow('positive finite number') + }) + + it('throws when timeoutMs is non-finite', () => { + expect(() => defineTool({ + name: 'x', description: 'd', parameters: {}, timeoutMs: Infinity, + async execute() { return [{ type: 'text' as const, text: 'ok' }] }, + })).toThrow('positive finite number') + }) }) describe('defineTool presentation (presentCall / presentResult)', () => { diff --git a/packages/fs/README.md b/packages/fs/README.md index 985a9f3ad6..ec3bb62afb 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -10,3 +10,7 @@ The filesystem stack: a provider seam (text IO + atomic mutation with an optiona | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`) | (registers on `ctx.tools`) | The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. + +## No timeouts on file IO + +`read`/`write`/`edit` take **no** `timeoutMs`, and the provider seam arms no deadline — unlike bash and web, which consume [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md). A local syscall is best-effort-abortable at most: a timeout could not force an in-progress `fsync`/`rename` to stop, so a deadline here would be a knob that cannot deliver on its promise. Adding one would also be an implicit default in the exact place explicit-over-implicit forbids. Both reference agents (Claude Code, Codex) leave file IO untimed for the same reason; cancellation still propagates through the tool-execution signal for best-effort abort at syscall boundaries. diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index b1b9c25397..f9af5375d2 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -129,6 +129,9 @@ export interface LocalDirEntry { * and intermediate directories are created by the write. Two input paths * reaching the same file via symlinks share one key. Falls back to the absolute * path only when no ancestor (not even the filesystem root) can be resolved. + * @param cwd - base directory a relative `path` resolves against. + * @param path - absolute or relative path; empty/whitespace-only throws `FS_NOT_FOUND`. + * @returns the absolute display path plus the realpath-derived stable target key. */ export async function resolveLocalTarget(cwd: string, path: string): Promise { if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') @@ -165,7 +168,11 @@ export async function resolveLocalTarget(cwd: string, path: string): Promise { try { const info = await stat(absolutePath) @@ -200,6 +207,9 @@ async function resolveListedChildTarget(parent: LocalTarget, name: string): Prom * List direct children of a directory in stable name order. Each child includes * a resolved target plus stat metadata when still available; file contents are * never read. + * @param target - the resolved directory to list; a missing or non-directory target throws. + * @param signal - aborts the listing, checked between children (`FS_ABORTED`). + * @returns one entry per direct child, sorted by name. */ export async function listDirectory(target: LocalTarget, signal?: AbortSignal): Promise { throwIfAborted(signal, 'list') @@ -290,6 +300,9 @@ async function statRegularFile(target: LocalTarget, verb: 'read', signal?: Abort /** * Read a whole regular UTF-8 text file into a single decoded string. Rejects * non-regular files, invalid UTF-8, and NUL-byte binary samples. + * @param target - the resolved file to read. + * @param signal - aborts the read (`FS_ABORTED`). + * @returns the full decoded text, byte-for-byte (no normalization). */ export async function readWholeText(target: LocalTarget, signal?: AbortSignal): Promise { await statRegularFile(target, 'read', signal) @@ -305,6 +318,9 @@ export async function readWholeText(target: LocalTarget, signal?: AbortSignal): * Stream a whole regular UTF-8 text file as decoded text chunks. Same text * semantics as {@link readWholeText} (regular-file check, binary/NUL rejection, * cross-chunk UTF-8 decoding), but never holds the whole file in memory. + * @param target - the resolved file to stream. + * @param signal - aborts the stream, including between chunks (`FS_ABORTED`). + * @returns decoded text chunks in file order; chunk boundaries carry no meaning. */ export async function* streamWholeText(target: LocalTarget, signal?: AbortSignal): AsyncIterable { await statRegularFile(target, 'read', signal) @@ -352,6 +368,11 @@ async function removeStagingDirOrThrow(stagingDir: string, originalError: unknow * (`0o700`) staging directory, fsync, optionally chmod to the final mode while * still private, then rename over the target. `mode` (when given) preserves an * existing file's permissions across the replace. + * @param absolutePath - the final destination (typically a target key); missing parent dirs are created. + * @param content - the full UTF-8 text to write. + * @param mode - final file mode applied before the rename (an existing file's, to preserve permissions); undefined leaves `0o600`. + * @param signal - aborts the write (`FS_ABORTED`); checked before the rename, so the target is never left torn. + * @param internals - test seam for pinning temp names and observing the staged file. */ export async function writeFileAtomic( absolutePath: string, @@ -409,6 +430,12 @@ export async function writeFileAtomic( /** Line ending style detected before LF normalization. */ export type LineEndings = 'LF' | 'CRLF' +/** + * Collapse CRLF to LF — the canonical in-memory form every edit/diff basis + * uses. Lone `\r` bytes (not followed by `\n`) are left untouched. + * @param content - decoded text in whatever line-ending style the file had. + * @returns the text with every `\r\n` pair replaced by `\n`. + */ function normalizeLineEndings(content: string): string { return content.replaceAll('\r\n', '\n') } @@ -420,6 +447,14 @@ function detectLineEndings(raw: string): LineEndings { return crlfCount > lfCount ? 'CRLF' : 'LF' } +/** + * Convert LF-normalized content back to the line-ending style detected at read + * time, for write-back. `LF` returns the content unchanged; `CRLF` re-normalizes + * first so an already-CRLF sequence is never doubled to `\r\r\n`. + * @param content - the LF-normalized (edited) text. + * @param lineEndings - the original file's style, as detected by {@link readForEdit}. + * @returns the text in the original file's line-ending style. + */ function restoreLineEndings(content: string, lineEndings: LineEndings): string { return lineEndings === 'LF' ? content : normalizeLineEndings(content).split('\n').join('\r\n') } @@ -438,6 +473,10 @@ function countOccurrences(content: string, needle: string): number { /** * Read and decode a file for editing: rejects binaries, returns LF-normalized * content plus the original line-ending style for write-back. + * @param absolutePath - the file to read (typically a target key). + * @param displayPath - the caller-facing path used in error messages. + * @param signal - aborts the read (`FS_ABORTED`). + * @returns the LF-normalized content and the detected style to restore on write-back. */ export async function readForEdit( absolutePath: string, @@ -459,6 +498,9 @@ export async function readForEdit( * prior bytes, so an undiffable prior file simply yields no contextual-hunk basis * (the caller treats `null` the same as an absent file: the result renders a * whole-file diff rather than an applied hunk). + * @param absolutePath - the file to read (typically a target key); it must exist. + * @param signal - aborts the read (`FS_ABORTED`). + * @returns the LF-normalized text, or null for a binary or non-UTF-8 file. */ export async function readTextForDiff(absolutePath: string, signal?: AbortSignal): Promise { const buffer = await readFileAbortable(absolutePath, 'read', signal) @@ -477,6 +519,12 @@ export async function readTextForDiff(absolutePath: string, signal?: AbortSignal * `FS_EDIT_NOT_FOUND` on empty `oldString` or zero matches and * `FS_AMBIGUOUS_EDIT` on multiple matches when `replaceAll` is false. Returns * the edited content (still LF-normalized) and the replacement count. + * @param content - the current file content, already LF-normalized. + * @param oldString - literal text to find; CRLF inside it is normalized to LF before matching. + * @param newString - literal replacement text, normalized the same way. + * @param replaceAll - replace every match instead of requiring exactly one. + * @param displayPath - the caller-facing path used in error messages. + * @returns the edited LF-normalized content plus how many occurrences were replaced. */ export function applyLiteralEdit( content: string, diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 0ad8365d4a..1a3ebc57f6 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -73,6 +73,7 @@ export class LocalFileSystem extends FileSystem { cwd: z.string().default(process.cwd()), }) + /** Validated config (schemastery applied the defaults before construction). */ readonly config: ResolvedConfig /** Test seam forwarded to fsio (force streaming path, pin temp names). */ internals: FsIoInternals = {} diff --git a/packages/fs/fs/src/types.ts b/packages/fs/fs/src/types.ts index 9351a373db..f6f5b8005f 100644 --- a/packages/fs/fs/src/types.ts +++ b/packages/fs/fs/src/types.ts @@ -29,7 +29,12 @@ import type { Branded } from '@deepseek-ai/dsh-brand' */ export type FsTargetKey = Branded<'FsTargetKey'> -/** Brand a string as an {@link FsTargetKey}. */ +/** + * Brand a string as an {@link FsTargetKey}. For backend use only — a consumer + * never manufactures a key, it receives one from `resolve()`. + * @param key - the backend's raw key string (the local backend passes a realpath). + * @returns the same string, branded; no validation is performed. + */ export function FsTargetKey(key: string): FsTargetKey { return key as FsTargetKey } @@ -42,7 +47,12 @@ export function FsTargetKey(key: string): FsTargetKey { */ export type FsVersion = Branded<'FsVersion'> -/** Brand a string as an {@link FsVersion}. */ +/** + * Brand a string as an {@link FsVersion}. For backend use only — a consumer + * never manufactures a version, it receives one from `stat`/write/edit outcomes. + * @param v - the backend's raw version string (the local backend derives it from mtime+size). + * @returns the same string, branded; no validation is performed. + */ export function FsVersion(v: string): FsVersion { return v as FsVersion } diff --git a/packages/fs/tool-fs/src/diff.ts b/packages/fs/tool-fs/src/diff.ts index a45082489a..da8b7a872e 100644 --- a/packages/fs/tool-fs/src/diff.ts +++ b/packages/fs/tool-fs/src/diff.ts @@ -40,6 +40,10 @@ export type FsDiffMeta = { diffs: FileDiff[] } * (a pure insertion) reports `oldText: null` (nothing to diff against), mirroring * the call-time card's new-file convention. The unified-diff "\ No newline at end * of file" markers are dropped — they annotate the patch, not file content. + * @param path - the path stamped on every produced diff (the model-facing `file_path`; the bridge relativizes it). + * @param before - the file text before the change (the backend's LF-normalized diff basis). + * @param after - the file text after the change, on the same basis. + * @returns one diff per applied hunk, in file order; empty when the texts are identical. */ export function computeHunkDiffs(path: string, before: string, after: string): FileDiff[] { const patch = structuredPatch('', '', before, after, undefined, undefined, { context: DIFF_CONTEXT }) @@ -83,6 +87,8 @@ function isFileDiff(value: unknown): value is FileDiff { * it validates defensively rather than trusting the payload — a bad `meta` yields * `undefined`, and the caller decides the fallback (edit → the generic result * rendering; write → an args-derived whole-file diff), never a thrown presenter. + * @param meta - the opaque `tool/result` meta payload (live or replayed from the session log). + * @returns the validated non-empty hunk list, or undefined for an absent/empty/malformed payload. */ export function diffsFromMeta(meta: unknown): FileDiff[] | undefined { if (typeof meta !== 'object' || meta === null || Array.isArray(meta)) return undefined diff --git a/packages/fs/tool-fs/src/edit.ts b/packages/fs/tool-fs/src/edit.ts index 5ede545b03..220c850d69 100644 --- a/packages/fs/tool-fs/src/edit.ts +++ b/packages/fs/tool-fs/src/edit.ts @@ -29,7 +29,13 @@ interface EditInput { replaceAll: boolean } -/** Validate value constraints the schema DSL can't express. */ +/** + * Validate value constraints the schema DSL can't express: a non-blank + * `file_path`, a non-empty `old_string`, and `old_string !== new_string` + * (an equal pair would be a guaranteed no-op edit). + * @param args - the schema-validated raw tool arguments. + * @returns the camelCased input with `replace_all` defaulted to false. + */ export function parseEditArgs(args: { file_path: string; old_string: string; new_string: string; replace_all?: boolean }): EditInput { if (args.file_path.trim().length === 0) throw new Error('file_path must be a non-empty string') if (args.old_string.length === 0) throw new Error('old_string must be a non-empty string') @@ -42,14 +48,22 @@ export function parseEditArgs(args: { file_path: string; old_string: string; new } } -/** Format an edit success (single-match or replace-all) as a Claude-style model-facing message. */ +/** + * Format an edit success (single-match or replace-all) as a Claude-style model-facing message. + * @param displayPath - the backend-resolved path shown to the model. + * @param replaceAll - selects the all-occurrences wording over the single-replacement one. + * @returns the confirmation sentence the model sees as the tool result. + */ export function formatEditOutput(displayPath: string, replaceAll: boolean): string { return replaceAll ? `The file ${displayPath} has been updated. All occurrences were successfully replaced.` : `The file ${displayPath} has been updated successfully.` } -/** Register the `edit` tool and its system-prompt guidance. */ +/** + * Register the `edit` tool and its system-prompt guidance. + * @param ctx - the plugin context; registrations are effects scoped to it, and execution uses its `fs` service. + */ export function applyEditTool(ctx: Context): void { ctx.systemPrompt.section({ name: 'tool:edit', diff --git a/packages/fs/tool-fs/src/read-render.ts b/packages/fs/tool-fs/src/read-render.ts index c54279baa9..8cafc0d400 100644 --- a/packages/fs/tool-fs/src/read-render.ts +++ b/packages/fs/tool-fs/src/read-render.ts @@ -119,6 +119,10 @@ function finish(acc: WindowAccumulator, request: ReadWindow, displayPath: string * path serves both. Scans for newlines with a capped line buffer (a newline-free * giant line is truncated, never buffered past `request.maxLineLength`), * enforces the byte cap, and throws `FS_NOT_FOUND` for an offset past EOF. + * @param chunks - decoded text chunks in file order; chunk boundaries carry no meaning. + * @param request - the resolved window; the caller has already applied its defaults and caps. + * @param displayPath - the caller-facing path used in the offset-out-of-range error. + * @returns the numbered window lines, the total line count seen, and the byte-cap truncation flag. */ export async function buildWindow( chunks: AsyncIterable | Iterable, @@ -156,7 +160,12 @@ export async function buildWindow( return finish(acc, request, displayPath) } -/** Format a read outcome as one OpenCode-style line-numbered text block body. */ +/** + * Format a read outcome as one OpenCode-style line-numbered text block body. + * @param displayPath - the backend-resolved path rendered in the envelope's `` element. + * @param outcome - the windowed read to render. + * @returns the model-facing envelope: numbered lines plus a continuation or end-of-file footer. + */ export function formatReadOutput(displayPath: string, outcome: FileReadOutcome): string { const endLine = outcome.lines.at(-1)?.number ?? Math.max(0, outcome.offset - 1) let footer: string diff --git a/packages/fs/tool-fs/src/read.ts b/packages/fs/tool-fs/src/read.ts index a9cc4a9806..039d8742e9 100644 --- a/packages/fs/tool-fs/src/read.ts +++ b/packages/fs/tool-fs/src/read.ts @@ -58,7 +58,12 @@ function parsePositiveInteger(value: number, name: string): number { return value } -/** Validate value constraints the schema DSL can't express. `maxLimit` is the deployment's line cap. */ +/** + * Validate value constraints the schema DSL can't express. `maxLimit` is the deployment's line cap. + * @param args - the schema-validated raw tool arguments; `offset`/`limit` must be positive integers when given. + * @param maxLimit - the configured line cap: both the default `limit` and the largest one accepted. + * @returns the validated input with `offset` defaulted to 1 and `limit` to `maxLimit`. + */ export function parseReadArgs(args: { file_path: string; offset?: number; limit?: number }, maxLimit: number): ReadInput { if (args.file_path.trim().length === 0) throw new Error('file_path must be a non-empty string') const offset = args.offset === undefined ? 1 : parsePositiveInteger(args.offset, 'offset') @@ -67,7 +72,11 @@ export function parseReadArgs(args: { file_path: string; offset?: number; limit? return { filePath: args.file_path, offset, limit } } -/** Register the `read` tool and its system-prompt guidance. */ +/** + * Register the `read` tool and its system-prompt guidance. + * @param ctx - the plugin context; registrations are effects scoped to it, and execution uses its `fs` service. + * @param caps - the deployment's resolved read caps (plugin config after defaulting). + */ export function applyReadTool(ctx: Context, caps: ReadToolCaps): void { ctx.systemPrompt.section({ name: 'tool:read', diff --git a/packages/fs/tool-fs/src/session-cwd.ts b/packages/fs/tool-fs/src/session-cwd.ts index b7774fb201..16ba17e2e5 100644 --- a/packages/fs/tool-fs/src/session-cwd.ts +++ b/packages/fs/tool-fs/src/session-cwd.ts @@ -18,7 +18,11 @@ import type { ToolExecution } from '@deepseek-ai/dsh-tools' -/** The session workspace cwd for this call, or `undefined` when none applies. */ +/** + * The session workspace cwd for this call, or `undefined` when none applies. + * @param exec - the tool-execution context; only its optional `agent` is read. + * @returns the calling agent's session cwd, or undefined for a non-agent caller (the backend then applies its own default). + */ export function sessionCwd(exec: ToolExecution): string | undefined { return exec.agent?.session.header.cwd } diff --git a/packages/fs/tool-fs/src/write.ts b/packages/fs/tool-fs/src/write.ts index 1054e2ff2f..fd4eec45f3 100644 --- a/packages/fs/tool-fs/src/write.ts +++ b/packages/fs/tool-fs/src/write.ts @@ -21,13 +21,23 @@ import type {} from '@deepseek-ai/dsh-system-prompt' import { computeHunkDiffs, diffsFromMeta, type FsDiffMeta } from './diff.ts' import { sessionCwd } from './session-cwd.ts' -/** Validate value constraints the schema DSL can't express. */ +/** + * Validate value constraints the schema DSL can't express: only a non-blank + * `file_path` — an empty `content` is legitimate (it writes an empty file). + * @param args - the schema-validated raw tool arguments. + * @returns the camelCased input; `content` passes through untouched. + */ export function parseWriteArgs(args: { file_path: string; content: string }): { filePath: string; content: string } { if (args.file_path.trim().length === 0) throw new Error('file_path must be a non-empty string') return { filePath: args.file_path, content: args.content } } -/** Format a write outcome as one model-facing text block body. */ +/** + * Format a write outcome as one model-facing text block body. + * @param displayPath - the backend-resolved path rendered in the envelope's `` element. + * @param outcome - the write outcome; its `operation` selects the Created/Updated wording. + * @returns the model-facing confirmation envelope (no file content is echoed back). + */ export function formatWriteOutput(displayPath: string, outcome: FsWriteOutcome): string { const verb = outcome.operation === 'create' ? 'Created' : 'Updated' return `${displayPath} @@ -37,7 +47,10 @@ ${verb} file ` } -/** Register the `write` tool and its system-prompt guidance. */ +/** + * Register the `write` tool and its system-prompt guidance. + * @param ctx - the plugin context; registrations are effects scoped to it, and execution uses its `fs` service. + */ export function applyWriteTool(ctx: Context): void { ctx.systemPrompt.section({ name: 'tool:write', diff --git a/packages/guard/README.md b/packages/guard/README.md new file mode 100644 index 0000000000..9198698b20 --- /dev/null +++ b/packages/guard/README.md @@ -0,0 +1,9 @@ +# guard/ — loop-hygiene guard family + +Behavioral guard plugins that watch the agent loop for unproductive patterns and nudge the model back on course. A single **product** package — there is no interface/implementation seam here, because a guard is a self-contained consumer of existing core seams (`tools/post-execute`, `agent/prompt-submit`, `agent/status`), not a swappable capability. + +| Package | Role | ctx key | +|---|---|---| +| `repeat-tool-guard/` | Advisory reminders when an agent loops on identical tool calls | (listens on `ctx.tools`' waterfalls) | + +Reminders travel as `additionalContext` on the `tools/post-execute` decision; the agent loop appends them as logged `context/message` events after the step's tool results (see [the tools package](../core/tools)), so everything a guard says to the model is reconstructable from the session log. diff --git a/packages/guard/repeat-tool-guard/README.md b/packages/guard/repeat-tool-guard/README.md new file mode 100644 index 0000000000..dc385bc033 --- /dev/null +++ b/packages/guard/repeat-tool-guard/README.md @@ -0,0 +1,37 @@ +# @deepseek-ai/dsh-repeat-tool-guard + +An advisory loop-breaker, not a model-facing tool: it never appears in the tool list, never vetoes or rewrites a call, and adds exactly one behavior — it watches each agent's stream of tool calls, counts runs of consecutive calls to the same tool with identical canonicalized arguments, and at configured run lengths injects an escalating advisory reminder telling the model to stop repeating itself, re-read the last result, and either change approach or conclude. The decision (retry differently, gather more evidence, or finish) stays entirely with the model: a legitimately repeated call is delayed by nothing and blocked by nothing. Decision record: [the repeat-tool-guard RFC](../../../docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md). + +## Config + +```yaml +- id: repeat-tool-guard + name: '@deepseek-ai/dsh-repeat-tool-guard' + config: + thresholds: [3, 5, 8] # default; consecutive counts that trigger a reminder + include: [] # tool-name patterns to track; empty ⇒ all tools + exclude: [todo_write] # tool-name patterns transparent to the chain + argumentsPreviewChars: 500 # default; cap on arguments quoted in the detailed reminder +``` + +`thresholds` fails loud at plugin load: an empty list, a non-integer, a value below 2, or a duplicate throws, never a silent fall-back to defaults; `argumentsPreviewChars` equally rejects anything but an integer >= 1. The list is normalized to ascending order; the FIRST threshold delivers a short generic nudge, every later threshold delivers the detailed form naming the tool, the run length, and the canonical arguments — head-truncated at `argumentsPreviewChars` with an omitted-count marker, so a looping `write`/`edit` payload cannot ride into the next request unbounded (the chain key always compares the FULL canonical string; the cap bounds the reminder, never the detection). + +`include`/`exclude` entries support `*` wildcards and are predicates over whatever tools exist at call time, not references to registry entries — a pattern matching no currently registered tool is NOT an error (`exclude: [mcp_*]` stays valid in a deployment that loads no MCP tools), unlike `toolOrder`'s referent check. + +## Chain semantics + +The chain key is `(tool name, canonical arguments)` — canonicalization is a deep key-sort plus `JSON.stringify`, so argument objects differing only in property order count as identical. A call identical to the previous tracked call increments the agent's consecutive counter; a different tracked call resets it to 1. + +- **Untracked calls are transparent to the chain.** A call excluded by `include`/`exclude` neither increments nor resets the counter, so `grep X → todo_write → grep X` still counts as two consecutive `grep X` when `todo_write` is excluded. This is what makes exclusion useful: bookkeeping tools interleaved into a loop must not launder it. +- **Denied calls count.** Detection sits on `tools/post-execute`, which also runs for calls a `tools/pre-execute` listener denied — a model hammering a denied call is exactly the loop worth breaking. +- **Calls without an agent are ignored.** A direct `ctx.tools.execute()` caller has no model to remind and no `AgentId` to key on. +- **Per-agent keying.** The tool registry is context-level and subagents interleave through the same waterfall, so chains are keyed by `AgentId`; one agent's repetition never trips another's reminder. A user prompt (`agent/prompt-submit`) resets the submitting agent's chain; agent disposal drops its state. +- **In-memory only.** A session resumed from persistence starts with a fresh chain — the guard is a heuristic nudge, not a logged invariant, later reminders are the accepted cost. + +## Reminder delivery + +Reminders ride the post-execute decision's `additionalContext` (source `{kind: 'plugin', plugin: 'repeat-tool-guard'}`), never a `content` replacement: the `tool/result` event stays the tool's own output for audit. The loop buffers the context and appends it as a `context/message` after the step's tool results, which the session renders as the tagged synthetic-user envelope — so the reminder is model-visible, source-attributed, and reconstructable from the session log with no new session event. The guard always delegates via `next()` and folds its reminder onto the downstream decision (both variants — a blocked call still gets the nudge); when a downstream listener attached its own `additionalContext`, the fold concatenates content and carries the guard's `source` (a `HookContext` holds one `MessageSource`; `source.kind` is what framing depends on). + +## Testing + +Unit suites drive a real agent loop against a mock adapter (no network) and cover the chain semantics above to per-file 100%. The snapshot tier owns the transcript surface: a scripted-replay scenario repeats a call five times and pins both reminder tiers (gentle at 3, detailed at 5) as `context/message`s in the ACP transcript. diff --git a/packages/guard/repeat-tool-guard/package.json b/packages/guard/repeat-tool-guard/package.json new file mode 100644 index 0000000000..9b085bb015 --- /dev/null +++ b/packages/guard/repeat-tool-guard/package.json @@ -0,0 +1,41 @@ +{ + "name": "@deepseek-ai/dsh-repeat-tool-guard", + "description": "Repeat-tool-call guard plugin: advisory reminders when an agent loops on identical tool calls", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "dependencies": { + "schemastery": "^3.18.0" + }, + "peerDependencies": { + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-tools": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-loop": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", + "@deepseek-ai/dsh-tools": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/guard/repeat-tool-guard/src/index.ts b/packages/guard/repeat-tool-guard/src/index.ts new file mode 100644 index 0000000000..919d0541ba --- /dev/null +++ b/packages/guard/repeat-tool-guard/src/index.ts @@ -0,0 +1,268 @@ +/** + * Repeat-tool-call guard: advisory loop-breaker for agents stuck re-issuing + * the same tool call with identical arguments. + * + * Not a model-facing tool — it registers no tool, never vetoes or rewrites a + * call, and adds exactly one behavior: watch each agent's stream of tool calls + * through the `tools/post-execute` waterfall, count runs of consecutive calls + * to the same tool with identical canonicalized arguments, and at configured + * run lengths fold an escalating advisory reminder onto the decision's + * `additionalContext`. The loop appends that context as a logged + * `context/message` after the step's tool results, so the reminder is + * model-visible, source-attributed, and reconstructable from the session log + * with no new session event. Decision record: + * docs/rfc/implemented/feature/2026-07-08-repeat-tool-guard.md. + * + * ```yaml + * - id: repeat-tool-guard + * name: '@deepseek-ai/dsh-repeat-tool-guard' + * config: + * thresholds: [3, 5, 8] # consecutive counts that trigger a reminder + * include: [] # tool-name patterns to track; empty = all tools + * exclude: [todo_write] # tool-name patterns transparent to the chain + * ``` + * + * Chain state is keyed per {@link AgentId} — the tool registry is a + * context-level singleton whose waterfalls interleave every agent's calls, so + * a shared counter would let one agent's repetition trip another's reminder. + * State is in-memory only: a session resumed from persistence starts with a + * fresh chain (the guard is a heuristic nudge, not a logged invariant). + * + * Plugin export shape: named exports, NO default. The cordis Loader's + * `unwrapExports` does `exports.default ?? exports`, so a stray default would + * collapse the module to the bare `apply` (see docs/postmortem/0001). + * + * @module @deepseek-ai/dsh-repeat-tool-guard + */ + +import type { Context } from 'cordis' +import z from 'schemastery' +import type { AgentId, HookContext, PromptDecision } from '@deepseek-ai/dsh-agent' +import type { MessageSource } from '@deepseek-ai/dsh-llm' +import type { PostToolDecision, ToolExecution } from '@deepseek-ai/dsh-tools' + +export const name = 'repeat-tool-guard' + +/** + * Plugin config, validated by the same-named schemastery schema plus the + * load-time checks in `apply` (misconfiguration fails loud: an empty + * `thresholds` list, a non-integer, a value below 2, or a duplicate throws at + * plugin load, never a silent fall-back). `include`/`exclude` entries are + * `*`-wildcard predicates over tool names at call time, not references to + * registry entries — a pattern matching no currently registered tool is valid + * (`exclude: [mcp_*]` must stay legal in a deployment that loads no MCP tools). + */ +export interface Config { + /** Consecutive-repeat counts that trigger a reminder (default `[3, 5, 8]`). */ + thresholds?: number[] + /** Tool-name patterns to track; empty means every tool is tracked. */ + include?: string[] + /** Tool-name patterns transparent to the chain (neither count nor reset). */ + exclude?: string[] + /** + * Maximum characters of canonical arguments quoted in the DETAILED reminder + * (default 500). Large payloads (a `write` body, a long command) would + * otherwise ride into the next request unbounded — precisely in a loop + * scenario; the cap bounds the reminder, never the detection (the chain key + * always compares the FULL canonical string). + */ + argumentsPreviewChars?: number +} + +export const Config: z = z.object({ + thresholds: z.array(z.number()).default([3, 5, 8]), + include: z.array(z.string()).default([]), + exclude: z.array(z.string()).default([]), + argumentsPreviewChars: z.number().default(500), +}) + +/** + * The `{kind:'plugin'}` source stamped on every reminder this guard injects — + * the label is load-bearing (an unlabeled context would render as a user + * prompt in derived history). + */ +const PLUGIN_SOURCE: MessageSource = { kind: 'plugin', plugin: 'repeat-tool-guard' } + +/** + * The gentle first-threshold reminder. Keyed to `thresholds[0]`, not a literal + * count, so a custom first threshold keeps the gentle-then-detailed escalation. + */ +const GENTLE_REMINDER = + 'You are repeating the exact same tool call with identical arguments. ' + + 'Carefully analyze the previous result before calling again: if the task is ' + + 'not complete, try a different approach or different arguments instead of ' + + 'repeating the call.' + +/** The detailed later-threshold reminder naming the tool, the run length, and the canonical arguments. */ +function detailedReminder(toolName: string, count: number, canonicalArguments: string): string { + return 'Repeated tool call detected:\n' + + `- tool: ${toolName}\n` + + `- consecutive_calls: ${count}\n` + + `- arguments: ${canonicalArguments}\n` + + 'The repeated calls are not making progress. Do not call this tool with ' + + 'these exact arguments again. Inspect the latest result and choose a ' + + 'different action, different arguments, or finish the task if enough ' + + 'evidence has been gathered.' +} + +/** + * Deep key-sort of a parsed-JSON value so two argument objects that differ + * only in property order canonicalize identically. Arguments reach the guard + * as the loop's `JSON.parse` output (or its raw-string fallback for malformed + * argument JSON), so JSON's value domain is the whole input domain — no + * bigint, cycle, or `undefined` handling exists because no input path can + * produce them. + */ +function sortJsonValue(value: unknown): unknown { + if (Array.isArray(value)) return value.map(sortJsonValue) + if (value !== null && typeof value === 'object') { + const record = value as Record + const sorted: Record = {} + for (const key of Object.keys(record).sort()) { + sorted[key] = sortJsonValue(record[key]) + } + return sorted + } + return value +} + +/** Canonical string form of a call's arguments: deep key-sort, then stringify. */ +function canonicalize(argumentsValue: unknown): string { + return JSON.stringify(sortJsonValue(argumentsValue)) +} + +/** Compile one `*`-wildcard pattern to an anchored RegExp (every other regex metacharacter is matched literally). */ +function wildcardToRegExp(pattern: string): RegExp { + const escaped = pattern.replace(/[|\\{}()[\]^$+?.]/g, String.raw`\$&`) + return new RegExp(`^${escaped.replaceAll('*', '.*')}$`) +} + +/** + * Head-truncate the canonical arguments for quoting in the detailed reminder, + * marking how much was omitted. Bounds only the model-visible text — the + * chain key always uses the full canonical string. + */ +function previewArguments(canonical: string, cap: number): string { + if (canonical.length <= cap) return canonical + return `${canonical.slice(0, cap)}… (+${canonical.length - cap} more chars)` +} + +/** + * Validate `thresholds` per the fail-loud contract and return them sorted + * ascending (the escalation rule reads `thresholds[0]` as the gentle tier, so + * order is normalized here, once). + */ +function validateThresholds(values: number[]): number[] { + if (values.length === 0) { + throw new Error('repeat-tool-guard: `thresholds` must not be empty') + } + for (const value of values) { + if (!Number.isInteger(value) || value < 2) { + throw new Error(`repeat-tool-guard: invalid threshold ${value} — every threshold must be an integer >= 2`) + } + } + if (new Set(values).size !== values.length) { + throw new Error('repeat-tool-guard: `thresholds` must not contain duplicates') + } + return [...values].sort((a, b) => a - b) +} + +/** + * Concatenate the guard's reminder context with a downstream listener's + * optional one so folding drops neither. The merged block carries the guard's + * `source` — a `HookContext` holds one `MessageSource` and the seam cannot + * represent mixed provenance; the rendered `context/message` only + * distinguishes by `source.kind`, so a downstream plugin's text is still + * correctly framed as plugin context. + */ +function concatContext(ours: HookContext, theirs: HookContext | undefined): HookContext { + if (!theirs) return ours + return { content: [...ours.content, ...theirs.content], source: ours.source } +} + +/** One agent's consecutive-repeat chain: the last tracked call's identity key and its run length. */ +interface Chain { + key: string + count: number +} + +/** + * Install the guard's listeners. + * @param ctx - plugin context; listeners are scoped to it and disposed with it. + * @param config - validated {@link Config}; `thresholds` is re-checked fail-loud here. + */ +export function apply(ctx: Context, config: Config): void { + // schemastery's .default() guarantees the fields are set after validation. + const thresholds = validateThresholds(config.thresholds as number[]) + const thresholdSet = new Set(thresholds) + const includePatterns = (config.include as string[]).map(wildcardToRegExp) + const excludePatterns = (config.exclude as string[]).map(wildcardToRegExp) + const argumentsPreviewChars = config.argumentsPreviewChars as number + if (!Number.isInteger(argumentsPreviewChars) || argumentsPreviewChars < 1) { + throw new Error(`repeat-tool-guard: invalid argumentsPreviewChars ${argumentsPreviewChars} — must be an integer >= 1`) + } + + const chains = new Map() + + /** Whether a tool participates in the chain (untracked calls are transparent: they neither count nor reset). */ + function tracked(toolName: string): boolean { + if (includePatterns.length > 0 && !includePatterns.some(pattern => pattern.test(toolName))) return false + return !excludePatterns.some(pattern => pattern.test(toolName)) + } + + /** + * Advance the calling agent's chain for one attempt and return the reminder + * to deliver, if this attempt's run length hits a configured threshold. + * Counting happens here — in post-execute — because denied calls also flow + * through this waterfall (`ToolRegistry.execute` routes a deny through the + * same pipeline), and a model hammering a denied call is exactly the loop + * worth breaking. + */ + function observe(exec: ToolExecution): HookContext | undefined { + // A direct `ctx.tools.execute()` caller has no model to remind and no id + // to key on; only agent-loop calls participate. + if (!exec.agent) return undefined + if (!tracked(exec.name)) return undefined + const canonical = canonicalize(exec.arguments) + const key = JSON.stringify([exec.name, canonical]) + const chain = chains.get(exec.agent.id) + const count = chain !== undefined && chain.key === key ? chain.count + 1 : 1 + chains.set(exec.agent.id, { key, count }) + if (!thresholdSet.has(count)) return undefined + const text = count === thresholds[0] + ? GENTLE_REMINDER + : detailedReminder(exec.name, count, previewArguments(canonical, argumentsPreviewChars)) + return { content: [{ type: 'text', text }], source: PLUGIN_SOURCE } + } + + // Observe-and-enrich, never veto: count first (state advances regardless of + // the downstream outcome), DELEGATE so a later listener can still block or + // replace, then fold the reminder onto whatever came back — additionalContext + // rides both decision variants, so a blocked call still gets the nudge. + ctx.on('tools/post-execute', async (exec, _result, next): Promise => { + const reminder = observe(exec) + const downstream = await next() + if (!reminder) return downstream + if (downstream.kind === 'block') { + return { kind: 'block', feedback: downstream.feedback, additionalContext: concatContext(reminder, downstream.additionalContext) } + } + return { + kind: 'accept', + ...downstream.content !== undefined ? { content: downstream.content } : {}, + additionalContext: concatContext(reminder, downstream.additionalContext), + } + }) + + // A user interjection changes the context; repetition across it is not a + // loop. Pure reset hook: always delegates (attaching nothing, vetoing + // nothing). + ctx.on('agent/prompt-submit', (agent, _content, _source, next): Promise => { + chains.delete(agent.id) + return next() + }) + + // Drop state when an agent goes away, bounding the map over harness lifetime. + ctx.on('agent/status', (agent, status) => { + if (status === 'disposed') chains.delete(agent.id) + }) +} diff --git a/packages/guard/repeat-tool-guard/tests/repeat-tool-guard.spec.ts b/packages/guard/repeat-tool-guard/tests/repeat-tool-guard.spec.ts new file mode 100644 index 0000000000..565f1076b5 --- /dev/null +++ b/packages/guard/repeat-tool-guard/tests/repeat-tool-guard.spec.ts @@ -0,0 +1,401 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import LlmService, { CallId } from '@deepseek-ai/dsh-llm' +import SessionStore, { type SessionEvent } from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { type ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' +import * as RepeatToolGuard from '@deepseek-ai/dsh-repeat-tool-guard' +import type { Config } from '@deepseek-ai/dsh-repeat-tool-guard' +import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' + +/** + * Behavior suite for the repeat-tool-call guard: chain semantics (identical / + * different-tracked / untracked-transparent / per-agent / resets), threshold + * escalation incl. the `thresholds[0]` gentle-text rule, canonicalization, + * fold-onto-downstream-decision, and fail-loud config validation — all driven + * through a real agent loop against a scripted mock adapter (no network). + */ + +/** Boot the core spine + the guard; the caller registers adapters and extra listeners. */ +async function harness(config: Config = {}): Promise { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(RepeatToolGuard, config) + ctx.tools.register(defineTool({ name: 'probe', description: 'p', parameters: {}, async execute() { return [{ type: 'text', text: 'ok' }] } })) + ctx.tools.register(defineTool({ name: 'other', description: 'o', parameters: {}, async execute() { return [{ type: 'text', text: 'ok' }] } })) + return ctx +} + +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { + return new Promise((resolve) => { const d = ctx.on('agent/status', (s, st) => { if (s === agent && st === 'idle') { d(); resolve() } }) }) +} + +/** Every `context/message` in the agent's log, flattened to joined text + source for terse assertions. */ +function reminders(agent: ReactLoopAgent): { text: string; source: unknown }[] { + return [...agent.session.events] + .filter((e): e is SessionEvent<'context/message'> => e.type === 'context/message') + .map(e => ({ + text: e.data.content.map(block => block.type === 'text' ? block.text : '').join('|'), + source: e.data.source, + })) +} + +const GUARD_SOURCE = { kind: 'plugin', plugin: 'repeat-tool-guard' } + +describe('threshold escalation', () => { + it('reminds gently at the first default threshold (3) and in detail at the second (5)', async () => { + const ctx = await harness() + const adapter = new MockAdapter([ + ...Array.from({ length: 5 }, (_, i) => toolCallResponse(`c${i}`, 'probe', { q: 'same' })), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(2) + expect(found[0]!.text).toContain('repeating the exact same tool call') + expect(found[0]!.source).toEqual(GUARD_SOURCE) + expect(found[1]!.text).toContain('consecutive_calls: 5') + expect(found[1]!.text).toContain('- tool: probe') + expect(found[1]!.text).toContain('{"q":"same"}') + expect(found[1]!.source).toEqual(GUARD_SOURCE) + }) + + it('keys the gentle text to thresholds[0], not the literal 3', async () => { + const ctx = await harness({ thresholds: [4, 2] }) // unsorted on purpose: normalized ascending + const adapter = new MockAdapter([ + ...Array.from({ length: 4 }, (_, i) => toolCallResponse(`c${i}`, 'probe', {})), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(2) + expect(found[0]!.text).toContain('repeating the exact same tool call') // gentle at 2 + expect(found[1]!.text).toContain('consecutive_calls: 4') // detailed at 4 + }) +}) + +describe('chain semantics', () => { + it('caps the detailed reminder arguments at argumentsPreviewChars (detection still keys on the full string)', async () => { + const ctx = await harness({ thresholds: [2, 3], argumentsPreviewChars: 24 }) + const bigPayload = 'x'.repeat(400) + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { body: bigPayload }), + toolCallResponse('c2', 'probe', { body: bigPayload }), + toolCallResponse('c3', 'probe', { body: bigPayload }), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(2) // gentle at 2, detailed at 3 — full-key matching survived the cap + const detailed = found[1]!.text + expect(detailed).toContain('- arguments: {"body":"xxxxxxxxxxxxxx') // 24-char head + expect(detailed).toContain('… (+387 more chars)') + expect(detailed).not.toContain(bigPayload) + }) + + it('a different tracked call resets the chain', async () => { + const ctx = await harness() + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + toolCallResponse('c2', 'probe', { q: 1 }), + toolCallResponse('c3', 'other', {}), // tracked, different → reset + toolCallResponse('c4', 'probe', { q: 1 }), + toolCallResponse('c5', 'probe', { q: 1 }), + toolCallResponse('c6', 'probe', { q: 1 }), // 3rd consecutive AFTER the reset + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + expect(reminders(agent)).toHaveLength(1) + }) + + it('excluded calls are transparent: they neither count nor reset', async () => { + const ctx = await harness({ exclude: ['other'] }) + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + toolCallResponse('c2', 'other', {}), // excluded → invisible to the chain + toolCallResponse('c3', 'probe', { q: 1 }), + toolCallResponse('c4', 'other', {}), + toolCallResponse('c5', 'probe', { q: 1 }), // 3rd consecutive probe + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(1) + expect(found[0]!.text).toContain('repeating the exact same tool call') + }) + + it('include patterns track only matching tools (wildcard star)', async () => { + const ctx = await harness({ include: ['pro*'] }) + const adapter = new MockAdapter([ + toolCallResponse('c1', 'other', {}), + toolCallResponse('c2', 'other', {}), + toolCallResponse('c3', 'other', {}), // 3 identical, but untracked + toolCallResponse('c4', 'probe', {}), + toolCallResponse('c5', 'probe', {}), + toolCallResponse('c6', 'probe', {}), // 3 identical, tracked + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(1) + expect(found[0]!.text).toContain('repeating the exact same tool call') + }) + + it('escapes regex metacharacters in patterns (a dot matches only a literal dot)', async () => { + const ctx = await harness({ exclude: ['pr.be'] }) // would match 'probe' as a regex; must not as a wildcard + const adapter = new MockAdapter([ + ...Array.from({ length: 3 }, (_, i) => toolCallResponse(`c${i}`, 'probe', {})), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + expect(reminders(agent)).toHaveLength(1) // probe was NOT excluded + }) + + it('canonicalization ignores property order, deeply', async () => { + const ctx = await harness() + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { a: 1, nested: { x: [1, 2], y: null } }), + toolCallResponse('c2', 'probe', { nested: { y: null, x: [1, 2] }, a: 1 }), + toolCallResponse('c3', 'probe', { a: 1, nested: { x: [1, 2], y: null } }), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + expect(reminders(agent)).toHaveLength(1) // all three canonicalize identically + }) + + it('keys chains per agent: one agent repeating never trips another', async () => { + const ctx = await harness() + ctx.llm.registerAdapter(['mock-a'], new MockAdapter([ + toolCallResponse('a1', 'probe', { q: 1 }), + toolCallResponse('a2', 'probe', { q: 1 }), + textResponse('done'), + ])) + ctx.llm.registerAdapter(['mock-b'], new MockAdapter([ + toolCallResponse('b1', 'probe', { q: 1 }), + toolCallResponse('b2', 'probe', { q: 1 }), + toolCallResponse('b3', 'probe', { q: 1 }), + textResponse('done'), + ])) + const agentA = ctx.agentLoop.create(AgentId('a'), { model: 'mock-a' }) + const agentB = ctx.agentLoop.create(AgentId('b'), { model: 'mock-b' }) + agentA.send([{ type: 'text', text: 'go' }]) + agentB.send([{ type: 'text', text: 'go' }]) + await Promise.all([waitForIdle(ctx, agentA), waitForIdle(ctx, agentB)]) + + expect(reminders(agentA)).toHaveLength(0) // 2 repeats < 3, despite B's 3 in the same registry + expect(reminders(agentB)).toHaveLength(1) + }) + + it('a new user prompt resets the chain', async () => { + const ctx = await harness() + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + toolCallResponse('c2', 'probe', { q: 1 }), + textResponse('turn one done'), + toolCallResponse('c3', 'probe', { q: 1 }), // without the reset this would be the 3rd + textResponse('turn two done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + agent.send([{ type: 'text', text: 'again' }]) + await waitForIdle(ctx, agent) + + expect(reminders(agent)).toHaveLength(0) + }) + + it('drops an agent chain on disposal', async () => { + const ctx = await harness({ thresholds: [2] }) + ctx.llm.registerAdapter(['mock'], new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + textResponse('done'), + toolCallResponse('c2', 'probe', { q: 1 }), // same id, fresh agent: count 1, not 2 + textResponse('done'), + ])) + // Loop agents are torn down by disposing the scope that created them + // (the loop.spec pattern): a child plugin fiber owns `first`. + let first!: ReactLoopAgent + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + first = inner.agentLoop.create(AgentId('reused'), { model: 'mock' }) + }, { inject: ['agentLoop'] })) + first.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, first) + await fiber.dispose() + await first.done + + const second = ctx.agentLoop.create(AgentId('reused'), { model: 'mock' }) + second.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, second) + + expect(reminders(second)).toHaveLength(0) + }) + + it('counts denied calls: hammering a denied tool still draws the reminder', async () => { + const ctx = await harness({ thresholds: [2] }) + ctx.on('tools/pre-execute', async () => ({ kind: 'deny' as const, reason: 'sealed' })) + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + toolCallResponse('c2', 'probe', { q: 1 }), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + expect(reminders(agent)).toHaveLength(1) + }) + + it('ignores direct executes with no agent (they neither crash nor advance any chain)', async () => { + const ctx = await harness({ thresholds: [2] }) + const direct = await ctx.tools.execute({ callId: CallId('d1'), name: 'probe', arguments: { q: 1 } }) + expect(direct.isError).toBe(false) + + ctx.llm.registerAdapter(['mock'], new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), // if the direct call had counted, this would be #2 + textResponse('done'), + ])) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + expect(reminders(agent)).toHaveLength(0) + }) +}) + +describe('fold onto the downstream decision', () => { + it('folds the reminder onto a downstream block and keeps its feedback', async () => { + const ctx = await harness({ thresholds: [2] }) + ctx.on('tools/post-execute', async () => ({ + kind: 'block' as const, + feedback: [{ type: 'text' as const, text: 'nope' }], + additionalContext: { content: [{ type: 'text' as const, text: 'downstream-ctx' }], source: { kind: 'plugin' as const, plugin: 'test' } }, + })) + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + toolCallResponse('c2', 'probe', { q: 1 }), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(2) + // Call 1: below threshold — the downstream context passes through untouched. + expect(found[0]!.text).toBe('downstream-ctx') + expect(found[0]!.source).toEqual({ kind: 'plugin', plugin: 'test' }) + // Call 2: reminder folded in front, single merged context, the guard's source. + expect(found[1]!.text).toContain('repeating the exact same tool call') + expect(found[1]!.text).toContain('|downstream-ctx') + expect(found[1]!.source).toEqual(GUARD_SOURCE) + // The block's feedback reached the tool result unchanged. + const results = [...agent.session.events].filter((e): e is SessionEvent<'tool/result'> => e.type === 'tool/result') + expect(results.every(r => r.data.isError)).toBe(true) + expect(results[1]!.data.content).toEqual([{ type: 'text', text: 'nope' }]) + }) + + it('preserves a downstream accept content replacement while folding', async () => { + const ctx = await harness({ thresholds: [2] }) + ctx.on('tools/post-execute', async () => ({ + kind: 'accept' as const, + content: [{ type: 'text' as const, text: 'replaced' }], + })) + const adapter = new MockAdapter([ + toolCallResponse('c1', 'probe', { q: 1 }), + toolCallResponse('c2', 'probe', { q: 1 }), + textResponse('done'), + ]) + ctx.llm.registerAdapter(['mock'], adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'go' }]) + await waitForIdle(ctx, agent) + + const found = reminders(agent) + expect(found).toHaveLength(1) + expect(found[0]!.text).toContain('repeating the exact same tool call') + const results = [...agent.session.events].filter((e): e is SessionEvent<'tool/result'> => e.type === 'tool/result') + expect(results[1]!.data.content).toEqual([{ type: 'text', text: 'replaced' }]) + }) +}) + +describe('config validation fails loud', () => { + async function spine(): Promise { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + return ctx + } + + it('rejects an empty thresholds list', async () => { + const ctx = await spine() + await expect(ctx.plugin(RepeatToolGuard, { thresholds: [] })).rejects.toThrow(/must not be empty/) + }) + + it('rejects a threshold below 2', async () => { + const ctx = await spine() + await expect(ctx.plugin(RepeatToolGuard, { thresholds: [1, 3] })).rejects.toThrow(/integer >= 2/) + }) + + it('rejects a non-integer threshold', async () => { + const ctx = await spine() + await expect(ctx.plugin(RepeatToolGuard, { thresholds: [2.5] })).rejects.toThrow(/integer >= 2/) + }) + + it('rejects duplicate thresholds', async () => { + const ctx = await spine() + await expect(ctx.plugin(RepeatToolGuard, { thresholds: [3, 3] })).rejects.toThrow(/duplicates/) + }) + + it('rejects a non-positive or fractional argumentsPreviewChars', async () => { + const ctx = await spine() + await expect(ctx.plugin(RepeatToolGuard, { argumentsPreviewChars: 0 })).rejects.toThrow(/argumentsPreviewChars/) + const ctx2 = await spine() + await expect(ctx2.plugin(RepeatToolGuard, { argumentsPreviewChars: 12.5 })).rejects.toThrow(/argumentsPreviewChars/) + }) +}) diff --git a/packages/guard/repeat-tool-guard/tsconfig.json b/packages/guard/repeat-tool-guard/tsconfig.json new file mode 100644 index 0000000000..66439bcd5f --- /dev/null +++ b/packages/guard/repeat-tool-guard/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../core/tools" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../llm/llm" + } + ] +} diff --git a/packages/hooks/hook-protocol/src/codec.ts b/packages/hooks/hook-protocol/src/codec.ts index 7cf97feca9..59de886e5e 100644 --- a/packages/hooks/hook-protocol/src/codec.ts +++ b/packages/hooks/hook-protocol/src/codec.ts @@ -75,6 +75,12 @@ function permissionDecisionOf(value: string | undefined): HookOutput['decision'] * (`decision`/`reason`/`continue`/`stopReason`/`systemMessage`) * are unaffected. Omit `expectedEventName` (or pass a matching one) to apply the * block as-is — a caller that doesn't key by event opts out of the check. + * + * @param exitCode - the process exit code; `undefined` when the hook could not be spawned at all. + * @param stdout - the captured stdout stream; consulted for structured JSON only on a 0 exit. + * @param stderr - the captured stderr stream; becomes the blocking `reason` on exit 2. + * @param expectedEventName - the event the hook is firing for; omit to apply a `hookSpecificOutput` block as-is. + * @returns the dialect-neutral decoded outcome. */ export function parseHookOutput(exitCode: number | undefined, stdout: string, stderr: string, expectedEventName?: string): HookOutput { const trimmedErr = stderr.trim() diff --git a/packages/hooks/hook-protocol/src/events.ts b/packages/hooks/hook-protocol/src/events.ts index 5e14f964fb..df6a10672a 100644 --- a/packages/hooks/hook-protocol/src/events.ts +++ b/packages/hooks/hook-protocol/src/events.ts @@ -66,6 +66,9 @@ export const DEFAULT_STDERR_SUMMARY_MAX_CHARS = 500 * `undefined` when empty, cut at `maxChars` with an ellipsis when over. The * bound is a parameter — like `runHook`'s `defaultTimeoutMs`, each bridge owns * the config default and passes it in. + * @param stderr - the hook's raw captured stderr. + * @param maxChars - the character cap for the summary (the bridge's config value). + * @returns the trimmed, capped summary, or `undefined` when stderr is blank. */ export function summarizeStderr(stderr: string, maxChars: number): string | undefined { const t = stderr.trim() @@ -73,7 +76,11 @@ export function summarizeStderr(stderr: string, maxChars: number): string | unde return t.length > maxChars ? t.slice(0, maxChars) + '…' : t } -/** Append a `hook/invoked` provenance event to `session`. */ +/** + * Append a `hook/invoked` provenance event to `session`. + * @param session - the session whose open turn records the event. + * @param invocation - the invocation identity; an absent `matcher` is omitted from the payload. + */ export function appendHookInvoked(session: Session, invocation: HookInvocation): void { session.append('hook/invoked', { turn: invocation.turn, @@ -91,6 +98,8 @@ export function appendHookInvoked(session: Session, invocation: HookInvocation): * else `'pass'`; `stderrSummary` is the trimmed stderr truncated to * `record.stderrSummaryMaxChars` characters (omitted when empty); `exitCode` * is omitted when the hook never ran. + * @param session - the session whose open turn records the event. + * @param record - the outcome to record: the decoded output plus the summary cap and duration. */ export function appendHookResult(session: Session, record: HookResultRecord): void { const { output } = record diff --git a/packages/hooks/hook-protocol/src/matcher.ts b/packages/hooks/hook-protocol/src/matcher.ts index ee1dd324b3..6863ee999f 100644 --- a/packages/hooks/hook-protocol/src/matcher.ts +++ b/packages/hooks/hook-protocol/src/matcher.ts @@ -34,6 +34,10 @@ const CLAUDE_LITERAL = /^[A-Za-z0-9_|]+$/ * pattern exact-matches the query (splitting `|` into alternatives); every other * `claude` pattern and ALL `codex` patterns are tested as an unanchored regex. * An invalid regex matches nothing (never throws). + * @param matcher - the configured pattern; absent/empty/`'*'` are the match-all sentinels. + * @param query - the candidate value (a tool name, a session source, …). + * @param mode - the dialect deciding literal-vs-regex interpretation of the pattern. + * @returns `true` when the pattern selects the query; `false` on a non-match or an invalid regex. */ export function matchesMatcher(matcher: string | undefined, query: string, mode: MatcherMode): boolean { if (isMatchAll(matcher)) return true diff --git a/packages/hooks/hook-protocol/src/merge.ts b/packages/hooks/hook-protocol/src/merge.ts index 1e53dbaaea..d219c4eb18 100644 --- a/packages/hooks/hook-protocol/src/merge.ts +++ b/packages/hooks/hook-protocol/src/merge.ts @@ -71,6 +71,8 @@ function decisionForRank(maxRank: number): MergedDecision { * into one {@link MergedHookOutcome} by the precedence rules above. An empty list * yields a neutral outcome (`decision: 'none'`, no stop, empty context) — the * caller treats that as "no hook had anything to say". + * @param outputs - every matched hook's decoded output, in hook order. + * @returns the single folded outcome the bridge maps onto its seam. */ export function mergeHookOutputs(outputs: HookOutput[]): MergedHookOutcome { let maxRank = 0 diff --git a/packages/hooks/hook-protocol/src/runner.ts b/packages/hooks/hook-protocol/src/runner.ts index f5a892c468..e4a2bc8a04 100644 --- a/packages/hooks/hook-protocol/src/runner.ts +++ b/packages/hooks/hook-protocol/src/runner.ts @@ -70,6 +70,11 @@ export interface RunHookResult { * `exitCode: undefined`, so the caller's merge logic treats it as a * non-blocking error rather than crashing the turn. `now` is injected for * testable durations. + * @param bash - the executor seam the command runs through. + * @param hook - the configured command; its `timeoutSec` (wire unit: seconds) overrides the default timeout. + * @param options - the invocation's payload, env, cwd, signal, stdin framing, and default timeout. + * @param now - millisecond clock used for the reported duration. + * @returns the decoded output plus the run's wall-clock duration. */ export async function runHook( bash: BashExecutor, diff --git a/packages/hooks/hooks-claude/src/config.ts b/packages/hooks/hooks-claude/src/config.ts index d78486e58a..b447c28533 100644 --- a/packages/hooks/hooks-claude/src/config.ts +++ b/packages/hooks/hooks-claude/src/config.ts @@ -43,7 +43,12 @@ function asObject(value: unknown): Record | undefined { : undefined } -/** Apply `${CLAUDE_PLUGIN_ROOT}` / `${CLAUDE_PROJECT_DIR}` substitution to a command string. */ +/** + * Apply `${CLAUDE_PLUGIN_ROOT}` / `${CLAUDE_PROJECT_DIR}` substitution to a command string. + * @param command - the raw command from config. + * @param vars - the substitution values; a token whose variable is unset stays verbatim. + * @returns the command with every occurrence of each set token replaced. + */ export function substituteCommand(command: string, vars: SubstitutionVars): string { let out = command if (vars.pluginRoot !== undefined) out = out.split('${CLAUDE_PLUGIN_ROOT}').join(vars.pluginRoot) @@ -57,6 +62,9 @@ export function substituteCommand(command: string, vars: SubstitutionVars): stri * Non-command hooks and malformed entries are dropped (recorded in `skipped` / * silently ignored) rather than throwing — a bad hook config must not crash boot. * `vars` are substituted into every surviving `command`. + * @param raw - the parsed JSON config: a settings object with a `hooks` key, or the bare event map. + * @param vars - substitution values applied to every surviving `command` (defaults to none). + * @returns the runnable per-event groups plus the skipped non-command hooks. */ export function parseClaudeConfig(raw: unknown, vars: SubstitutionVars = {}): ParsedClaudeConfig { const config: ClaudeHookConfig = {} diff --git a/packages/hooks/hooks-codex/src/config.ts b/packages/hooks/hooks-codex/src/config.ts index 411f058eea..505f8d2bda 100644 --- a/packages/hooks/hooks-codex/src/config.ts +++ b/packages/hooks/hooks-codex/src/config.ts @@ -41,6 +41,8 @@ function asObject(value: unknown): Record | undefined { * `type !== 'command'` and `async: true` command hooks are skipped (recorded in * `skipped`). Malformed entries are ignored rather than thrown — a bad config * must not crash boot. No command substitution (Codex does none). + * @param raw - the parsed JSON config: a `{ hooks: … }` wrapper or the bare event map. + * @returns the runnable per-event groups plus the skipped hooks with their reasons. */ export function parseCodexConfig(raw: unknown): ParsedCodexConfig { const config: CodexHookConfig = {} diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index 9e5e5179b6..e0627c5695 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -13,7 +13,9 @@ import { parseSse } from './sse.ts' import { translate } from './translate.ts' import type { WireError } from './types.ts' +/** Constructor options for {@link DeepSeekAdapter}; the plugin's `apply` resolves them from Config + environment. */ export interface DeepSeekAdapterOptions { + /** Bearer token sent in the `authorization` header on every request. */ apiKey: string /** Endpoint base; `/chat/completions` is appended. */ baseURL: string @@ -21,7 +23,11 @@ export interface DeepSeekAdapterOptions { defaults?: RequestDefaults } -/** Map an HTTP status to a stable LlmError code. */ +/** + * Map an HTTP status to a stable LlmError code. + * @param status - status of a non-2xx provider response. + * @returns `AUTH` (401/403), `RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), or `HTTP_` for anything else. + */ export function httpErrorCode(status: number): string { if (status === 401 || status === 403) return 'AUTH' if (status === 429) return 'RATE_LIMIT' diff --git a/packages/llm/llm-deepseek/src/index.ts b/packages/llm/llm-deepseek/src/index.ts index 79313f910f..3a3d7bb4a1 100644 --- a/packages/llm/llm-deepseek/src/index.ts +++ b/packages/llm/llm-deepseek/src/index.ts @@ -34,6 +34,12 @@ export type * from './types.ts' export const name = 'llm-deepseek' export const inject = ['llm'] +/** + * Plugin config, validated by the same-named schemastery schema. Every field + * is optional in yml: credentials/endpoint fall back to the environment (a + * missing API key fails plugin load, not the first call), and omitted + * thinking fields send nothing on the wire, so the provider default applies. + */ export interface Config { /** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */ apiKey?: string diff --git a/packages/llm/llm-deepseek/src/serialize.ts b/packages/llm/llm-deepseek/src/serialize.ts index bbca37223f..10ef905d7e 100644 --- a/packages/llm/llm-deepseek/src/serialize.ts +++ b/packages/llm/llm-deepseek/src/serialize.ts @@ -66,6 +66,8 @@ function serializeAssistant(message: Message): WireMessage { * `{role: 'tool'}` messages; the harness puts each tool result in its own * user-role message, so a mixed user message contributes its text first and * its tool results as separate wire messages after. + * @param messages - the harness conversation, in order. + * @returns the wire messages; order preserved, each tool result expanded into its own entry. */ export function serializeMessages(messages: Message[]): WireMessage[] { const wire: WireMessage[] = [] @@ -97,7 +99,14 @@ export function serializeMessages(messages: Message[]): WireMessage[] { return wire } -/** Build the full wire request. */ +/** + * Build the full wire request. Always streaming (`stream: true`, usage + * reporting on); optional fields are omitted rather than sent as null, so + * provider defaults apply. + * @param options - the harness request (model, history, system, tools, sampling). + * @param defaults - adapter-level thinking defaults; undefined fields put nothing on the wire. + * @returns the chat-completions request body. + */ export function serializeRequest(options: GenerateOptions, defaults: RequestDefaults = {}): WireRequest { const messages: WireMessage[] = [] if (options.system !== undefined) { diff --git a/packages/llm/llm-deepseek/src/sse.ts b/packages/llm/llm-deepseek/src/sse.ts index 252870471a..6856289aec 100644 --- a/packages/llm/llm-deepseek/src/sse.ts +++ b/packages/llm/llm-deepseek/src/sse.ts @@ -37,6 +37,8 @@ function eventData(block: string): string | undefined { * Parse a byte stream into SSE data payloads. Yields `[DONE]` as the final * value and returns; throws `LlmError('STREAM_CLOSED')` when the stream ends * without it (truncated response — the model call cannot be trusted). + * @param stream - raw SSE bytes; reads may split anywhere, including mid-UTF-8 sequence. + * @returns each event's data payload in arrival order, the `[DONE]` sentinel last. */ export async function* parseSse(stream: AsyncIterable): AsyncGenerator { const decoder = new TextDecoder() diff --git a/packages/llm/llm-deepseek/src/translate.ts b/packages/llm/llm-deepseek/src/translate.ts index 08cc019b61..cd6bc8f108 100644 --- a/packages/llm/llm-deepseek/src/translate.ts +++ b/packages/llm/llm-deepseek/src/translate.ts @@ -29,7 +29,11 @@ interface OpenBlock { name?: string } -/** Map the wire finish_reason vocabulary to the harness FinishReason. */ +/** + * Map the wire finish_reason vocabulary to the harness FinishReason. + * @param reason - the wire `finish_reason` string. + * @returns the mapped reason; unrecognized values (content_filter, …) become `{kind: 'error'}` with the uppercased value as `code`. + */ export function mapFinishReason(reason: string): FinishReason { switch (reason) { case 'stop': return { kind: 'stop' } @@ -46,6 +50,8 @@ export function mapFinishReason(reason: string): FinishReason { * (`prompt_tokens = prompt_cache_hit_tokens + prompt_cache_miss_tokens`, * api/create-chat-completion); the harness TokenUsage convention is * DISJOINT counts, so cache reads are subtracted out of `inputTokens`. + * @param usage - wire usage from the finish chunk or the trailing usage-only chunk. + * @returns disjoint harness counts; cache/reasoning fields present only when the wire reported them. */ export function mapUsage(usage: WireUsage): TokenUsage { const cacheRead = usage.prompt_tokens_details?.cached_tokens ?? usage.prompt_cache_hit_tokens @@ -75,6 +81,8 @@ function closeBlock(block: OpenBlock): ContentBlock { /** * Consume SSE data payloads (ending with `[DONE]`) and yield StreamChunks. * Malformed JSON payloads abort the stream with `MALFORMED_RESPONSE`. + * @param payloads - SSE data payloads from {@link parseSse}, `[DONE]`-terminated. + * @returns deltas as they arrive; `block-end`s, `usage`, and `finish` are all deferred to the `[DONE]` sentinel. */ export async function* translate(payloads: AsyncIterable): AsyncGenerator { let nextIndex = 0 diff --git a/packages/llm/llm-deepseek/src/types.ts b/packages/llm/llm-deepseek/src/types.ts index 072c43babb..a9d7403a35 100644 --- a/packages/llm/llm-deepseek/src/types.ts +++ b/packages/llm/llm-deepseek/src/types.ts @@ -48,12 +48,18 @@ export interface WireToolMessage { content: string } +/** One entry of the request `messages` array, discriminated on `role`. */ export type WireMessage = | WireSystemMessage | WireUserMessage | WireAssistantMessage | WireToolMessage +/** + * Assistant-role history message. The harness replays `content: ""` (never + * null) on tool-call-only turns — some gateways reject null — and sends null + * only when the turn carried neither text nor tool calls. + */ export interface WireAssistantMessage { role: 'assistant' content: string | null @@ -66,12 +72,14 @@ export interface WireAssistantMessage { tool_calls?: WireToolCall[] } +/** A completed tool call replayed on an assistant history message; `arguments` is the raw JSON string. */ export interface WireToolCall { id: string type: 'function' function: { name: string; arguments: string } } +/** One entry of the request `tools` array; `parameters` is a JSON Schema object. */ export interface WireTool { type: 'function' function: { @@ -88,11 +96,13 @@ export interface WireChunk { usage?: WireUsage | null } +/** One streamed choice (requests always ask for a single one); `finish_reason` is non-null only on its terminal chunk. */ export interface WireChoice { delta?: WireDelta finish_reason?: string | null } +/** The incremental content of one streamed choice; any subset of fields may be present per chunk. */ export interface WireDelta { role?: string /** Visible text. Null/empty on reasoning/tool-call chunks. */ @@ -105,6 +115,7 @@ export interface WireDelta { tool_calls?: WireToolCallDelta[] } +/** A streamed fragment of one tool call; fragments sharing an `index` concatenate into one call. */ export interface WireToolCallDelta { /** Disambiguates parallel tool calls; stable across a call's deltas. */ index: number @@ -119,6 +130,13 @@ export interface WireToolCallDelta { } } +/** + * Wire token accounting. `prompt_tokens` INCLUDES cache hits (it equals + * `prompt_cache_hit_tokens + prompt_cache_miss_tokens`); `mapUsage` subtracts + * them to keep the harness convention of disjoint counts. + * `prompt_tokens_details.cached_tokens` is the OpenAI-compat spelling of the + * hit count. + */ export interface WireUsage { prompt_tokens: number completion_tokens: number diff --git a/packages/llm/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts index a24e335d8b..00107e8e1c 100644 --- a/packages/llm/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -21,14 +21,22 @@ import { toPiContext, toStreamChunks } from './convert.ts' /** Reasoning levels surfaced by this adapter (DeepSeek wire: high|max). */ export type PiAiReasoning = 'off' | 'high' | 'xhigh' +/** Constructor options for {@link PiAiAdapter}; the plugin's `apply` resolves them from Config + environment. */ export interface PiAiAdapterOptions { + /** Bearer token pi-ai sends on every request. */ apiKey: string + /** Endpoint base; `/chat/completions` is appended. */ baseURL: string /** Thinking level applied to every request ('off' disables thinking). */ reasoning?: PiAiReasoning | undefined } -/** Build the inline pi-ai model descriptor for one DeepSeek model name. */ +/** + * Build the inline pi-ai model descriptor for one DeepSeek model name. + * @param modelId - harness model name; sent verbatim on the wire. + * @param options - adapter options; only `baseURL` is read here (key and reasoning apply per request, not per descriptor). + * @returns a descriptor with every DeepSeek compat flag explicit — pi-ai's URL-based auto-detection is never relied on. + */ export function buildModel(modelId: string, options: PiAiAdapterOptions): Model<'openai-completions'> { return { id: modelId, diff --git a/packages/llm/llm-pi-ai/src/convert.ts b/packages/llm/llm-pi-ai/src/convert.ts index 6fa96a0597..9652cb7d56 100644 --- a/packages/llm/llm-pi-ai/src/convert.ts +++ b/packages/llm/llm-pi-ai/src/convert.ts @@ -55,6 +55,8 @@ function parseArguments(raw: string): Record { * NAME (pi-ai's `toolName`), which the harness doesn't carry on the result * block — it is recovered from the preceding assistant tool-call with the * same id. + * @param options - the harness request; `options.system` maps to pi-ai's single `systemPrompt` slot. + * @returns the pi-ai context; `tools` is omitted entirely when the request declares none. */ export function toPiContext(options: GenerateOptions): PiContext { const toolNames = new Map() @@ -159,7 +161,11 @@ function emptyPiUsage(): PiUsage { } } -/** Map pi-ai usage (reasoning folded into output by pi-ai). */ +/** + * Map pi-ai usage (reasoning folded into output by pi-ai). + * @param usage - cumulative usage from the terminal pi-ai event. + * @returns harness counts; cache fields appear only when non-zero (pi-ai reports zeros, not absence). + */ export function mapUsage(usage: PiUsage): TokenUsage { return { inputTokens: usage.input, @@ -177,7 +183,11 @@ function classifyPiAiError(message: string): string { return 'PI_AI_ERROR' } -/** Map a terminal pi-ai event to the harness finish reason. */ +/** + * Map a terminal pi-ai event to the harness finish reason. + * @param message - the assistant message carried by the `done` or `error` event. + * @returns the harness reason; `error` yields `{kind: 'error'}` with a code classified from the error text. + */ export function mapStopReason(message: AssistantMessage): FinishReason { switch (message.stopReason) { case 'stop': return { kind: 'stop' } @@ -195,6 +205,9 @@ export function mapStopReason(message: AssistantMessage): FinishReason { * Translate the pi-ai event stream into StreamChunks. pi-ai never throws * mid-stream — failures arrive as `error` events, which become error/aborted * `finish` chunks (the harness protocol's other error-delivery style). + * @param events - one assistant turn's pi-ai event stream. + * @returns the harness chunks, ending with `usage` then `finish`; throws + * `LlmError` (`STREAM_CLOSED`) if the source ends without a terminal event. */ export async function* toStreamChunks(events: AsyncIterable): AsyncGenerator { // pi-ai contentIndex ↔ our block index map 1:1 (both count blocks from 0 diff --git a/packages/llm/llm-pi-ai/src/index.ts b/packages/llm/llm-pi-ai/src/index.ts index bef0d4b3f5..43468ba507 100644 --- a/packages/llm/llm-pi-ai/src/index.ts +++ b/packages/llm/llm-pi-ai/src/index.ts @@ -29,6 +29,11 @@ export { mapStopReason, mapUsage, toPiContext, toStreamChunks } from './convert. export const name = 'llm-pi-ai' export const inject = ['llm'] +/** + * Plugin config, validated by the same-named schemastery schema. Every field + * is optional in yml: credentials/endpoint fall back to the environment (a + * missing API key fails plugin load, not the first call). + */ export interface Config { /** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */ apiKey?: string diff --git a/packages/llm/llm/src/assembler.ts b/packages/llm/llm/src/assembler.ts index 328ef01c54..1b8ba6e60c 100644 --- a/packages/llm/llm/src/assembler.ts +++ b/packages/llm/llm/src/assembler.ts @@ -40,6 +40,8 @@ export class BlockAssembler { /** * Feed one chunk. Returns the completed block when the chunk closes one * (an explicit `block-end`), otherwise undefined. + * @param chunk - the next raw chunk, in stream order. + * @returns the authoritative block from the first `block-end` at its index; undefined for every other chunk. */ push(chunk: StreamChunk): ContentBlock | undefined { switch (chunk.type) { @@ -123,20 +125,29 @@ export class BlockAssembler { return partial } - /** Assemble all blocks seen so far, in stream order. */ + /** + * Assemble all blocks seen so far, in stream order. + * @returns one block per seen index; an open block assembles from its + * accumulated deltas (an unknown block type never closed by `block-end` throws). + */ blocks(): ContentBlock[] { return this.order.map(index => this.assemble(this.mustGet(index), index)) } + /** Usage from the `usage` chunk; undefined until one arrives. */ get usage(): TokenUsage | undefined { return this._usage } + /** Finish reason from the `finish` chunk; `{kind: 'stop'}` when the stream ended without one. */ get finish(): FinishReason { return this._finish ?? { kind: 'stop' } } - /** The assembled assistant message. */ + /** + * The assembled assistant message. + * @returns an assistant-role message over `blocks()` (same open-block assembly rules). + */ message(): Message { return { role: 'assistant', content: this.blocks() } } diff --git a/packages/llm/llm/src/attribution.ts b/packages/llm/llm/src/attribution.ts index 61ee2f9ddd..0250062eec 100644 --- a/packages/llm/llm/src/attribution.ts +++ b/packages/llm/llm/src/attribution.ts @@ -54,6 +54,8 @@ export const APP_IDENTITY: AppIdentity = { * The standard `User-Agent` value: `product/version (+url)`. The * parenthesized `+url` comment is the conventional self-identification form * (RFC 9110 §10.1.5 product + comment syntax). + * @param identity - the identity to render; defaults to {@link APP_IDENTITY}. + * @returns the ready-to-send header value. */ export function userAgent(identity: AppIdentity = APP_IDENTITY): string { return `${identity.product}/${identity.version} (+${identity.url})` @@ -63,6 +65,8 @@ export function userAgent(identity: AppIdentity = APP_IDENTITY): string { * Build the attribution headers an adapter must send on every provider * request. Header names are lowercase (HTTP field names are case-insensitive * on the wire). + * @param identity - the identity to send; defaults to {@link APP_IDENTITY} — omission cannot suppress attribution. + * @returns headers to merge into the provider request (currently just `user-agent`). */ export function attributionHeaders( identity: AppIdentity = APP_IDENTITY, diff --git a/packages/llm/llm/src/brand.ts b/packages/llm/llm/src/brand.ts index 3082cc0141..ee1cf786b1 100644 --- a/packages/llm/llm/src/brand.ts +++ b/packages/llm/llm/src/brand.ts @@ -17,7 +17,11 @@ import type { Branded } from '@deepseek-ai/dsh-brand' */ export type CallId = Branded<'CallId'> -/** Brand a string as a {@link CallId}. */ +/** + * Brand a string as a {@link CallId}. + * @param id - the provider-issued (or synthesized) call id. + * @returns the same string, branded; no validation is performed. + */ export function CallId(id: string): CallId { return id as CallId } diff --git a/packages/llm/llm/src/error.ts b/packages/llm/llm/src/error.ts index 6f28898798..2627455789 100644 --- a/packages/llm/llm/src/error.ts +++ b/packages/llm/llm/src/error.ts @@ -18,6 +18,7 @@ * `ErrorOptions`. `name` defaults to the subclass constructor name. */ export class HarnessError extends Error { + /** Stable machine-routable failure class (e.g. `RATE_LIMIT`); route on this, never by parsing `message`. */ readonly code: string constructor(message: string, code: string, options?: ErrorOptions) { @@ -27,7 +28,11 @@ export class HarnessError extends Error { } } -/** Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams). */ +/** + * Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams). + * @param value - the caught value (`unknown` in catch clauses). + * @returns true only for real instances; duck-typed or cross-realm errors do not narrow. + */ export function isHarnessError(value: unknown): value is HarnessError { return value instanceof HarnessError } diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index 0bab45e1bb..667e9bca78 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -73,7 +73,11 @@ export class LlmError extends HarnessError { * same value to the wire. */ export abstract class LlmAdapter { - /** Stream one model call as raw chunks. The only required method. */ + /** + * Stream one model call as raw chunks. The only required method. + * @param options - the fully-assembled request; implementations must honor `options.signal`. + * @returns the chunk stream, obeying the adapter contract documented on `StreamChunk`. + */ abstract stream(options: GenerateOptions): AsyncIterable } diff --git a/packages/llm/llm/src/never.ts b/packages/llm/llm/src/never.ts index 8eed137f15..1243611415 100644 --- a/packages/llm/llm/src/never.ts +++ b/packages/llm/llm/src/never.ts @@ -26,6 +26,9 @@ * variant was added without updating the switch (compile error at the call * site — the desired outcome) or a value escaped its type (runtime throw * with diagnostics — the safety net). + * @param value - the impossible value; typed `never` so an unhandled variant fails compilation at the call site. + * @param context - optional label (e.g. the switch site) prefixed into the throw message. + * @returns never — it always throws, with the offending value JSON-rendered in the message. */ export function assertNever(value: never, context?: string): never { // JSON.stringify is typed string but returns undefined for undefined input; diff --git a/packages/llm/llm/src/types.ts b/packages/llm/llm/src/types.ts index 7163ddc1d9..1644885a73 100644 --- a/packages/llm/llm/src/types.ts +++ b/packages/llm/llm/src/types.ts @@ -70,7 +70,9 @@ export interface ContentBlockMap { 'tool-result': ToolResultBlock } +/** The block `type` tag vocabulary; widens as plugins merge new shapes into {@link ContentBlockMap}. */ export type ContentBlockType = keyof ContentBlockMap +/** Any known content block, derived from {@link ContentBlockMap}; switch on `type` and fall through unknowns (merge-extensible). */ export type ContentBlock = ContentBlockMap[ContentBlockType] /** A single message in a conversation history. */ @@ -88,6 +90,7 @@ export interface MessageSourceMap { plugin: { kind: 'plugin'; plugin: string } } +/** Any known message source, derived from {@link MessageSourceMap}; switch on `kind` and fall through unknowns (merge-extensible). */ export type MessageSource = MessageSourceMap[keyof MessageSourceMap] /** @@ -102,6 +105,7 @@ export interface FinishReasonMap { 'error': { kind: 'error'; message: string; code?: string } } +/** Any known finish reason, derived from {@link FinishReasonMap}; switch on `kind` and fall through unknowns (merge-extensible). */ export type FinishReason = FinishReasonMap[keyof FinishReasonMap] /** diff --git a/packages/session-persistence/session-persistence-jsonl/src/format.ts b/packages/session-persistence/session-persistence-jsonl/src/format.ts index 63cf899e45..5ec01283b4 100644 --- a/packages/session-persistence/session-persistence-jsonl/src/format.ts +++ b/packages/session-persistence/session-persistence-jsonl/src/format.ts @@ -27,7 +27,11 @@ export interface HeaderLine { seedLength?: number } -/** Build the header line object from a {@link SessionHeader}. */ +/** + * Build the header line object from a {@link SessionHeader}. + * @param header - the immutable session metadata to serialize. + * @returns the `type: 'session'`-tagged line object, absent optional fields omitted (never null). + */ export function toHeaderLine(header: SessionHeader): HeaderLine { return { type: 'session', @@ -40,7 +44,11 @@ export function toHeaderLine(header: SessionHeader): HeaderLine { } } -/** Parse a header line back into a {@link SessionHeader}. */ +/** + * Parse a header line back into a {@link SessionHeader}. + * @param line - the shape-checked first line of a log (see the `isHeaderLine` guard). + * @returns the header, absent optional fields omitted. + */ export function fromHeaderLine(line: HeaderLine): SessionHeader { return { version: line.version, @@ -77,6 +85,8 @@ function isHeaderLine(value: unknown): value is HeaderLine { * `Buffer.from(…, 'utf8')` would do, breaking injectivity). `.` is in the safe * set for readability but the whole-segment tokens `.`/`..` are escaped so they * can never traverse. + * @param raw - the string to encode; must be non-empty (throws on `''`). + * @returns the escaped single path segment, decodable back to `raw`. */ export function encodeSegment(raw: string): string { if (raw.length === 0) throw new Error('cannot encode an empty path segment') @@ -97,9 +107,12 @@ export function encodeSegment(raw: string): string { /** * The directory a session's files live in: the configured root, then a per-cwd - * subdirectory so sessions group by project. The cwd subdir is a stable hash - * (short, collision-resistant, filesystem-safe) plus an encoded suffix for - * readability; sessions without a cwd go in a shared `_no-cwd` bucket. + * subdirectory so sessions group by project. The cwd subdir is a stable hash of + * the cwd (short, collision-resistant, filesystem-safe); sessions without a + * cwd go in a shared `_no-cwd` bucket. + * @param root - the backend's session root directory. + * @param cwd - the session's project directory; `undefined` selects the shared `_no-cwd` bucket. + * @returns the per-cwd bucket directory path under `root`. */ export function sessionDir(root: string, cwd: string | undefined): string { if (cwd === undefined) return join(root, '_no-cwd') @@ -107,12 +120,22 @@ export function sessionDir(root: string, cwd: string | undefined): string { return join(root, `cwd-${hash}`) } -/** The append-only event-log file path for a session. */ +/** + * The append-only event-log file path for a session. + * @param root - the backend's session root directory. + * @param cwd - the session's project directory (picks the per-cwd bucket; `undefined` → `_no-cwd`). + * @param id - the session id, path-encoded via {@link encodeSegment} before filesystem use. + * @returns the session's `.jsonl` log file path. + */ export function logPath(root: string, cwd: string | undefined, id: SessionId): string { return join(sessionDir(root, cwd), `${encodeSegment(id)}.jsonl`) } -/** Serialize one event as a JSONL line (no trailing newline). */ +/** + * Serialize one event as a JSONL line (no trailing newline). + * @param event - the event to serialize verbatim. + * @returns the event's single-line JSON text; the writer adds the newline. + */ export function eventLine(event: SessionEvent): string { return JSON.stringify(event) } @@ -135,6 +158,9 @@ export function eventLine(event: SessionEvent): string { * This relies on the session-log invariant that every event lives inside a turn * (`Session.append` enforces it): only the final turn can be open, so the * preserved tail is at most one unclosed turn. + * @param buffer - the raw bytes of the log file (header line first). + * @returns the header, the preserved event prefix, and `committedBytes` — the + * byte offset the next append truncates any torn tail to. */ export function scanLog(buffer: Buffer): { meta: SessionHeader; events: SessionEvent[]; committedBytes: number } { const text = buffer.toString('utf8') @@ -239,6 +265,8 @@ export function scanLog(buffer: Buffer): { meta: SessionHeader; events: SessionE * `undefined` if it is missing/not a header. Used by `list()` to read session * metadata WITHOUT parsing the whole log: a session picker scales with the * number of sessions, not the total size of every conversation. + * @param firstLine - the first line of a log file (without its trailing newline). + * @returns the parsed header, or `undefined` when the line is not a well-formed session header. */ export function parseHeaderMeta(firstLine: string): SessionHeader | undefined { let parsed: unknown diff --git a/packages/session-persistence/session-persistence-jsonl/src/index.ts b/packages/session-persistence/session-persistence-jsonl/src/index.ts index 6de7eafeb3..a69c979756 100644 --- a/packages/session-persistence/session-persistence-jsonl/src/index.ts +++ b/packages/session-persistence/session-persistence-jsonl/src/index.ts @@ -31,6 +31,7 @@ import { encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine, } from './format.ts' +/** Plugin config: where the JSONL backend keeps its session logs (`root` is required — no default). */ export interface Config { /** * Root directory for all session files. Required (no default): a default of diff --git a/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts index c6b7903357..6eed63a2f4 100644 --- a/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts +++ b/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts @@ -23,6 +23,15 @@ afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) +function appendClosedTurn(session: Session): void { + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { + content: [{ type: 'text', text: 'hello' }], + source: { kind: 'user' }, + }, { surfaceOp: 'append' }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) +} + // Run the shared backend contract against the real JSONL backend. runPersistenceContract('jsonl', async () => { const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-')) @@ -131,6 +140,23 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => { expect(loaded.events).toEqual(log) // chunks preserved, contiguous seqs }) + it('persists a forked child seed through the existing session write path', async () => { + const source = ctx.sessions.create(SessionId('persist-parent'), { meta: { cwd: '/workspace' } }) + appendClosedTurn(source) + + const child = ctx.sessions.fork(source, undefined, SessionId('persist-child')) + await ctx.parallel('session/flush', child) + const loaded = await ctx.sessionPersistence.load(child.id) + + expect(loaded.events).toEqual(source.events) + expect(loaded.meta).toMatchObject({ + id: SessionId('persist-child'), + cwd: '/workspace', + parentSession: SessionId('persist-parent'), + seedLength: source.events.length, + }) + }) + it('crash recovery: load preserves the interrupted turn and closes it with a synthetic turn/end {interrupted}', async () => { const m = meta('crash', '/proj') await ctx.sessionPersistence.create(m) diff --git a/packages/session-persistence/session-persistence-sqlite/README.md b/packages/session-persistence/session-persistence-sqlite/README.md index d7095633c9..c14d3a70ea 100644 --- a/packages/session-persistence/session-persistence-sqlite/README.md +++ b/packages/session-persistence/session-persistence-sqlite/README.md @@ -8,7 +8,7 @@ A SQLite durable session-persistence backend — a second `SessionPersistence` i Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../docs/rfc/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`) lives in a `sessions` row. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row), so no separate column is needed. -The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and the configured `journal_mode` (default `wal`; pick a rollback-journal mode like `delete` on filesystems where WAL's shared-memory files do not work, e.g. network mounts). The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software). +The repo's `engines.node` is `^22.19.0 || >=24.0.0` (Node 22.19+ or 24+), matching the LTS floor required by the installed Pi adapter dependency; `node:sqlite` itself ships without the `--experimental-sqlite` flag from Node 22.13 (LTS) and 23.4 / 24 (Current) on. The range deliberately excludes Node 23 because that line is non-LTS/EOL and still has flagged runtime features before 23.6. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and the configured `journal_mode` (default `wal`; pick a rollback-journal mode like `delete` on filesystems where WAL's shared-memory files do not work, e.g. network mounts). The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software). ## Contract semantics over rows diff --git a/packages/session-persistence/session-persistence-sqlite/src/schema.ts b/packages/session-persistence/session-persistence-sqlite/src/schema.ts index 2ed6f5853c..2dacbe04a0 100644 --- a/packages/session-persistence/session-persistence-sqlite/src/schema.ts +++ b/packages/session-persistence/session-persistence-sqlite/src/schema.ts @@ -76,6 +76,9 @@ export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist' * is the merged layout carrying every column; bumping past the collided v3 * makes the version check reject both sibling v3 databases instead of opening * one against columns it does not have. + * @param path - the SQLite database file to open (created when absent). + * @param journalMode - the journal pragma to apply — a closed in-code union, validated by the plugin Config. + * @returns the open handle with pragmas applied and both tables ensured. */ export function openDatabase(path: string, journalMode: JournalMode): DatabaseSync { const db = new DatabaseSync(path) @@ -120,7 +123,11 @@ export function openDatabase(path: string, journalMode: JournalMode): DatabaseSy return db } -/** Reconstruct the {@link SessionHeader} from a `sessions` row. */ +/** + * Reconstruct the {@link SessionHeader} from a `sessions` row. + * @param row - the `sessions` table row. + * @returns the header, `NULL` columns mapped to omitted optional fields. + */ export function rowToMeta(row: SessionRow): SessionHeader { return { version: row.version, @@ -132,7 +139,12 @@ export function rowToMeta(row: SessionRow): SessionHeader { } } -/** Reconstruct a {@link SessionEvent} from an `events` row (parses `data`). */ +/** + * Reconstruct a {@link SessionEvent} from an `events` row (parses `data`). + * @param row - the `events` table row; `data` and the surface columns hold JSON text. + * @returns the reconstructed event; throws when a JSON column fails to parse + * ({@link scanRows} treats that as a hole, not corruption, in the tail). + */ export function rowToEvent(row: EventRow): SessionEvent { // Surface-metadata fields are conditional on the event type in the type // system; spread them so each variant gets only the fields it declares. @@ -172,6 +184,9 @@ export function rowToEvent(row: EventRow): SessionEvent { * This relies on the session-log invariant that every event lives inside a turn * (`Session.append` enforces it): only the final turn can be open, so the * preserved tail is at most one unclosed turn. + * @param rows - one session's event rows, ordered by seq ascending. + * @returns the preserved event prefix, plus `tornFrom` — the seq the physical + * delete starts at — when a torn tail exists. */ export function scanRows(rows: readonly EventRow[]): { preserved: SessionEvent[]; tornFrom?: number } { // Pass 1: parse each row's data; a row whose data is not valid JSON is a hole. diff --git a/packages/session-persistence/session-persistence/src/coordinator.ts b/packages/session-persistence/session-persistence/src/coordinator.ts index 7c7b044ee1..0180999842 100644 --- a/packages/session-persistence/session-persistence/src/coordinator.ts +++ b/packages/session-persistence/session-persistence/src/coordinator.ts @@ -186,6 +186,7 @@ export class PersistenceCoordinator { /** * Register a new session's metadata (lazy: no physical write until the first * {@link append}). Rejects if the id is already tracked or already persisted. + * @param meta - the immutable header (id, version, cwd, lineage) to record; snapshotted at call time. */ create(meta: SessionHeader): Promise { // Snapshot the metadata at call time: the op runs later (behind the @@ -216,6 +217,8 @@ export class PersistenceCoordinator { /** * Durably persist a batch of events. Honors the append-only and contiguous-seq * contracts; rejects non-JSON-serializable `event.data`. + * @param id - the session the batch belongs to. + * @param events - the contiguous batch to persist, in seq order; deep-cloned at call time. */ async append(id: SessionId, events: readonly SessionEvent[]): Promise { // Validate serializability BEFORE cloning so a bad event surfaces the typed @@ -252,6 +255,8 @@ export class PersistenceCoordinator { * Reload a session: its {@link SessionHeader} plus the event log up to the last * durable checkpoint, with any interrupted final turn durably closed (synthetic * boundary events) during load. + * @param id - the persisted session to reload. + * @returns the header plus the event log, ending on a balanced `turn/end`. */ load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { return this.serialize(id, () => this.loadCore(id)) diff --git a/packages/session-persistence/session-persistence/src/index.ts b/packages/session-persistence/session-persistence/src/index.ts index 5e1af6e563..1588ed2526 100644 --- a/packages/session-persistence/session-persistence/src/index.ts +++ b/packages/session-persistence/session-persistence/src/index.ts @@ -45,6 +45,9 @@ declare module 'cordis' { * * The comparison includes the full event payload, not just seq/type/time, so a * mutated seed cannot be grafted onto a durable log with the same envelope. + * @param seed - the live session's creation-time event snapshot. + * @param prefix - the persisted prefix the seed must reproduce. + * @returns `true` when the prefix fits within the seed and every event matches by JSON text. */ export function seedCoversPrefix(seed: readonly SessionEvent[], prefix: readonly SessionEvent[]): boolean { return prefix.length <= seed.length @@ -58,6 +61,7 @@ export function seedCoversPrefix(seed: readonly SessionEvent[], prefix: readonly * Reject non-JSON-serializable event data before a backend serializes a batch. * Live session appends already enforce this; persistence append paths also * accept replay/fork batches that may bypass a live session instance. + * @param events - the batch to validate; throws naming the offending event's type and seq. */ export function assertSerializable(events: readonly SessionEvent[]): void { for (const event of events) { diff --git a/packages/subagent/subagent-acp/src/run.ts b/packages/subagent/subagent-acp/src/run.ts index 74291b7e65..d7631f4d8d 100644 --- a/packages/subagent/subagent-acp/src/run.ts +++ b/packages/subagent/subagent-acp/src/run.ts @@ -121,7 +121,12 @@ export const DEFAULT_DISPOSE_GRACE_MS = 3_000 */ export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i -/** The ambient env minus credential-shaped vars, plus the spec's explicit env. */ +/** + * The ambient env minus credential-shaped vars, plus the spec's explicit env. + * @param extra - explicit vars layered on top AFTER the scrub, so a + * credential-shaped name supplied deliberately still reaches the child. + * @returns the environment to spawn the child with. + */ export function buildChildEnv(extra: Record): NodeJS.ProcessEnv { const env: NodeJS.ProcessEnv = {} for (const [key, value] of Object.entries(process.env)) { @@ -130,7 +135,12 @@ export function buildChildEnv(extra: Record): NodeJS.ProcessEnv return { ...env, ...extra } } -/** Map an ACP {@link StopReason} to a harness {@link SubagentStopReason}. */ +/** + * Map an ACP {@link StopReason} to a harness {@link SubagentStopReason}. + * @param reason - the terminal reason from the child's `session/prompt` response. + * @returns the harness equivalent; `max_turn_requests` and any unknown future + * variant map to `error`, so an unclean stop is never reported as `completed`. + */ export function acpStopReason(reason: StopReason): SubagentStopReason { switch (reason) { case 'end_turn': @@ -155,12 +165,20 @@ export function acpStopReason(reason: StopReason): SubagentStopReason { } } -/** Collect the text of an ACP content block (non-text blocks contribute nothing). */ +/** + * Collect the text of an ACP content block (non-text blocks contribute nothing). + * @param content - the content block off a streamed `agent_message_chunk`. + * @returns the block's text, or `''` for a non-text block. + */ export function acpContentText(content: AcpContentBlock): string { return content.type === 'text' ? content.text : '' } -/** Translate the harness prompt blocks into ACP prompt blocks (text only). */ +/** + * Translate the harness prompt blocks into ACP prompt blocks (text only). + * @param prompt - the harness prompt; non-text blocks are dropped. + * @returns the ACP text blocks, in order. + */ export function toAcpPrompt(prompt: ContentBlock[]): AcpContentBlock[] { const blocks: AcpContentBlock[] = [] for (const block of prompt) { @@ -206,6 +224,11 @@ function exitsWithin(child: ChildProcess, ms: number): Promise { * failure (a spawn/transport/RPC error resolves with `stopReason: 'error'`), per * the seam contract. `cancel()` sends `session/cancel`; `dispose()` kills the * subprocess and awaits its exit (quiescent teardown). + * @param request - the start request; the driver consumes `prompt` and `signal` + * (an already-aborted signal yields an inert `aborted` run with no spawn). + * @param spec - the resolved spawn spec: command/args/cwd, env, permission + * policy, dispose graces, and the optional error sink. + * @returns the live run handle for the child subprocess. */ export function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpec): SubagentRun { const id = AgentId(randomUUID()) diff --git a/packages/subagent/subagent-fork/README.md b/packages/subagent/subagent-fork/README.md index c691d56355..1abd7951fd 100644 --- a/packages/subagent/subagent-fork/README.md +++ b/packages/subagent/subagent-fork/README.md @@ -12,7 +12,7 @@ The seam this rides on: `CreateAgentOptions.seed` (added on `dsh-agent`, threade ## Capabilities -`{ outputSchema: false, depthLimit: true, toolFilter: false }` — identical to spawn (the depth/model/output behavior is the shared driver's). +`{ outputSchema: true, depthLimit: true, toolFilter: false }` — identical to spawn (the depth/model/structured-output behavior is the shared driver's). ## Config diff --git a/packages/subagent/subagent-fork/src/index.ts b/packages/subagent/subagent-fork/src/index.ts index b6d0c10e44..d8c77a03ac 100644 --- a/packages/subagent/subagent-fork/src/index.ts +++ b/packages/subagent/subagent-fork/src/index.ts @@ -28,6 +28,10 @@ import type { SubagentCapabilities, SubagentProvider, SubagentStartRequest } fro import { startInProcessRun } from '@deepseek-ai/dsh-subagent-inprocess' export const name = 'subagent-fork' +// `tools` is deliberately NOT injected — same rationale as subagent-spawn: the +// per-run structured runtime gates its capture-tool registration on `tools` +// itself, so this backend's apply timing (and the delegation tool's position +// in the model-visible tool list) is unchanged by structured output. export const inject = ['subagents', 'agents'] /** Config: the registry name to register the provider under. */ @@ -47,6 +51,8 @@ export const Config: z = z.object({ * empty — i.e. fresh — child). The result is contiguous from seq 0 (the live * log keeps `seq === index`), so it is a valid session seed; the in-flight, * unbalanced turn is dropped so the invariants replay accepts it. + * @param parent - the agent whose session log to slice. + * @returns the seed events, contiguous from seq 0; empty when no turn has completed. */ export function completedTurnPrefix(parent: Agent): SessionEvent[] { const events = parent.session.events @@ -57,11 +63,12 @@ export function completedTurnPrefix(parent: Agent): SessionEvent[] { } /** - * The fork provider. Supports `depthLimit`; NOT `outputSchema`/`toolFilter` this - * cut (the service rejects a request needing either before `start` runs). + * The fork provider. Supports `depthLimit` and `outputSchema` (via the shared + * in-process structured runtime); NOT `toolFilter` this cut (the service + * rejects a request needing it before `start` runs). */ class ForkProvider implements SubagentProvider { - readonly capabilities: SubagentCapabilities = { outputSchema: false, depthLimit: true, toolFilter: false } + readonly capabilities: SubagentCapabilities = { outputSchema: true, depthLimit: true, toolFilter: false } // Context contract: a forked child IS seeded with the parent's completed-turn prefix. readonly inheritsParentContext = true diff --git a/packages/subagent/subagent-fork/tests/subagent-fork.spec.ts b/packages/subagent/subagent-fork/tests/subagent-fork.spec.ts index 56441a656f..90e2d583d8 100644 --- a/packages/subagent/subagent-fork/tests/subagent-fork.spec.ts +++ b/packages/subagent/subagent-fork/tests/subagent-fork.spec.ts @@ -9,9 +9,10 @@ import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' import AgentLoop from '@deepseek-ai/dsh-agent-loop' import * as Invariants from '@deepseek-ai/dsh-invariants' import SubagentService from '@deepseek-ai/dsh-subagent' -import { MockAdapter, textResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' +import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' import type { StreamChunk } from '@deepseek-ai/dsh-llm' import * as fork from '../src/index.ts' +import { STRUCTURED_OUTPUT_TOOL } from '@deepseek-ai/dsh-subagent-inprocess' import { completedTurnPrefix } from '../src/index.ts' type Script = ConstructorParameters[0] @@ -141,6 +142,26 @@ describe('dsh-subagent-fork', () => { await run.dispose() }) + it('captures structured output through the shipped plugin (seeded child, driver runtime)', async () => { + const { ctx, parent } = await setup([ + textResponse('parent turn'), + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 9 }), + ]) + parent.send([{ type: 'text', text: 'warm up' }]) + await parent.whenIdle() + const run = ctx.subagents.start('fork', { + prompt: [{ type: 'text', text: 'report structured' }], + parent, + outputSchema: { type: 'object', properties: { answer: { type: 'number' } }, required: ['answer'] }, + }) + const result = await run.result + expect(result.stopReason).toBe('completed') + expect(result.structured).toEqual({ answer: 9 }) + // Run-scoped runtime: nothing stays registered after the settle. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + await run.dispose() + }) + it('does NOT return the seeded parent output when the child produces no message of its own', async () => { // Regression: readResult must scope to the child's OWN events (after the // seed). The parent completes a turn with a distinctive assistant message, @@ -161,9 +182,9 @@ describe('dsh-subagent-fork', () => { await run.dispose() }) - it('advertises depthLimit but not outputSchema/toolFilter', async () => { + it('advertises depthLimit and outputSchema but not toolFilter', async () => { const { ctx } = await setup([]) - expect(ctx.subagents.getProvider('fork')!.capabilities).toEqual({ outputSchema: false, depthLimit: true, toolFilter: false }) + expect(ctx.subagents.getProvider('fork')!.capabilities).toEqual({ outputSchema: true, depthLimit: true, toolFilter: false }) }) it('unregisters the provider when its fiber is disposed (HMR safety)', async () => { diff --git a/packages/subagent/subagent-inprocess/README.md b/packages/subagent/subagent-inprocess/README.md index af6d5792a9..f6870929a8 100644 --- a/packages/subagent/subagent-inprocess/README.md +++ b/packages/subagent/subagent-inprocess/README.md @@ -8,10 +8,10 @@ The shared **in-process subagent run driver**. A pure library (no provider, no r Runs a child as a child [`Agent`](../../core/agent) on the same cordis context (`ctx.agents`): -1. computes child depth = `depthOf(parent) + 1`; if `request.maxDepth` is set and exceeded, throws `SubagentDepthError` (the `depthLimit` capability); -2. creates a child via `ctx.agents.create` with a fresh `AgentId`/`SessionId`, the parent's `cwd` + `parentSession` lineage, the optional `options.seed` (fork's completed-turn prefix; omitted for a fresh child), and `agentOptions` (the child inherits the **parent's model** by default — a child with no model can't run — overridable via `request.agentOptions.model`; the system prompt is NOT inherited); -3. drives the one-shot: `child.send(prompt)` then `await child.whenIdle()` (ordering matters — `send` enqueues synchronously, so `whenIdle` observes the queued work and resolves on the child's `running → idle` transition, never before the turn starts); -4. reads the result, scoped to the child's OWN events (everything at or after `seedLength`, so a seeded child that produced no message of its own never returns the seeded parent's last message): the last `assistant/message` content (deep-cloned — the log is frozen) and the last `turn/end.reason` mapped to a `SubagentStopReason`. +1. computes child depth = `depthOf(parent) + 1`; if `request.maxDepth` is set and exceeded, throws `SubagentDepthError` (the `depthLimit` capability); a `request.outputSchema` is asserted against the supported subset (`assertSupportedOutputSchema` from [dsh-tools](../../core/tools/README.md)) and then snapshotted with `structuredClone` before any child exists — assertion first so a hostile value fails as `OutputSchemaError` (never a raw clone error), the snapshot so a post-`start()` caller mutation cannot drift the enforced schema; +2. creates a child via `ctx.agents.create` with a fresh `AgentId`/`SessionId`, the parent's `cwd` + `parentSession` lineage, the optional `options.seed` (fork's completed-turn prefix; omitted for a fresh child), and `agentOptions` (the child inherits the **parent's model** by default — a child with no model can't run — overridable via `request.agentOptions.model`; the deployment persona needs no inheritance — it is a context-wide prompt section); +3. drives the one-shot: `child.send(prompt)` then `await child.whenIdle()` (ordering matters — `send` enqueues synchronously, so `whenIdle` observes the queued work and resolves on the child's `running → idle` transition, never before the turn starts); there is deliberately NO re-prompt for a structured child that finished cleanly without calling `structured_output` — the shortfall maps to an `error` result for the parent; +4. reads the result, scoped to the child's OWN events (everything at or after `seedLength`, so a seeded child that produced no message of its own never returns the seeded parent's last message): the last `assistant/message` content (deep-cloned — the log is frozen) and the last `turn/end.reason` mapped to a `SubagentStopReason`. A structured run surfaces the captured value as `result.structured`; a structured child that finished cleanly WITHOUT ever capturing settles `error` (a clean finish without the demanded result is a failure, not a success with a missing field). `dispose()` delegates to `AgentHandle.dispose()` (stop loop → await quiescence → remove session); `cancel()` cancels the child's in-flight turn. A cancel landing before any `turn/end` (the pre-turn window) still settles `aborted`, honoring the cancel contract rather than the generic no-turn `error`. @@ -19,6 +19,19 @@ Runs a child as a child [`Agent`](../../core/agent) on the same cordis context ( `{ providerName: string; seed?: SessionEvent[] }` — the per-backend inputs: the provider name (for error context) and the optional child-session seed. +### Structured output (package-internal runtime) + +The mechanism behind `outputSchema` for in-process children — acquired per structured RUN inside `startInProcessRun` (nothing is registered on a context that never runs a structured child; only the model-facing constants `STRUCTURED_OUTPUT_TOOL`/`STRUCTURED_OUTPUT_INSTRUCTION` are exported). One globally registered `structured_output` capture tool (its registered parameters are a placeholder) plus four listeners: + +- a `system-prompt/assemble` waterfall listener registered `prepend: true` that post-processes `await next()` — **final-assembly enforcement**: the assembly the loop renders never carries `structured_output` for an agent without a structured run, and for one that has it always carries the run's OWN schema (as the tool's `parameters`) plus the calling instruction as a trailing prompt section (the demand travels with the tool — `AgentOptions` has no per-agent prompt field to carry it). The loop logs the rendered assembly as the step's `request/header`, so the injection is reconstructable log state, never a wire-only mutation. Per-agent shaping lives here because the tool registry and prompt assembly are context-global while schemas differ per concurrent child (FIXME in the module doc: per-agent/per-session scoping would dissolve this); cooperative mutate-then-`next()` would not survive a downstream listener returning a replacement assembly. +- a `tools/post-execute` listener (`prepend: true` = outermost, so `await next()` yields the composed final decision) that COMMITS the capture: the tool body only stages the validated value, and it becomes the run's result only when the final decision accepts the call — a downstream block (a PostToolUse hook) turns the logged result into `isError`, and the run must not report `structured` success for a call the model and session log saw fail. +- a `tools/pre-execute` deny for any call arriving after the agent's capture — terminal means terminal WITHIN the step: a response listing `structured_output` before further tool calls cannot run side effects after the final answer was accepted. +- an `agent/turn-continuation` listener (also `prepend: true` — an earlier-registered force-continue listener returning without `next()` must not decide the turn before the veto runs) that stops a child's turn once its output is captured, so a successful capture doesn't buy a wasted extra model step. + +The capture tool validates each call against the run's schema (`validateStructuredValue`) — violations become an `INVALID_ARGS` isError result the model retries in-turn; a valid call stages the value for the post-execute commit. + +Lifetime is refcounted by live structured runs: each acquires at start and releases at settle, so the registrations exist exactly while at least one structured child is live, a backend hot-reload mid-run cannot unregister the capture tool under a live child, and the last settle disposes everything. `release()` is idempotent per acquisition. + ### `depthOf(agent): number` Delegation depth rides on a merge-extensible `AgentOptions.subagentDepth` field (0 for a top-level agent, parent + 1 for a child), so a nested spawn reads its parent's depth from `parent.options.subagentDepth`. `depthOf` reads it (absent ⇒ 0). diff --git a/packages/subagent/subagent-inprocess/package.json b/packages/subagent/subagent-inprocess/package.json index f3bd774554..4e6b72533a 100644 --- a/packages/subagent/subagent-inprocess/package.json +++ b/packages/subagent/subagent-inprocess/package.json @@ -26,6 +26,8 @@ "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "@deepseek-ai/dsh-subagent": "^0.0.1", + "@deepseek-ai/dsh-system-prompt": "^0.0.1", + "@deepseek-ai/dsh-tools": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { diff --git a/packages/subagent/subagent-inprocess/src/index.ts b/packages/subagent/subagent-inprocess/src/index.ts index 1107926aa2..9026954b97 100644 --- a/packages/subagent/subagent-inprocess/src/index.ts +++ b/packages/subagent/subagent-inprocess/src/index.ts @@ -18,7 +18,20 @@ import type { Context } from 'cordis' import { AgentId, type Agent, type AgentHandle, type AgentOptions } from '@deepseek-ai/dsh-agent' import { SessionId, type SessionEvent, type TurnEndReason } from '@deepseek-ai/dsh-session' import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import { assertSupportedOutputSchema } from '@deepseek-ai/dsh-tools' import type { SubagentResult, SubagentRun, SubagentStartRequest, SubagentStopReason } from '@deepseek-ai/dsh-subagent' +import { + acquireStructuredRuntime, + type StructuredAcquisition, +} from './structured.ts' + +// The runtime itself (acquire/attach/release) is package-internal: runs +// acquire it inside startInProcessRun, and no other package drives it. Only +// the model-facing vocabulary is public. +export { + STRUCTURED_OUTPUT_TOOL, + STRUCTURED_OUTPUT_INSTRUCTION, +} from './structured.ts' declare module '@deepseek-ai/dsh-agent' { interface AgentOptions { @@ -34,7 +47,11 @@ declare module '@deepseek-ai/dsh-agent' { } } -/** Read an agent's delegation depth (absent ⇒ a top-level agent, depth 0). */ +/** + * Read an agent's delegation depth (absent ⇒ a top-level agent, depth 0). + * @param agent - the agent whose options may carry `subagentDepth`. + * @returns 0 for a top-level agent, its parent's depth + 1 for a subagent. + */ export function depthOf(agent: Agent): number { return agent.options.subagentDepth ?? 0 } @@ -88,6 +105,13 @@ export interface InProcessRunOptions { * the matching `turn/end.reason` the stop reason. `dispose()` delegates to the * factory's {@link AgentHandle.dispose} (stop loop → await quiescence → remove * session); `cancel()` cancels the child's in-flight turn. + * + * Throws {@link SubagentDepthError} before creating anything when the child's + * depth (parent depth + 1) would exceed `request.maxDepth`. + * @param ctx - the context whose `agents` factory creates and owns the child. + * @param request - the start request (prompt, parent, signal, per-child options). + * @param options - the backend's inputs: provider name plus the optional seed. + * @returns the live run handle for the child agent. */ export function startInProcessRun( ctx: Context, @@ -98,6 +122,18 @@ export function startInProcessRun( if (request.maxDepth !== undefined && childDepth > request.maxDepth) { throw new SubagentDepthError(childDepth, request.maxDepth) } + // Assert, then snapshot, the schema subset BEFORE any child exists (the + // service has already capability-gated; this rejects a schema outside the + // enforced subset loud). Assertion comes FIRST so a hostile value fails as + // OutputSchemaError, never as structuredClone's raw DataCloneError — the + // asserted subset is plain JSON data, which always clones. The snapshot is + // load-bearing: the caller keeps its reference, so attaching the ORIGINAL + // would let a post-start() mutation drift the enforced schema away from the + // asserted one — the clone (taken synchronously with the assertion, no + // interleaving possible) pins assertion, the model-visible parameters, and + // validateStructuredValue to one isolation-immutable value. + if (request.outputSchema !== undefined) assertSupportedOutputSchema(request.outputSchema) + const schema = request.outputSchema === undefined ? undefined : structuredClone(request.outputSchema) const childId = AgentId(randomUUID()) // The child's OWN events begin after the seed (fork seeds the parent's @@ -109,13 +145,20 @@ export function startInProcessRun( // Inherit the parent's model by default (a child with no model cannot run); // an explicit `request.agentOptions.model` overrides it. The persona needs // no inheritance: the deployment persona is a context-wide prompt section, - // so parent and child render the same one. + // so parent and child render the same one. A structured run's + // structured_output instruction is NOT prompt state either — the structured + // runtime's final-request listener appends it per request (see structured.ts). const agentOptions: AgentOptions = { ...request.parent.options.model !== undefined ? { model: request.parent.options.model } : {}, ...request.agentOptions, subagentDepth: childDepth, } + // The structured runtime is held for the WHOLE run (acquired before the child + // exists, released when the result settles), so a backend hot-reload mid-run + // cannot unregister the capture tool out from under this live child. + const structured: StructuredAcquisition | undefined = schema !== undefined ? acquireStructuredRuntime(ctx) : undefined + const handle: AgentHandle = ctx.agents.create({ agentId: childId, sessionId: SessionId(randomUUID()), @@ -130,6 +173,7 @@ export function startInProcessRun( agentOptions, }) const child = handle.agent + if (structured && schema !== undefined) structured.attach(child, schema) // Bridge the request's abort signal to the child (the consumer also bridges // its own exec.signal, but a backend-level bridge keeps the contract local). @@ -138,6 +182,10 @@ export function startInProcessRun( // `turn/end` is logged — settles as `aborted` (honoring the cancel contract) // rather than falling through to the no-turn `error` mapping. let cancelled = false + // An accessor, not an inline read: `cancelled` mutates from closures (the + // abort listener, run.cancel), which control-flow narrowing cannot see — an + // inline read at the result mapping would narrow to the initializer. + const isCancelled = (): boolean => cancelled const requestCancel = (reason: string): void => { cancelled = true child.cancel(reason) @@ -154,9 +202,16 @@ export function startInProcessRun( if (request.signal?.aborted) return { output: [], stopReason: 'aborted' } child.send(request.prompt) await child.whenIdle() - return readResult(child, seedLength, cancelled) + // Deliberately NO re-prompt when a structured child finishes cleanly + // without calling structured_output: readResult maps that to `error` — + // the shortfall goes to the parent instead of buying extra model turns. + return readResult(child, seedLength, isCancelled(), structured ? { captured: structured.captured(child) } : undefined) } finally { request.signal?.removeEventListener('abort', onAbort) + if (structured) { + structured.detach(child) + structured.release() + } } })() @@ -184,12 +239,32 @@ export function startInProcessRun( * logged (a cancel landed in the pre-turn window, before any turn ran), the * run settles `aborted` per the {@link SubagentRun.cancel} contract rather than * the generic no-turn `error`. + * + * A structured run (`structured` present) additionally reports the captured + * value on {@link SubagentResult.structured}. A structured child that finished + * CLEANLY without ever capturing (the nudges ran out) settles `error` — a clean + * finish without the demanded structured result is a failure, not a success + * with a missing field; a non-`completed` reason keeps its own honest mapping. */ -function readResult(child: Agent, seedLength: number, cancelled: boolean): SubagentResult { +function readResult( + child: Agent, + seedLength: number, + cancelled: boolean, + structured?: { captured?: { value: unknown } | undefined }, +): SubagentResult { const own = child.session.events.slice(seedLength) const lastMessage = own.findLast((e): e is SessionEvent<'assistant/message'> => e.type === 'assistant/message') const lastEnd = own.findLast((e): e is SessionEvent<'turn/end'> => e.type === 'turn/end') const output: ContentBlock[] = lastMessage ? structuredClone(lastMessage.data.content) : [] - if (lastEnd === undefined && cancelled) return { output, stopReason: 'aborted' } - return { output, stopReason: toStopReason(lastEnd?.data.reason) } + const stopReason: SubagentStopReason = lastEnd === undefined && cancelled + ? 'aborted' + : toStopReason(lastEnd?.data.reason) + if (structured) { + if (structured.captured) return { output, structured: structured.captured.value, stopReason } + // No capture on a cleanly-completed turn: an ERROR when the run was left + // to finish (the nudges ran out), but ABORTED when a cancel is why the + // nudging stopped — the cancel contract outranks the schema shortfall. + if (stopReason === 'completed') return { output, stopReason: cancelled ? 'aborted' : 'error' } + } + return { output, stopReason } } diff --git a/packages/subagent/subagent-inprocess/src/structured.ts b/packages/subagent/subagent-inprocess/src/structured.ts new file mode 100644 index 0000000000..a557e44785 --- /dev/null +++ b/packages/subagent/subagent-inprocess/src/structured.ts @@ -0,0 +1,312 @@ +/** + * Structured-output support for the in-process subagent backends: the mechanism + * behind `SubagentStartRequest.outputSchema` for children that run as agents on + * the same context. + * + * The model-facing surface is one globally registered `structured_output` tool + * whose REGISTERED parameters are a placeholder — the real schema is per run. + * Because the tool registry and prompt assembly are context-global while + * schemas differ per child (two concurrent structured runs may carry different + * schemas), per-agent shaping happens on the `system-prompt/assemble` + * waterfall with a `prepend: true` listener that post-processes `await next()` + * — FINAL-ASSEMBLY enforcement: whatever downstream listeners mutated or + * replaced, the assembly the loop renders never carries `structured_output` + * for an agent without a structured run, and for one that has it always + * carries the run's OWN schema plus a trailing + * {@link STRUCTURED_OUTPUT_INSTRUCTION} section (the demand travels with the + * tool). The loop logs what the assembly produced as the request header, so + * the injection is a reconstructable fact of the session log, never a + * wire-only mutation (the reconstructability RFC). + * (Cooperative mutate-then-`next()` would not survive a downstream listener + * returning a replacement assembly — see the waterfall composition caveat in + * docs/architecture.md.) + * + * FIXME: the whole enforcement dance above exists because the tool registry + * and prompt assembly are context-global. If they become per-agent or + * per-session scoped, a structured run just registers its own schema'd tool on + * the child's scope and this module reduces to the capture tool plus the + * turn-stop — no placeholder, no final-assembly swap, no strip-for-everyone- + * else, no global-registration lifetime dance. + * + * A companion `agent/turn-continuation` listener stops a child's turn once its + * output is captured — without it, the loop's default "had tool calls ⇒ + * continue" buys a wasted extra model step per structured child. It is also + * `prepend: true`: the veto must run before any earlier-registered listener + * that could short-circuit the chain into a forced continue. A third listener + * closes the within-step window the continuation veto cannot: a + * `tools/pre-execute` deny for any call arriving after the agent's capture, so + * a response that lists `structured_output` before further tool calls cannot + * run side effects after the final answer was accepted. A fourth, + * `tools/post-execute`, is the capture COMMIT: the tool body only stages the + * validated value, and it becomes the run's captured result only when the + * final post-execute decision accepts the call — a blocking hook downstream + * yields `isError` in the log, and the run must not report success for it. + * + * Lifetime is refcounted by structured RUNS: each acquires from start to + * settle, so the registrations exist exactly while at least one structured + * child is live — a plain deployment that never passes `outputSchema` carries + * no always-on global state, and a backend hot-reload mid-run cannot + * unregister the capture tool out from under a live child (the run holds its + * own acquisition). Registrations land on the ROOT context and the refcount + * disposes them when the last run settles; the next structured run + * re-registers them. + * + * @module @deepseek-ai/dsh-subagent-inprocess/structured + */ + +import type { Context } from 'cordis' +import type { Agent } from '@deepseek-ai/dsh-agent' +import type { ContentBlock, ToolSchema } from '@deepseek-ai/dsh-llm' +import type { ContinuationDecision } from '@deepseek-ai/dsh-agent' +import type { AssembleContext, PromptAssembly } from '@deepseek-ai/dsh-system-prompt' +import type { PostToolDecision, PreToolDecision, ToolExecution, ToolExecutionResult } from '@deepseek-ai/dsh-tools' +import { ToolArgsError, validateStructuredValue, type StructuredOutputSchema } from '@deepseek-ai/dsh-tools' + +/** The model-facing tool name a structured child must call to finish. */ +export const STRUCTURED_OUTPUT_TOOL = 'structured_output' + +/** + * The instruction the assembly listener appends to a structured child's + * system prompt as a trailing section on every assembly. Per-assembly state, + * NOT agent prompt state: `AgentOptions` has no prompt field (the persona is + * deployment config on the system-prompt plugin), so the same final-assembly + * enforcement that injects the schema'd tool carries the instruction that + * demands calling it. + */ +export const STRUCTURED_OUTPUT_INSTRUCTION + = 'When you have your final answer, you MUST report it by calling the ' + + `\`${STRUCTURED_OUTPUT_TOOL}\` tool with arguments matching its parameter schema exactly. ` + + 'Do not finish with a plain text answer: only the tool call counts as your result.' + +/** One structured run's state: the schema to enforce and the captured value, once recorded. */ +interface RunState { + readonly schema: StructuredOutputSchema + /** + * A validated value awaiting the post-execute verdict on ITS OWN call. Set + * by the capture tool's body, promoted to {@link RunState.captured} only + * when the final `tools/post-execute` decision accepts the call — a + * downstream block turns the logged result into `isError`, and a value + * committed at body time would let the run report success for a call the + * model saw fail. + */ + pending?: { value: unknown } + captured?: { value: unknown } +} + +/** The per-root-context runtime: run states plus the shared registrations. */ +interface StructuredRuntime { + refs: number + readonly states: WeakMap + readonly disposers: (() => void)[] +} + +/** One root context ⇒ one runtime (multi-app test isolation). */ +const runtimes = new WeakMap() + +/** + * One holder's handle on the shared structured runtime. `release()` is + * idempotent per acquisition; the runtime's registrations are disposed when the + * LAST holder (backend plugin or live run) releases. + */ +export interface StructuredAcquisition { + /** Enforce `schema` on `agent`'s requests and start capturing its `structured_output` call. */ + attach(agent: Agent, schema: StructuredOutputSchema): void + /** The captured value, once the child called the tool with valid arguments. */ + captured(agent: Agent): { value: unknown } | undefined + /** Stop enforcing/capturing for `agent` (WeakMap-backed; safe to call twice). */ + detach(agent: Agent): void + /** Drop this holder's reference (idempotent); the last release unregisters everything. */ + release(): void +} + +/** + * Acquire the per-root-context structured runtime, registering the capture tool + * and the runtime's listeners on the FIRST acquisition. See the module doc + * for the enforcement and lifetime design. + * @param ctx - any context of the app; the runtime keys off `ctx.root`. + * @returns this holder's handle (attach/captured/detach + idempotent release). + */ +export function acquireStructuredRuntime(ctx: Context): StructuredAcquisition { + const root: Context = ctx.root + let runtime = runtimes.get(root) + if (!runtime) { + runtime = { refs: 0, states: new WeakMap(), disposers: [] } + runtimes.set(root, runtime) + registerRuntime(root, runtime) + } + runtime.refs += 1 + + let released = false + return { + attach(agent: Agent, schema: StructuredOutputSchema): void { + runtime.states.set(agent, { schema }) + }, + captured(agent: Agent): { value: unknown } | undefined { + return runtime.states.get(agent)?.captured + }, + detach(agent: Agent): void { + runtime.states.delete(agent) + }, + release(): void { + if (released) return + released = true + runtime.refs -= 1 + if (runtime.refs > 0) return + runtimes.delete(root) + for (const dispose of runtime.disposers.splice(0)) dispose() + }, + } +} + +/** Register the capture tool + the two listeners on the root context (first acquire). */ +function registerRuntime(root: Context, runtime: StructuredRuntime): void { + // The registered parameters are a PLACEHOLDER: the request listener below + // swaps in the run's real schema per child, and strips the tool entirely for + // every agent without a structured run — so this shape is never model-visible. + // + // Registration does NOT ride on the acquiring backend's plugin-level + // `inject`: a backend that waited on `tools` would apply later than it did + // before this module existed, shifting when its PROVIDER registers — and the + // delegation tool mirrors provider lifecycle, so that shift would reorder + // the model-visible tool list of every existing prompt. Instead the capture + // tool registers synchronously when `tools` is already live (the common + // case), and through a scoped inject fiber when the Loader happens to start + // the backend first. Either way the registration lands on root and is + // disposed by the runtime's refcount; disposing the fiber also covers the + // never-activated case. + let disposeTool: (() => void) | undefined + const registerCapture = (tools: Context['tools']): void => { + disposeTool = tools.register({ + name: STRUCTURED_OUTPUT_TOOL, + description: + 'Report your final structured result. Call this exactly once, when your answer is complete; ' + + 'the arguments must match this tool\'s parameter schema exactly.', + parameters: { type: 'object', properties: {} }, + execute(args: unknown, exec: ToolExecution): Promise { + const state = exec.agent ? runtime.states.get(exec.agent) : undefined + if (!state) { + // Reachable only if a non-structured agent somehow calls the tool (the + // request listener strips it, so the model never sees it) — fail loud + // rather than capture into nowhere. + throw new Error(`${STRUCTURED_OUTPUT_TOOL} is only available to subagents started with an output schema`) + } + const violations = validateStructuredValue(state.schema, args) + // ToolArgsError → isError result with INVALID_ARGS: the model retries + // within the same turn, exactly like a schema-validated defineTool call. + if (violations.length > 0) throw new ToolArgsError(violations) + // Two-phase commit: the body only STAGES the value; the post-execute + // listener below promotes it once the final decision accepts the call. + state.pending = { value: args } + return Promise.resolve([{ type: 'text', text: 'Structured output recorded.' }]) + }, + }) + } + const liveTools = root.get('tools') + const toolsFiber = liveTools ? undefined : root.inject(['tools'], (childCtx: Context) => { + registerCapture(childCtx.root.tools) + }) + if (liveTools) registerCapture(liveTools) + runtime.disposers.push(() => { + disposeTool?.() + void toolsFiber?.dispose() + }) + + // FINAL-ASSEMBLY enforcement (prepend: true = first registered = OUTERMOST + // wrapper): post-process whatever the downstream listeners and the registry + // produced, so a downstream listener returning a replacement assembly cannot + // leak the tool to other agents or erase the child's schema. The loop logs + // the rendered assembly as the step's request header, so the swap is + // reconstructable log state, never a wire-only mutation. + runtime.disposers.push(root.on('system-prompt/assemble', async function ( + this: unknown, _assembly: PromptAssembly, context: AssembleContext, next: () => Promise, + ): Promise { + const final = await next() + const state = context.agent ? runtime.states.get(context.agent) : undefined + if (state) { + const schemaEntry: ToolSchema = { + name: STRUCTURED_OUTPUT_TOOL, + description: + 'Report your final structured result. Call this exactly once, when your answer is complete; ' + + 'the arguments must match this tool\'s parameter schema exactly.', + // ToolSchema.parameters is the wire-level JSON Schema object; the + // asserted subset type is structurally exactly that. + parameters: state.schema as unknown as Record, + } + final.tools = [...final.tools.filter(tool => tool.name !== STRUCTURED_OUTPUT_TOOL), schemaEntry] + // The demand travels WITH the tool: a trailing section in the + // tool-guidance order band, appended after next() so it renders last + // (renderPrompt joins in array order). + final.sections = [...final.sections, { name: `tool:${STRUCTURED_OUTPUT_TOOL}`, order: 190, text: STRUCTURED_OUTPUT_INSTRUCTION }] + return final + } + // No structured run: strip the placeholder so it is never model-visible. + // An empty tools array canonicalizes to an absent header/wire field + // (canonicalHeader pins empty ≡ absent), so no re-shaping is needed here. + final.tools = final.tools.filter(tool => tool.name !== STRUCTURED_OUTPUT_TOOL) + return final + }, { prepend: true })) + + // Stop a structured child's turn once its output is captured: the default + // "had tool calls ⇒ continue" would otherwise buy a wasted extra model step + // after every successful capture. `prepend: true` puts the veto OUTERMOST — + // an earlier-registered listener that short-circuits the chain (a goal-style + // force-continue returning without `next()`) would otherwise decide the turn + // before this listener ever ran, and no downstream decision may resurrect a + // structured turn that is already finished. + runtime.disposers.push(root.on('agent/turn-continuation', function ( + this: unknown, agent: Agent, _turn: number, _decision: ContinuationDecision, next: () => Promise, + ): Promise { + if (runtime.states.get(agent)?.captured) return Promise.resolve({ action: 'stop' }) + return next() + }, { prepend: true })) + + // The capture COMMIT: promote the staged value only when the final + // post-execute decision accepts the call. The capture tool's body cannot + // decide — `tools/post-execute` runs after it, and a blocking listener (a + // PostToolUse hook) turns the logged result into `isError` feedback; a value + // committed at body time would make readResult report `structured` success + // for a call whose result the model and session log saw fail. `prepend: + // true` = outermost at registration time, so `await next()` returns the + // COMPOSED downstream decision — the same final verdict the registry maps + // onto the result. (A later-registered outer listener that blocks without + // delegating skips this commit entirely: the staged value is dropped and the + // run errors — failure-safe in the same direction.) The staging slot clears + // on every path, including a rejecting downstream listener. + runtime.disposers.push(root.on('tools/post-execute', async function ( + this: unknown, exec: ToolExecution, _result: ToolExecutionResult, next: () => Promise, + ): Promise { + const state = exec.agent ? runtime.states.get(exec.agent) : undefined + if (!state || exec.name !== STRUCTURED_OUTPUT_TOOL || state.pending === undefined) return next() + const pending = state.pending + try { + const decision = await next() + if (decision.kind === 'accept') state.captured = pending + return decision + } finally { + delete state.pending + } + }, { prepend: true })) + + // Terminal means terminal WITHIN the step, not only at its end: the + // turn-continuation veto above runs after every call in the current model + // response has executed, so a response that puts `structured_output` before + // further tool calls would still perform those side effects after the final + // answer was accepted. Deny every later call for a captured agent at the + // allow/deny gate — dispatch is skipped and the model sees an `isError` + // result naming the contract. Calls that PRECEDE the capture in the same + // response ran before `captured` was set and are untouched; a second + // `structured_output` is denied like any other call. `prepend: true` for the + // same reason as the continuation veto: no earlier-registered allow may + // short-circuit past the terminal contract. + runtime.disposers.push(root.on('tools/pre-execute', function ( + this: unknown, exec: ToolExecution, next: () => Promise, + ): Promise { + if (exec.agent && runtime.states.get(exec.agent)?.captured) { + return Promise.resolve({ + kind: 'deny', + reason: `structured output already recorded: the run is complete, so \`${exec.name}\` is not executed`, + }) + } + return next() + }, { prepend: true })) +} diff --git a/packages/subagent/subagent-inprocess/tests/structured.spec.ts b/packages/subagent/subagent-inprocess/tests/structured.spec.ts new file mode 100644 index 0000000000..0de036a0a0 --- /dev/null +++ b/packages/subagent/subagent-inprocess/tests/structured.spec.ts @@ -0,0 +1,606 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import LlmService, { CallId, type ContentBlock, type GenerateOptions } from '@deepseek-ai/dsh-llm' +import SessionStore from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import type { Agent, ContinuationDecision } from '@deepseek-ai/dsh-agent' +import AgentLoop from '@deepseek-ai/dsh-agent-loop' +import * as Invariants from '@deepseek-ai/dsh-invariants' +import SubagentService, { type SubagentStartRequest } from '@deepseek-ai/dsh-subagent' +import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools' +import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' +import { startInProcessRun } from '../src/index.ts' +import { + acquireStructuredRuntime, + STRUCTURED_OUTPUT_INSTRUCTION, + STRUCTURED_OUTPUT_TOOL, +} from '../src/structured.ts' + +type Script = ConstructorParameters[0] + +const SCHEMA: StructuredOutputSchema = { + type: 'object', + properties: { answer: { type: 'number' }, note: { type: 'string' } }, + required: ['answer'], +} + +/** + * Real loop + scripted mock model + an INLINE spawn-shaped provider over the + * shared driver. The concrete backend plugins are deliberately NOT loaded — + * they would devDep-cycle this package (spawn/fork already depend on the + * driver), and the runtime under test is the driver's; plugin-level structured + * coverage lives in the spawn/fork specs. The mock model script drives the + * child's structured_output calls. + */ +async function setup(script: Script) { + const ctx = new Context() + const adapter = new MockAdapter(script) + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(Invariants) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(SubagentService) + const disposeProvider = ctx.subagents.registerProvider({ + name: 'spawn', + capabilities: { outputSchema: true, depthLimit: true, toolFilter: false }, + inheritsParentContext: false, + start: (request: SubagentStartRequest) => startInProcessRun(ctx, request, { providerName: 'spawn' }), + }) + ctx.llm.registerAdapter(['mock'], adapter) + const parent = ctx.agentLoop.create(AgentId('parent'), { model: 'mock' }) + return { ctx, parent, adapter, disposeProvider } +} + +function structuredRequest(parent: SubagentStartRequest['parent'], extra?: Partial): SubagentStartRequest { + return { prompt: [{ type: 'text', text: 'produce the answer' }], parent, outputSchema: SCHEMA, ...extra } +} + +/** The tool names of one recorded model request. */ +function toolNames(request: GenerateOptions): string[] { + return (request.tools ?? []).map(tool => tool.name) +} + +describe('in-process structured output', () => { + it('captures a valid structured_output call and surfaces result.structured', async () => { + const { ctx, parent } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 42, note: 'done' }), + ]) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.stopReason).toBe('completed') + expect(result.structured).toEqual({ answer: 42, note: 'done' }) + await run.dispose() + }) + + it('stops the turn after a successful capture — no extra model step is spent', async () => { + const { ctx, parent, adapter } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 }), + textResponse('MUST NOT BE CONSUMED'), + ]) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + await run.result + // Default continuation would run a second step after the tool call; the + // structured runtime's turn-continuation veto stops the turn instead. + expect(adapter.requests.length).toBe(1) + await run.dispose() + }) + + it('denies tool calls that FOLLOW the capture in the same response — terminal means terminal', async () => { + // One model response carrying structured_output FIRST and a side-effecting + // call after it: the continuation veto only fires at step end, so without + // the pre-execute deny the trailing call would still run after the final + // answer was accepted. + const response = [ + ...toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 5 }).slice(0, -2), + { type: 'block-start', index: 1, blockType: 'tool-call' }, + { type: 'block-end', index: 1, block: { type: 'tool-call', id: CallId('c2'), name: 'side_effect', arguments: '{}' } }, + { type: 'usage', usage: { inputTokens: 10, outputTokens: 5 } }, + { type: 'finish', reason: { kind: 'tool-calls' } }, + ] as Script[number] + const { ctx, parent } = await setup([response]) + let sideEffectRan = false + ctx.tools.register({ + name: 'side_effect', + description: 'probe', + parameters: { type: 'object', properties: {} }, + execute(): Promise { + sideEffectRan = true + return Promise.resolve([{ type: 'text', text: 'ran' }]) + }, + }) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.stopReason).toBe('completed') + expect(result.structured).toEqual({ answer: 5 }) + // The deny skipped dispatch entirely: the probe body never ran. + expect(sideEffectRan).toBe(false) + await run.dispose() + }) + + it('leaves tool calls that PRECEDE the capture in the same response untouched', async () => { + const response = [ + { type: 'block-start', index: 0, blockType: 'tool-call' }, + { type: 'block-end', index: 0, block: { type: 'tool-call', id: CallId('c1'), name: 'side_effect', arguments: '{}' } }, + ...toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, { answer: 6 }).map(chunk => + 'index' in chunk ? { ...chunk, index: 1 } : chunk), + ] as Script[number] + const { ctx, parent } = await setup([response]) + let sideEffectRan = false + ctx.tools.register({ + name: 'side_effect', + description: 'probe', + parameters: { type: 'object', properties: {} }, + execute(): Promise { + sideEffectRan = true + return Promise.resolve([{ type: 'text', text: 'ran' }]) + }, + }) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + // The call ran BEFORE captured was set: the deny gate only guards the + // window after the terminal answer landed. + expect(sideEffectRan).toBe(true) + expect(result.structured).toEqual({ answer: 6 }) + await run.dispose() + }) + + it('snapshots the schema at start(): caller mutation after start cannot drift enforcement', async () => { + const mutable: StructuredOutputSchema = { + type: 'object', + properties: { answer: { type: 'number' } }, + required: ['answer'], + additionalProperties: false, + } + const pristine = structuredClone(mutable) + const { ctx, parent, adapter } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 3 }), + ]) + const run = ctx.subagents.start('spawn', structuredRequest(parent, { outputSchema: mutable })) + // Mutate the caller's object AFTER start() returned but before the child's + // first request assembles: with a live reference this would reach both the + // model-visible parameters and validateStructuredValue. + ;(mutable.properties as Record).answer = { type: 'string' } + const result = await run.result + expect(result.structured).toEqual({ answer: 3 }) + // The child's request carried the PRISTINE schema, not the mutated one. + const childRequest = adapter.requests.at(-1) + const captureTool = (childRequest?.tools ?? []).find(tool => tool.name === STRUCTURED_OUTPUT_TOOL) + expect(captureTool?.parameters).toEqual(pristine) + await run.dispose() + }) + + it('the captured-turn veto is prepend: an EARLIER force-continue listener cannot short-circuit it', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + // Registered BEFORE the structured runtime exists — without prepend, this + // goal-style listener would decide the turn first (returning WITHOUT + // calling next()) and the veto would never run. + ctx.on('agent/turn-continuation', () => Promise.resolve({ action: 'continue' })) + const acquisition = acquireStructuredRuntime(ctx) + const agent = { id: AgentId('structured-child') } as unknown as Agent + acquisition.attach(agent, SCHEMA) + const captured = await ctx.tools.execute({ + callId: 'call-1' as never, + name: STRUCTURED_OUTPUT_TOOL, + arguments: { answer: 1 }, + agent, + }) + expect(captured.isError).toBeFalsy() + const decision = await ctx.waterfall( + 'agent/turn-continuation', agent, 1, + { action: 'continue' }, + () => Promise.resolve({ action: 'continue' }), + ) + expect(decision).toEqual({ action: 'stop' }) + acquisition.detach(agent) + acquisition.release() + }) + + it('an invalid call gets an INVALID_ARGS isError result and the model retries in-turn', async () => { + const { ctx, parent } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 'not-a-number' }), + toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, { answer: 7 }), + ]) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.structured).toEqual({ answer: 7 }) + expect(result.stopReason).toBe('completed') + // The child's log carries the isError tool/result for the invalid call. + const child = ctx.agents.get(run.id)! + const results = child.session.events.filter(e => e.type === 'tool/result') + expect(results.length).toBe(2) + expect((results[0]!.data as { isError?: boolean }).isError).toBe(true) + await run.dispose() + }) + + it('a clean finish without a capture is an immediate error to the parent — deliberately NO re-prompt', async () => { + const { ctx, parent, adapter } = await setup([ + textResponse('here is my answer in prose'), + textResponse('MUST NOT BE CONSUMED'), + ]) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.stopReason).toBe('error') + expect(result.structured).toBeUndefined() + // Exactly one model request and one user message: no nudge turn exists. + expect(adapter.requests.length).toBe(1) + const child = ctx.agents.get(run.id)! + expect(child.session.events.filter(e => e.type === 'user/message').length).toBe(1) + await run.dispose() + }) + + it('an errored child keeps its honest error result (no capture expected)', async () => { + // Script exhaustion on the first call → the child turn errors. + const { ctx, parent, adapter } = await setup([]) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.stopReason).toBe('error') + expect(adapter.requests.length).toBe(1) + await run.dispose() + }) + + it('a cancel landing after a clean capture-less turn settles aborted, not error', async () => { + const { ctx, parent } = await setup([textResponse('prose, no capture')]) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const child = ctx.agents.get(run.id)! + // Cancel synchronously inside the turn's end recording: the cancel + // contract outranks the schema shortfall, so the result maps to aborted. + ctx.on('session/event', (session, event) => { + if (session === child.session && event.type === 'turn/end') run.cancel('cancelled at turn end') + }) + const result = await run.result + expect(result.stopReason).toBe('aborted') + await run.dispose() + }) + + it('rejects a schema outside the subset loud, before any child exists', async () => { + const { ctx, parent } = await setup([]) + expect(() => ctx.subagents.start('spawn', structuredRequest(parent, { + outputSchema: { type: 'object', oneOf: [] } as unknown as StructuredOutputSchema, + }))).toThrow(/unsupported output schema/) + expect(ctx.agents.get(AgentId('parent'))).toBeDefined() + }) + + it('a schema carrying non-JSON values fails as OutputSchemaError, never as a raw clone error', async () => { + const { ctx, parent } = await setup([]) + // Assertion runs BEFORE the defensive structuredClone: a function-valued + // annotation must surface as the subset violation it is, not escape as + // structuredClone's DataCloneError. + expect(() => ctx.subagents.start('spawn', structuredRequest(parent, { + outputSchema: { type: 'object', default: () => {} } as unknown as StructuredOutputSchema, + }))).toThrow(/unsupported output schema.*annotation must be JSON data/) + }) + + it('a post-execute BLOCK on the capture call denies the capture: log and result agree on failure', async () => { + const { ctx, parent, adapter } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 7 }), + textResponse('continues after the blocked capture'), + ]) + // A PostToolUse-style hook, registered AFTER the runtime (so the runtime's + // prepend commit listener stays outermost and composes this verdict). + ctx.on('tools/post-execute', (exec, _result, next) => { + if (exec.name === STRUCTURED_OUTPUT_TOOL) { + return Promise.resolve({ kind: 'block' as const, feedback: [{ type: 'text' as const, text: 'capture rejected by hook' }] }) + } + return next() + }) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + // No capture was committed: the run reports the schema shortfall... + expect(result.structured).toBeUndefined() + expect(result.stopReason).toBe('error') + // ...the logged tool result is the blocked isError with the feedback... + const child = ctx.agents.get(run.id)! + const results = child.session.events.filter(e => e.type === 'tool/result') + expect((results[0]!.data as { isError?: boolean }).isError).toBe(true) + expect(JSON.stringify((results[0]!.data as { content: unknown }).content)).toContain('capture rejected by hook') + // ...and the turn CONTINUED past the blocked call (no captured veto): + // the model got to react to the failure with a second step. + expect(adapter.requests.length).toBe(2) + await run.dispose() + }) + + it('a post-execute accept-with-replacement still commits the capture', async () => { + const { ctx, parent } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 8 }), + ]) + ctx.on('tools/post-execute', (exec, _result, next) => { + if (exec.name === STRUCTURED_OUTPUT_TOOL) { + return Promise.resolve({ kind: 'accept' as const, content: [{ type: 'text' as const, text: 'recorded (rewritten)' }] }) + } + return next() + }) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.stopReason).toBe('completed') + expect(result.structured).toEqual({ answer: 8 }) + await run.dispose() + }) + + it('appends the structured instruction to the child REQUEST\'s system text (base prompt preserved)', async () => { + const { ctx, parent, adapter } = await setup([toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 })]) + // A context-wide section stands in for the deployment persona: the + // instruction must APPEND to whatever the prompt pipeline assembled, not + // replace it (AgentOptions has no prompt field — the instruction is + // per-request wire state added by the final-request listener). + ctx.systemPrompt.section({ name: 'test:persona', order: 10, text: 'You are a counter.' }) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + await run.result + const childRequest = adapter.requests.at(-1)! + expect(childRequest.system).toContain('You are a counter.') + expect(childRequest.system!.endsWith(STRUCTURED_OUTPUT_INSTRUCTION)).toBe(true) + expect(childRequest.system!.indexOf(STRUCTURED_OUTPUT_INSTRUCTION)).toBeGreaterThan(0) + await run.dispose() + }) + + it('the instruction rides ONLY structured requests: appended for the child, absent for a plain agent', async () => { + const { ctx, parent, adapter } = await setup([ + textResponse('parent answer'), + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 }), + ]) + parent.send([{ type: 'text', text: 'hello' }]) + await parent.whenIdle() + expect(adapter.requests[0]!.system ?? '').not.toContain(STRUCTURED_OUTPUT_INSTRUCTION) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + await run.result + // The loop always assembles a base prompt (the harness identity section), + // so the instruction APPENDS — never replaces. + const childSystem = adapter.requests.at(-1)!.system! + expect(childSystem.endsWith(STRUCTURED_OUTPUT_INSTRUCTION)).toBe(true) + expect(childSystem.length).toBeGreaterThan(STRUCTURED_OUTPUT_INSTRUCTION.length) + await run.dispose() + }) + + describe('final-request enforcement (the prepend agent/request listener)', () => { + it('a plain agent assembling while the runtime is LIVE gets the placeholder stripped', async () => { + // Run-scoped acquisition means a plain deployment never registers the + // tool at all; the strip branch exists for the CONCURRENT case — a plain + // agent taking a turn while some structured child holds the runtime open. + const { ctx, parent, adapter } = await setup([textResponse('parent answer')]) + const hold = acquireStructuredRuntime(ctx) + parent.send([{ type: 'text', text: 'hello' }]) + await parent.whenIdle() + // The placeholder IS in the registry during this turn; the assembly the + // loop rendered must not carry it for an agent without a structured run. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeDefined() + expect(toolNames(adapter.requests[0]!)).not.toContain(STRUCTURED_OUTPUT_TOOL) + hold.release() + }) + + it('a structured child sees structured_output with ITS schema; a plain agent never sees the tool', async () => { + const { ctx, parent, adapter } = await setup([ + // Parent turn (a plain agent): must NOT see the tool. + textResponse('parent answer'), + // Child turn: must see it, with the run's schema. + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 42 }), + ]) + parent.send([{ type: 'text', text: 'hello' }]) + await parent.whenIdle() + expect(toolNames(adapter.requests[0]!)).not.toContain(STRUCTURED_OUTPUT_TOOL) + + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + await run.result + const childRequest = adapter.requests[1]! + expect(toolNames(childRequest)).toContain(STRUCTURED_OUTPUT_TOOL) + const entry = childRequest.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)! + expect(entry.parameters).toEqual(SCHEMA) + await run.dispose() + }) + + it('two concurrent structured children each see their OWN schema', async () => { + const otherSchema: StructuredOutputSchema = { + type: 'object', + properties: { verdict: { type: 'string', enum: ['real', 'bogus'] } }, + required: ['verdict'], + } + const { ctx, parent, adapter } = await setup([ + (options: GenerateOptions) => { + // Answer with whatever schema this child was given — proves each + // request carried the right one regardless of scheduling order. + const entry = options.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)! + const args = 'verdict' in (entry.parameters.properties as Record) + ? { verdict: 'real' } + : { answer: 1 } + return toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, args) + }, + (options: GenerateOptions) => { + const entry = options.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)! + const args = 'verdict' in (entry.parameters.properties as Record) + ? { verdict: 'real' } + : { answer: 1 } + return toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, args) + }, + ]) + const runA = ctx.subagents.start('spawn', structuredRequest(parent)) + const runB = ctx.subagents.start('spawn', structuredRequest(parent, { outputSchema: otherSchema })) + const [a, b] = await Promise.all([runA.result, runB.result]) + expect(a.structured).toEqual({ answer: 1 }) + expect(b.structured).toEqual({ verdict: 'real' }) + const schemas = adapter.requests.map(request => + request.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!.parameters) + expect(schemas).toContainEqual(SCHEMA) + expect(schemas).toContainEqual(otherSchema) + await runA.dispose() + await runB.dispose() + }) + + it('wins against a downstream listener that REPLACES the assembly object', async () => { + const { ctx, parent, adapter } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 5 }), + ]) + // A downstream (non-prepend) listener that returns a brand-new assembly — + // the composition caveat that erases cooperative mutations. Registered + // AFTER the runtime's prepend listener, so it runs INSIDE it. + ctx.on('system-prompt/assemble', async (_assembly, _context, next) => { + const replaced = await next() + return { sections: [...replaced.sections], tools: [...replaced.tools], variables: { ...replaced.variables } } + }) + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + expect(result.structured).toEqual({ answer: 5 }) + const entry = adapter.requests[0]!.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL) + expect(entry).toBeDefined() + expect(entry!.parameters).toEqual(SCHEMA) + await run.dispose() + }) + + it('a non-structured agent request keeps tools ABSENT when it had none (no tools: [] materialized)', async () => { + const { parent, adapter } = await setup([ + // The registry contributes the placeholder via prompt assembly, so + // tools is an array in the raw request — but after stripping the + // placeholder (its ONLY entry), the field must not be re-added as a + // different shape. + textResponse('plain'), + ]) + parent.send([{ type: 'text', text: 'q' }]) + await parent.whenIdle() + const request = adapter.requests[0]! + expect(toolNames(request)).not.toContain(STRUCTURED_OUTPUT_TOOL) + await new Promise(resolve => setTimeout(resolve, 0)) + }) + + it('shapes a bare assembly on the waterfall: no-agent context strips the placeholder; a structured agent gains schema + trailing instruction section', async () => { + // Drive ctx.systemPrompt.assemble directly — the enforcement listener + // must tolerate a context with NO agent (a bare diagnostic assemble) + // and shape a structured agent's assembly on the same path the loop + // renders and logs as the request header. + const { ctx, parent } = await setup([]) + const acquisition = acquireStructuredRuntime(ctx) + // Bare assemble WHILE the runtime is live: the no-agent branch must + // strip the registered placeholder (before the acquisition there is + // nothing to strip — run-scoped registration). + const bare = await ctx.systemPrompt.assemble({}) + expect(bare.tools.map(tool => tool.name)).not.toContain(STRUCTURED_OUTPUT_TOOL) + + acquisition.attach(parent, SCHEMA) + const shaped = await ctx.systemPrompt.assemble({ agent: parent }) + expect(shaped.tools.map(tool => tool.name)).toContain(STRUCTURED_OUTPUT_TOOL) + expect(shaped.tools.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!.parameters).toEqual(SCHEMA) + // The demand travels with the tool: the instruction renders LAST + // (appended post-next(); renderPrompt joins in array order). + expect(shaped.sections.at(-1)).toMatchObject({ name: `tool:${STRUCTURED_OUTPUT_TOOL}`, text: STRUCTURED_OUTPUT_INSTRUCTION }) + acquisition.detach(parent) + acquisition.release() + }) + }) + + describe('runtime lifetime (refcount: live structured runs)', () => { + it('the runtime exists exactly while structured runs are live: nothing before, nothing after', async () => { + const { ctx, parent } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 4 }), + ]) + // No always-on global state: a context that has run no structured child + // carries no capture tool. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + const run = ctx.subagents.start('spawn', structuredRequest(parent)) + const result = await run.result + // The capture succeeded — the registrations existed while the run lived. + expect(result.structured).toEqual({ answer: 4 }) + // The run's settle released the last acquisition. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + await run.dispose() + }) + + it('concurrent structured runs share one runtime; the last settle disposes it', async () => { + const { ctx, parent } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 }), + toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, { answer: 2 }), + ]) + const first = ctx.subagents.start('spawn', structuredRequest(parent)) + const second = ctx.subagents.start('spawn', structuredRequest(parent)) + const [a, b] = await Promise.all([first.result, second.result]) + expect([a.structured, b.structured].sort()).toEqual([{ answer: 1 }, { answer: 2 }].sort()) + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + await first.dispose() + await second.dispose() + }) + + it('acquisition release is idempotent (double release cannot underflow the refcount)', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + const first = acquireStructuredRuntime(ctx) + const second = acquireStructuredRuntime(ctx) + first.release() + first.release() + // The second holder still keeps the tool registered. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeDefined() + second.release() + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + }) + + it('registers the capture tool through the scoped fiber when tools loads after the acquisition', async () => { + // The Loader starts sibling plugins concurrently, so a backend can + // acquire the runtime before dsh-tools has applied. The capture tool + // must then register as soon as `tools` exists — via the inject fiber, + // not by deferring the backend (which would reorder the prompt's tools). + const ctx = new Context() + const acquisition = acquireStructuredRuntime(ctx) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + // Fiber activation completes asynchronously after the service appears. + await new Promise(resolve => setImmediate(resolve)) + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeDefined() + acquisition.release() + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + }) + + it('releasing before tools ever loads disposes the pending fiber without registering', async () => { + const ctx = new Context() + const acquisition = acquireStructuredRuntime(ctx) + acquisition.release() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await new Promise(resolve => setImmediate(resolve)) + // The disposed fiber never fires: nothing registers after the fact. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + }) + + it('attach/captured/detach manage per-agent state through the acquisition surface', async () => { + const { ctx, parent } = await setup([]) + const acquisition = acquireStructuredRuntime(ctx) + expect(acquisition.captured(parent)).toBeUndefined() + acquisition.attach(parent, SCHEMA) + expect(acquisition.captured(parent)).toBeUndefined() + acquisition.detach(parent) + acquisition.detach(parent) + acquisition.release() + // That manual acquisition was the ONLY holder - release disposes. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + }) + }) + + it('a direct structured_output call from an agent WITHOUT a structured run is an isError', async () => { + const { ctx, parent } = await setup([]) + // Hold the runtime open (run-scoped: nothing is registered otherwise) so + // the call reaches the capture tool's own fail-loud guard, not UNKNOWN_TOOL. + const hold = acquireStructuredRuntime(ctx) + const result = await ctx.tools.execute({ + callId: 'x' as never, + name: STRUCTURED_OUTPUT_TOOL, + arguments: { answer: 1 }, + agent: parent, + }) + expect(result.isError).toBe(true) + expect(JSON.stringify(result.content)).toContain('only available to subagents') + hold.release() + }) + + it('a structured_output call with NO calling agent at all is an isError', async () => { + const { ctx } = await setup([]) + const hold = acquireStructuredRuntime(ctx) + const result = await ctx.tools.execute({ + callId: 'x' as never, + name: STRUCTURED_OUTPUT_TOOL, + arguments: { answer: 1 }, + }) + expect(result.isError).toBe(true) + hold.release() + }) +}) diff --git a/packages/subagent/subagent-inprocess/tsconfig.json b/packages/subagent/subagent-inprocess/tsconfig.json index 4cb435d4fb..7b7a015cc9 100644 --- a/packages/subagent/subagent-inprocess/tsconfig.json +++ b/packages/subagent/subagent-inprocess/tsconfig.json @@ -25,6 +25,12 @@ }, { "path": "../subagent" + }, + { + "path": "../../core/system-prompt" + }, + { + "path": "../../core/tools" } ] } diff --git a/packages/subagent/subagent-spawn/README.md b/packages/subagent/subagent-spawn/README.md index 97dfae9304..b976ef5a63 100644 --- a/packages/subagent/subagent-spawn/README.md +++ b/packages/subagent/subagent-spawn/README.md @@ -10,7 +10,7 @@ The run mechanics live in the shared [`@deepseek-ai/dsh-subagent-inprocess`](../ ## Capabilities -`{ outputSchema: false, depthLimit: true, toolFilter: false }`. It constructs the child, so it enforces a recursion cap; structured output and tool-scoping are deferred (the service rejects a request needing either before `start` runs). +`{ outputSchema: true, depthLimit: true, toolFilter: false }`. It constructs the child, so it enforces a recursion cap, and it supports structured output via the driver's [structured runtime](../subagent-inprocess/README.md) (acquired per structured run inside the driver — this backend registers nothing at apply). Tool-scoping is deferred (the service rejects a request needing it before `start` runs). ## Config diff --git a/packages/subagent/subagent-spawn/src/index.ts b/packages/subagent/subagent-spawn/src/index.ts index e6cf5039a7..2d8f118b4e 100644 --- a/packages/subagent/subagent-spawn/src/index.ts +++ b/packages/subagent/subagent-spawn/src/index.ts @@ -9,6 +9,11 @@ * ({@link startInProcessRun}); this backend just passes NO seed (a fresh * child). The fork backend is an independent peer over the same driver. * + * Structured output (`outputSchema`) is supported via the driver's shared + * structured runtime: the backend acquires it for its plugin lifetime (so the + * capture tool and request-shaping listeners exist before any run), and each + * structured run holds its own acquisition until it settles. + * * Plugin export shape: named `name`/`inject`/`Config`/`apply`, NO default. * * @module @deepseek-ai/dsh-subagent-spawn @@ -20,6 +25,11 @@ import type { SubagentCapabilities, SubagentProvider, SubagentStartRequest } fro import { startInProcessRun } from '@deepseek-ai/dsh-subagent-inprocess' export const name = 'subagent-spawn' +// `tools` is deliberately NOT injected: the shared driver's structured runtime +// (acquired per structured RUN, not at apply) gates its own capture-tool +// registration on `tools` availability, so this backend's apply timing — and +// with it the provider-mirroring delegation tool's position in the +// model-visible tool list — stays what it was before structured output existed. export const inject = ['subagents', 'agents'] /** Config: the registry name to register the provider under. */ @@ -34,11 +44,12 @@ export const Config: z = z.object({ /** * The spawn provider. Supports `depthLimit` (it constructs the child, so it can - * enforce a recursion cap) but NOT `outputSchema` or `toolFilter` in this cut — - * a request that needs either is rejected by the service before `start` runs. + * enforce a recursion cap) and `outputSchema` (via the shared in-process + * structured runtime); NOT `toolFilter` in this cut — a request that needs it + * is rejected by the service before `start` runs. */ class SpawnProvider implements SubagentProvider { - readonly capabilities: SubagentCapabilities = { outputSchema: false, depthLimit: true, toolFilter: false } + readonly capabilities: SubagentCapabilities = { outputSchema: true, depthLimit: true, toolFilter: false } // Context contract: a spawned child starts fresh — it never sees the parent conversation. readonly inheritsParentContext = false @@ -46,7 +57,8 @@ class SpawnProvider implements SubagentProvider { start(request: SubagentStartRequest) { // Fresh child: no seed. The shared driver mints ids, stamps cwd/lineage/ - // depth, drives the one-shot, and maps the result. + // depth, drives the one-shot (including the structured capture when the + // request carries an outputSchema), and maps the result. return startInProcessRun(this.ctx, request, { providerName: this.name }) } } diff --git a/packages/subagent/subagent-spawn/tests/subagent-spawn.spec.ts b/packages/subagent/subagent-spawn/tests/subagent-spawn.spec.ts index ccfd6492f4..1dad9748e9 100644 --- a/packages/subagent/subagent-spawn/tests/subagent-spawn.spec.ts +++ b/packages/subagent/subagent-spawn/tests/subagent-spawn.spec.ts @@ -10,9 +10,9 @@ import { SessionId } from '@deepseek-ai/dsh-session' import AgentLoop from '@deepseek-ai/dsh-agent-loop' import * as Invariants from '@deepseek-ai/dsh-invariants' import SubagentService from '@deepseek-ai/dsh-subagent' -import { MockAdapter, maxTokensResponse, textResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' +import { MockAdapter, maxTokensResponse, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' import * as spawn from '../src/index.ts' -import { depthOf, SubagentDepthError } from '@deepseek-ai/dsh-subagent-inprocess' +import { depthOf, STRUCTURED_OUTPUT_TOOL, SubagentDepthError } from '@deepseek-ai/dsh-subagent-inprocess' type Script = ConstructorParameters[0] @@ -241,10 +241,10 @@ describe('dsh-subagent-spawn', () => { await parentHandle.dispose() }) - it('advertises depthLimit but not outputSchema/toolFilter', async () => { + it('advertises depthLimit and outputSchema but not toolFilter', async () => { const { ctx } = await setup([]) const provider = ctx.subagents.getProvider('spawn')! - expect(provider.capabilities).toEqual({ outputSchema: false, depthLimit: true, toolFilter: false }) + expect(provider.capabilities).toEqual({ outputSchema: true, depthLimit: true, toolFilter: false }) }) it('unregisters the provider when its fiber is disposed (HMR safety)', async () => { @@ -257,6 +257,54 @@ describe('dsh-subagent-spawn', () => { expect(ctx.subagents.list()).toEqual([]) }) + it('captures structured output through the shipped plugin (driver runtime, plugin wiring)', async () => { + const { ctx, parent } = await setup([ + toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 42 }), + ]) + const run = ctx.subagents.start('spawn', { + prompt: [{ type: 'text', text: 'produce the answer' }], + parent, + outputSchema: { type: 'object', properties: { answer: { type: 'number' } }, required: ['answer'] }, + }) + const result = await run.result + expect(result.stopReason).toBe('completed') + expect(result.structured).toEqual({ answer: 42 }) + // Run-scoped runtime: the settle released the last acquisition. + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + await run.dispose() + }) + + it('a backend unload mid-structured-run settles the run and releases the runtime', async () => { + // Rebuild the stack by hand so we hold the backend's fiber. + const ctx = new Context() + const adapter = new MockAdapter(['hang']) + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(Invariants) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(SubagentService) + const fiber = await ctx.plugin(spawn, { providerName: 'spawn' }) + ctx.llm.registerAdapter(['mock'], adapter) + const parent = ctx.agentLoop.create(AgentId('parent'), { model: 'mock' }) + const run = ctx.subagents.start('spawn', { + prompt: [{ type: 'text', text: 'q' }], + parent, + outputSchema: { type: 'object', properties: { a: { type: 'number' } } }, + }) + // Let the child's step start streaming, then unload the backend. The + // backend owns the child agent, so the unload tears the child down and + // the run settles — releasing its own runtime acquisition on the way out. + await new Promise(resolve => setTimeout(resolve, 30)) + await fiber.dispose() + const result = await run.result + expect(result.stopReason).toBe('error') + expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined() + await run.dispose() + }) + it('has the namespace-plugin export shape (no stray default)', () => { expect('default' in spawn).toBe(false) expect(spawn.name).toBe('subagent-spawn') diff --git a/packages/subagent/subagent/src/types.ts b/packages/subagent/subagent/src/types.ts index ef76a96e5d..fb512c0bdb 100644 --- a/packages/subagent/subagent/src/types.ts +++ b/packages/subagent/subagent/src/types.ts @@ -8,7 +8,7 @@ import type { Agent, AgentId, AgentOptions } from '@deepseek-ai/dsh-agent' import type { ContentBlock } from '@deepseek-ai/dsh-llm' -import type { SchemaSpec } from '@deepseek-ai/dsh-tools' +import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools' /** * Which START-TIME features a provider supports. Checked by the service @@ -56,12 +56,16 @@ export interface SubagentStartRequest { /** Per-child agent options (model, system prompt). */ agentOptions?: AgentOptions /** - * Optional structured-output schema. When set AND the provider's - * {@link SubagentCapabilities.outputSchema} is `true`, the child's final - * answer is shaped to this schema and surfaced as {@link SubagentResult.structured}. + * Optional structured-output schema — an object-rooted JSON Schema within the + * enforced subset (see `assertSupportedOutputSchema` in dsh-tools; a schema + * outside the subset is rejected loud at start). When set AND the provider's + * {@link SubagentCapabilities.outputSchema} is `true`, the child is driven to + * report a value matching this schema, surfaced as + * {@link SubagentResult.structured}. The schema must be plain host-realm JSON + * data — a caller holding foreign-realm data materializes it first. * Requesting it against a provider that lacks the capability is rejected at start. */ - outputSchema?: SchemaSpec + outputSchema?: StructuredOutputSchema /** * Optional recursion cap (max delegation depth below this child). Requires * {@link SubagentCapabilities.depthLimit}; rejected at start otherwise. @@ -93,6 +97,7 @@ export interface SubagentStopReasonMap { refusal: 'refusal' } +/** The union over {@link SubagentStopReasonMap} — widens automatically as backends merge in variants. */ export type SubagentStopReason = SubagentStopReasonMap[keyof SubagentStopReasonMap] /** diff --git a/packages/subagent/subagent/tests/service.spec.ts b/packages/subagent/subagent/tests/service.spec.ts index e49b4c12e0..f70abf7e72 100644 --- a/packages/subagent/subagent/tests/service.spec.ts +++ b/packages/subagent/subagent/tests/service.spec.ts @@ -178,7 +178,7 @@ describe('SubagentService', () => { describe('start-time capability validation (fail loud, before any child)', () => { it.each([ - { field: 'outputSchema', request: baseRequest({ outputSchema: { x: { type: 'string' } } }) }, + { field: 'outputSchema', request: baseRequest({ outputSchema: { type: 'object', properties: { x: { type: 'string' } } } }) }, { field: 'maxDepth', request: baseRequest({ maxDepth: 2 }) }, { field: 'toolFilter', request: baseRequest({ toolFilter: { deny: ['bash'] } }) }, ])('rejects $field against a provider that lacks the capability — before start() runs', ({ request }) => { @@ -203,7 +203,7 @@ describe('SubagentService', () => { await ctx.plugin(SubagentService) const provider = new StubProvider('strong', ALL_CAPS) ctx.subagents.registerProvider(provider) - ctx.subagents.start('strong', baseRequest({ outputSchema: { x: { type: 'string' } }, maxDepth: 1 })) + ctx.subagents.start('strong', baseRequest({ outputSchema: { type: 'object', properties: { x: { type: 'string' } } }, maxDepth: 1 })) expect(provider.startCount).toBe(1) }) }) diff --git a/packages/support/README.md b/packages/support/README.md index 233a32d77f..2a08063bad 100644 --- a/packages/support/README.md +++ b/packages/support/README.md @@ -4,8 +4,9 @@ Packages that exist to serve development, testing, and the examples rather than | Package | Role | ctx key | |---|---|---| +| `acp-snapshot/` | ACP snapshot suite kit: subprocess scenario harness + golden normalizers + the `defineAcpSnapshotSuite` factory | (library — imported by example `*.snapshot.ts` suites) | | `invariants/` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) | | `llm-replay/` | Record/replay adapter: short-circuits `llm/stream` from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) | | `subagent-mock/` | Scripted `SubagentProvider` for deterministic seam/tool tests | (registers on `ctx.subagents`) | -`invariants` runs only in dev mode (contract checks, not runtime behavior). `llm-replay` backs the demos and the snapshot test tier under the per-file coverage gate. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers. +`invariants` runs only in dev mode (contract checks, not runtime behavior). `llm-replay` backs the demos and the snapshot test tier under the per-file coverage gate. `acp-snapshot` carries the snapshot tier's harness/normalizer/suite machinery so every example's suite is a scenario table over one shared, gate-covered implementation. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers. diff --git a/packages/support/acp-snapshot/README.md b/packages/support/acp-snapshot/README.md new file mode 100644 index 0000000000..8c0b514c07 --- /dev/null +++ b/packages/support/acp-snapshot/README.md @@ -0,0 +1,36 @@ +# `@deepseek-ai/dsh-acp-snapshot` + +The ACP snapshot suite kit: the shared machinery behind the keyless snapshot tier (`pnpm run test:snapshot`, [testing policy](../../../docs/testing.md)). An example gets a full snapshot suite from a scenario table plus a fixtures directory; every compare/guard mechanic lives here, under the per-file coverage gate, instead of being copied per example. + +Three layers, importable separately: + +- **`runScenario` (harness)** — boots the real agent bin as a subprocess via tsx (unbuilt, Loader path), drives it over ACP JSON-RPC stdio from a deterministic `input.json` script, tees raw stdout for the golden + purity check, and harvests every persisted session JSONL (parent + subagent children, primary-first) after a graceful stdin-EOF shutdown. Parameterized by `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath` — absolute paths; the subprocess cwd is a temp dir outside the repo). +- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), and the composable `scrubRequestHeaders` (header bulk → `{{system}}`/`{{tools}}`, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)). +- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record-mode fixture write-back, the per-suite header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin, non-pinning fixtures header-scrubbed). Must be called at vitest collection time. + +A consuming `*.snapshot.ts` is the scenario table plus one factory call: + +```ts +import { dirname, join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { defineAcpSnapshotSuite, type Scenario } from '@deepseek-ai/dsh-acp-snapshot' + +const SCENARIOS: Scenario[] = [ + { name: 'text-turn', hasModelTurn: true, recorded: true, pinsHeader: true }, +] + +defineAcpSnapshotSuite({ + agent: { // absolute paths, resolved from the suite's own location + binScript: fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)), + configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)), + tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)), + }, + snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'), + scenarios: SCENARIOS, // exactly one entry sets pinsHeader + mode: process.env.DSH_SNAPSHOT === 'record' ? 'record' : 'replay', +}) +``` + +The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config RFC](../../../docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. Fixture roles, record/replay semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md). + +Constraints: `suite.ts` imports vitest, so the package is importable only inside a vitest run (the harness and normalizers have no such dependency but ship from the same entry). ACP-specific by design — the harness speaks the SDK's `ClientSideConnection`. Permission round-trips are scriptable: `InputScript.permissionAnswers` is a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) the client maps to the agent-issued `optionId` at answer time; an absent or exhausted queue answers `cancelled`, and a kind the request never offered rejects the run (the agent is answered `cancelled`, so a tolerant agent cannot absorb the scenario bug). diff --git a/packages/support/acp-snapshot/package.json b/packages/support/acp-snapshot/package.json new file mode 100644 index 0000000000..363bc86e25 --- /dev/null +++ b/packages/support/acp-snapshot/package.json @@ -0,0 +1,35 @@ +{ + "name": "@deepseek-ai/dsh-acp-snapshot", + "description": "ACP snapshot suite kit: real-subprocess scenario harness, golden normalizers, and the suite factory behind the keyless snapshot tier", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "dependencies": { + "@agentclientprotocol/sdk": "0.25.1", + "tsx": "^4.22.4", + "vitest": "^4.1.8" + }, + "peerDependencies": { + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "cordis": "^4.0.0-rc.6" + } +} diff --git a/examples/acp-agent/tests/snapshot-harness.ts b/packages/support/acp-snapshot/src/harness.ts similarity index 71% rename from examples/acp-agent/tests/snapshot-harness.ts rename to packages/support/acp-snapshot/src/harness.ts index 8285b870bf..538b81d57e 100644 --- a/examples/acp-agent/tests/snapshot-harness.ts +++ b/packages/support/acp-snapshot/src/harness.ts @@ -1,16 +1,19 @@ /** - * Shared harness for the ACP snapshot tests. A plain module (NOT a *.spec.ts / - * *.snapshot.ts) so importing it never re-registers another file's tests. + * Shared subprocess harness for ACP snapshot suites. A library module driven by + * the suite factory in ./suite.ts (and directly by harness-level specs); each + * example's `*.snapshot.ts` names its own agent-under-test paths. * - * It boots the REAL examples/acp-agent subprocess via the cordis Loader (so the + * It boots the REAL agent bin subprocess via the cordis Loader (so the * export-shape bug class stays guarded — see docs/postmortem/0001), drives it * over real ACP JSON-RPC stdio with a deterministic input script, tees raw * stdout (for the golden + a purity check) into an SDK `ClientSideConnection`, * and — in record mode — harvests the persisted session JSONL after a graceful - * shutdown flush. Two pure normalizers turn the captured stdout frames and the - * session-log events into stable, snapshot-able text. + * shutdown flush. The pure normalizers in ./normalize.ts turn the captured + * stdout frames and the session-log events into stable, snapshot-able text. * * See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md. + * + * @module @deepseek-ai/dsh-acp-snapshot/harness */ import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process' @@ -31,19 +34,36 @@ import { type SessionNotification, } from '@agentclientprotocol/sdk' -// The dsh-acp-agent bin (the demo:acp entry) and this example's cordis.yml. -// The bin resolves its config-path arg from CWD and, under DSH_SNAPSHOT=replay, -// swaps it for the sibling cordis.snapshot.yml. The child's cwd is a temp dir -// OUTSIDE the repo, so pass the example config's ABSOLUTE path. -const binScript = fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)) -const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) +// Resolve tsx's ESM loader to an ABSOLUTE path once: the child runs with its +// cwd in a temp dir OUTSIDE the repo, where a bare `--import tsx` would not +// resolve from node_modules. import.meta.resolve gives this package's tsx +// regardless of the child cwd. const tsxLoader = fileURLToPath(import.meta.resolve('tsx')) -// The repo-root tsconfig: dev/test run UNBUILT and the `@deepseek-ai/dsh-*` -// imports resolve through its `paths` map. The child's cwd is a temp dir -// OUTSIDE the repo, so tsx's upward search would miss it — point tsx at the -// repo tsconfig explicitly (same fix the e2e harness uses). Repo root is four -// levels up from this file (examples/acp-agent/tests). -const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)) + +/** + * The agent composition a scenario runs against: which bin to boot and which + * leaf config it loads. All paths are ABSOLUTE — the subprocess cwd is a temp + * dir outside the repo, so relative resolution would miss; a suite resolves + * them from its own `import.meta.url`. + */ +export interface AgentUnderTest { + /** The agent bin entry (e.g. `packages/ui/acp-agent/src/bin.ts`), run unbuilt via tsx. */ + binScript: string + /** + * The example's live `cordis.yml`. Under `DSH_SNAPSHOT=replay` the bin swaps + * it for the sibling `cordis.snapshot.yml` (the keyless replay overlay), so + * one path serves both modes. + */ + configPath: string + /** + * The repo-root tsconfig whose `paths` map resolves the unbuilt workspace + * imports. Passed to the child as `TSX_TSCONFIG_PATH`: tsx finds a tsconfig + * by searching UP from the child's cwd — a temp dir outside the repo — so + * without the explicit pin the dsh-* imports fail before the bin writes a + * byte. + */ + tsconfigPath: string +} /** * One step of a scenario's deterministic input script (`input.json`). The @@ -57,7 +77,7 @@ const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta * the only way to exercise a cancel deterministically (a plain `prompt` step * awaits the response, which a cancel/hang scenario would block on forever). */ -type InputStep = +export type InputStep = | { op: 'initialize'; terminalOutput?: boolean } | { op: 'newSession' } | { op: 'newSessionExpectError'; additionalDirectories?: string[] } @@ -69,6 +89,25 @@ type InputStep = /** A scenario's `input.json`: an ordered list of input steps. */ export interface InputScript { steps: InputStep[] + /** + * Ordered answers for the agent's `session/request_permission` round-trips, + * consumed FIFO — the Nth request gets the Nth answer. Each answer selects + * by option KIND: option ids are agent-issued randoms a committed script + * cannot know, while kinds are the ACP-stable vocabulary, so the client maps + * kind → the offered `optionId` at answer time. A request beyond the queue + * (or with no queue at all) is answered `cancelled` — the stub behavior a + * scenario without approvals relies on. A scripted kind the request does + * not offer REJECTS the run: the scenario scripted an impossible click, + * and {@link runScenario} throws once the in-flight step settles (the + * agent itself just sees `cancelled`, so it cannot absorb the bug). + */ + permissionAnswers?: PermissionAnswer[] +} + +/** One scripted answer to a permission request: which offered option kind to select. */ +export interface PermissionAnswer { + /** The `PermissionOption.kind` to select (`allow_once`, `reject_always`, …). */ + kind: 'allow_once' | 'allow_always' | 'reject_once' | 'reject_always' } /** One harvested session log plus the identifying facts off its header line. */ @@ -102,7 +141,10 @@ export interface RunResult { sessionLogs: HarvestedLog[] } -interface RunOptions { +/** How to run one scenario: the agent to boot, the mode, and the fixture wiring. */ +export interface RunOptions { + /** The agent composition to boot. */ + agent: AgentUnderTest /** `replay` (default, keyless) or `record` (real API, harvests the log). */ mode: 'replay' | 'record' /** The recorded session JSONL fixture path (replay reads it; record writes near it). */ @@ -130,6 +172,10 @@ interface RunOptions { * Run a scenario end-to-end against a freshly-spawned subprocess. Owns the * child and its temp dirs; always tears them down. Returns the captured stdout * and (record mode) the harvested session-log path. + * + * @param input The scenario's input script (steps + optional permission answers). + * @param opts The agent to boot, the mode, and the fixture wiring. + * @returns The captured stdout/stderr, session id, temp cwd, and harvested logs. */ export async function runScenario(input: InputScript, opts: RunOptions): Promise { const cwd = await mkdtemp(join(tmpdir(), 'acp-snap-cwd-')) @@ -151,7 +197,7 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise } const env: NodeJS.ProcessEnv = { ...process.env, - TSX_TSCONFIG_PATH: repoTsconfig, + TSX_TSCONFIG_PATH: opts.agent.tsconfigPath, DSH_SNAPSHOT: opts.mode, DSH_SNAPSHOT_FILE: opts.fixtureFile, DSH_SNAPSHOT_SESSIONS_ROOT: sessionsRoot, @@ -163,7 +209,7 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise child = spawn( process.execPath, - ['--import', tsxLoader, binScript, configPath], + ['--import', tsxLoader, opts.agent.binScript, opts.agent.configPath], { cwd, env, stdio: ['pipe', 'pipe', 'pipe'] }, ) @@ -193,25 +239,59 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise const waitForUpdate = (match: (u: SessionNotification['update']) => boolean): Promise => new Promise(resolve => updateWaiters.push({ match, resolve })) + // Permission answers are consumed FIFO across the whole run; exhaustion + // falls back to `cancelled` so approval-free scenarios keep the plain stub. + const permissionQueue = [...input.permissionAnswers ?? []] + // A scenario bug detected inside a client callback (a scripted permission + // kind the agent never offered). It cannot fail the run from in there: a + // callback throw only becomes a JSON-RPC error RESPONSE to the agent, and + // a tolerant agent treats that as a denial and carries on — the run (or + // worse, a record) would absorb the impossible click silently. So the + // callback answers `cancelled` (a well-defined path for the agent), + // captures the error here, and the step loop fails the run on it. + let scriptError: Error | undefined const makeClient = (_agent: AcpAgent): Client => ({ sessionUpdate(params: SessionNotification): Promise { for (let i = updateWaiters.length - 1; i >= 0; i--) { const waiter = updateWaiters[i] - if (waiter !== undefined && waiter.match(params.update)) { + // The index is always in-bounds (i only decreases; splice removes at + // i, so lower entries stay valid); the guard satisfies + // noUncheckedIndexedAccess. + /* v8 ignore next 1 -- unreachable in-bounds guard, see above */ + if (waiter === undefined) continue + if (waiter.match(params.update)) { updateWaiters.splice(i, 1) waiter.resolve() } } return Promise.resolve() }, - requestPermission(_params: RequestPermissionRequest): Promise { - return Promise.resolve({ outcome: { outcome: 'cancelled' } }) + requestPermission(params: RequestPermissionRequest): Promise { + const answer = permissionQueue.shift() + if (answer === undefined) return Promise.resolve({ outcome: { outcome: 'cancelled' } }) + const option = params.options.find(o => o.kind === answer.kind) + if (option === undefined) { + // The scenario scripted a click the agent never offered — a scenario + // bug. Captured (last one wins; same bug class either way) and + // answered `cancelled`; the step loop rejects the run on it. + scriptError = new Error( + `snapshot-harness: scripted permission answer ${answer.kind} not among ` + + `the offered options [${params.options.map(o => o.kind).join(', ')}]`, + ) + return Promise.resolve({ outcome: { outcome: 'cancelled' } }) + } + return Promise.resolve({ outcome: { outcome: 'selected', optionId: option.optionId } }) }, }) const client = new ClientSideConnection(makeClient, stream) for (const step of input.steps) { await runStep(client, step, cwd, waitForUpdate, () => sessionId, (id) => { sessionId = id }) + // A permission exchange happens while a step's request is in flight, so + // by the time the step settles any script bug it exposed is captured — + // fail the run HERE, as a harness error, rather than hoping the agent's + // reaction to the answer perturbs the transcript. + if (scriptError !== undefined) throw scriptError } // Done driving: close stdin so the server disposes gracefully (flushing // persistence) and exits. Then await exit so the harvested log is complete. @@ -302,9 +382,9 @@ async function runStep( // its own). To pin frame order deterministically, wait until the client // has OBSERVED the hang's streamed agent_message_chunk before cancelling — // so those update frames always precede the cancelled prompt response in - // the transcript (without this, the late chunk and the response race; see - // the Codex review of commit 5). Then cancel and await the prompt, which - // the bridge settles as `cancelled` once the abort propagates. + // the transcript (without this, the late chunk and the response race). + // Then cancel and await the prompt, which the bridge settles as + // `cancelled` once the abort propagates. const promptDone = client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) await waitForUpdate(u => u.sessionUpdate === 'agent_message_chunk') await client.cancel({ sessionId }) @@ -324,6 +404,10 @@ async function runStep( /** Resolve once the child process exits (any code/signal). */ function waitForExit(child: ChildProcessWithoutNullStreams): Promise { + // Race guard: both call sites run within one synchronous frame of + // stdin.end()/kill(), so the exit event cannot have been delivered yet; + // kept for any future caller that awaits in between. + /* v8 ignore next 1 -- unreachable race guard, see above */ if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve() return new Promise(resolve => child.once('exit', () => { resolve() })) } @@ -335,8 +419,8 @@ function waitForExit(child: ChildProcessWithoutNullStreams): Promise { * * The JSONL backend lays sessions out as `//.jsonl` * (one bucket per cwd), so a parent and its same-cwd in-process child land in - * the SAME bucket — collecting all files across all buckets catches both (the - * old first-match short-circuit silently dropped the child). Returns `[]` if no + * the SAME bucket — collecting all files across all buckets catches both (a + * first-match short-circuit would silently drop the child). Returns `[]` if no * log was produced (a no-session scenario). */ async function harvestSessionLogs(root: string): Promise { diff --git a/packages/support/acp-snapshot/src/index.ts b/packages/support/acp-snapshot/src/index.ts new file mode 100644 index 0000000000..bbe74030f2 --- /dev/null +++ b/packages/support/acp-snapshot/src/index.ts @@ -0,0 +1,38 @@ +/** + * ACP snapshot suite kit — the shared machinery behind the keyless snapshot + * tier (`pnpm run test:snapshot`). Three layers, composable per example: + * the subprocess scenario harness ({@link runScenario}), the pure golden + * normalizers ({@link normalizeStdout} / {@link normalizeSessionLog} / + * {@link scrubRequestHeaders}), and the suite factory + * ({@link defineAcpSnapshotSuite}) that registers a scenario table as a full + * describe/it tree. An example's `*.snapshot.ts` supplies only its + * {@link AgentUnderTest} paths, its snapshots directory, and its + * {@link Scenario} table. + * + * NOTE: ./suite.ts imports vitest, so this package is importable only inside a + * vitest run — a support-tier constraint stated in the README. + * + * @module @deepseek-ai/dsh-acp-snapshot + */ + +export { + runScenario, + type AgentUnderTest, + type HarvestedLog, + type InputScript, + type InputStep, + type PermissionAnswer, + type RunOptions, + type RunResult, +} from './harness.ts' +export { + normalizeSessionLog, + normalizeStdout, + scrubRequestHeaders, + type NormalizeContext, +} from './normalize.ts' +export { + defineAcpSnapshotSuite, + type Scenario, + type SnapshotSuiteOptions, +} from './suite.ts' diff --git a/examples/acp-agent/tests/snapshot-normalize.ts b/packages/support/acp-snapshot/src/normalize.ts similarity index 91% rename from examples/acp-agent/tests/snapshot-normalize.ts rename to packages/support/acp-snapshot/src/normalize.ts index 28c10f102d..2cbe914b42 100644 --- a/examples/acp-agent/tests/snapshot-normalize.ts +++ b/packages/support/acp-snapshot/src/normalize.ts @@ -15,12 +15,15 @@ * A separate, composable normalizer — {@link scrubRequestHeaders} — replaces * the bulky request-header CONTENT (the composed system prompt and the tool * schema list) with `{{system}}`/`{{tools}}` tokens. It is deliberately NOT - * folded into {@link normalizeSessionLog}: the one header-pinning scenario - * compares that content verbatim, every other scenario composes the scrub in - * (the `pinsHeader` flag in acp.snapshot.ts; see the pinned-header RFC, + * folded into {@link normalizeSessionLog}: each suite's one header-pinning + * scenario compares that content verbatim, every other scenario composes the + * scrub in (the `pinsHeader` flag on the scenario table, consumed by the suite + * factory in ./suite.ts; see the pinned-header RFC, * docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md). * * See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md. + * + * @module @deepseek-ai/dsh-acp-snapshot/normalize */ const SESSION_ID = '{{sessionId}}' @@ -69,6 +72,10 @@ function scrubValue(value: unknown, ctx: NormalizeContext): unknown { * (1, 2, 3, …) and all volatile strings scrubbed. Throws if any non-empty line * is not valid JSON — that doubles as the stdout-purity check (no logger leaked * onto the protocol). + * + * @param rawStdout The captured stdout bytes, decoded utf8. + * @param ctx The run's volatile values to scrub. + * @returns The normalized NDJSON transcript, one frame per line. */ export function normalizeStdout(rawStdout: string, ctx: NormalizeContext): string { const lines = rawStdout.split('\n').filter(line => line.trim().length > 0) @@ -97,6 +104,10 @@ export function normalizeStdout(rawStdout: string, ctx: NormalizeContext): strin * zeroed/scrubbed, all volatile strings scrubbed, and `seq` is LEFT INTACT * (deterministic by contract). Output is JSONL in the same shape as the input — * one compact record per line. + * + * @param rawLog The raw session `.jsonl` content. + * @param ctx The run's volatile values to scrub. + * @returns The normalized JSONL log, one record per line. */ export function normalizeSessionLog(rawLog: string, ctx: NormalizeContext): string { const lines = rawLog.split('\n').filter(line => line.trim().length > 0) @@ -140,7 +151,10 @@ export function normalizeSessionLog(rawLog: string, ctx: NormalizeContext): stri * Only lines with something to scrub are re-serialized; every other line * passes through byte-for-byte, so the transform is idempotent and applying * it to an already-scrubbed fixture is a no-op — the on-disk-fixtures guard - * in acp.snapshot.ts relies on exactly that. + * in ./suite.ts relies on exactly that. + * + * @param rawLog The raw session `.jsonl` content. + * @returns The JSONL with header content tokenized, other lines byte-identical. */ export function scrubRequestHeaders(rawLog: string): string { const lines = rawLog.split('\n') diff --git a/packages/support/acp-snapshot/src/suite.ts b/packages/support/acp-snapshot/src/suite.ts new file mode 100644 index 0000000000..9718fc7d5e --- /dev/null +++ b/packages/support/acp-snapshot/src/suite.ts @@ -0,0 +1,384 @@ +/** + * The ACP snapshot suite factory (REPLAY by default, keyless). A suite is a + * scenario table plus a snapshots directory: each scenario under + * `//` ships an `input.json` (the client stdin script) and + * a `session.jsonl` fixture; replay boots the real agent subprocess + * (./harness.ts), drives it, and diffs the normalized stdout transcript + * against the committed `stdout.golden.jsonl`. For model scenarios it ALSO + * checks the re-persisted session log — against the `session.jsonl` fixture + * itself, not a separate golden: the fixture doubles as the replay source + * (recorded scenarios) and the expected produced log (both sides normalized + * before comparing). + * + * Request-header content (the composed system prompt + tool schemas riding on + * `request/header` events) is pinned by exactly ONE scenario per suite — the + * one with `pinsHeader` — and scrubbed to `{{system}}`/`{{tools}}` tokens in + * every other fixture and compare, so a prompt or tool-schema edit churns one + * committed line instead of every fixture. A per-run uniformity guard keeps + * the single pin sound: every live header must equal the pinned one, and no + * header-delta may appear outside the pinning scenario (see the + * pinned-header RFC, + * docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md). + * + * `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the + * `session.jsonl` fixtures against the real API and refreshes the stdout golden + * in one pass; the caller resolves that env into {@link SnapshotSuiteOptions} + * (env reading stays at the suite edge, not in this library). + * + * @module @deepseek-ai/dsh-acp-snapshot/suite + */ + +import { readFile, readdir, writeFile } from 'node:fs/promises' +import { existsSync } from 'node:fs' +import { join } from 'node:path' +import { describe, expect, it } from 'vitest' +import { type AgentUnderTest, type HarvestedLog, type InputScript, runScenario } from './harness.ts' +import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from './normalize.ts' + +/** A snapshot scenario and how its fixtures are produced. */ +export interface Scenario { + name: string + /** Whether the scenario drives at least one model turn (so a JSONL golden applies). */ + hasModelTurn: boolean + /** + * Whether the run persists a comparable session log to diff against the + * `session.jsonl` fixture. Defaults to {@link hasModelTurn} (a model turn + * always produces a log worth comparing). Set it independently for a scenario + * that produces a non-trivial log WITHOUT a model turn — e.g. a prompt blocked + * by a `UserPromptSubmit` hook, which opens a `rejected` turn carrying `hook/*` + * events but never calls the model. + */ + comparesLog?: boolean + /** + * Whether `test:snapshot:record` regenerates this scenario's `session.jsonl` + * from the LIVE API. `recorded` scenarios are model-driven and reproducible; + * `authored` scenarios (fixtures hand-written or hand-harvested — e.g. a + * provider error or a cancel the live API can't be coaxed into + * deterministically, a deterministic hook scenario, or a scripted repetition + * a live model won't reproduce) are NEVER re-recorded. + */ + recorded: boolean + /** + * Whether replay is driven by a hand-written `replay.override.json` sidecar + * (a `ReplayEntry[]` that REPLACES the script derived from `session.jsonl`) + * — the throw/hang cases chunks cannot express. The fixture guard requires + * the sidecar exactly when this is set: the harness forwards the file purely + * on existence, so an unregistered stray sidecar would silently replace the + * derived script — the guard fails loud on either mismatch. Defaults to + * false (replay derives from the fixture's `assistant/chunk` events). + */ + overridden?: boolean + /** + * How many SUBAGENT child sessions this scenario records beyond the top-level + * one (0 for a single-session scenario). Each child rides in a sibling fixture + * `session..jsonl` (1-based); replay forwards them to `dsh-llm-replay` so + * each child session replays from its own script, and record mode writes the + * harvested child logs back to those files. Defaults to 0. + */ + childSessions?: number + /** + * Whether THIS scenario's fixtures keep the full request-header content (the + * composed system prompt and tool schema list on `request/header` / + * `request/header-delta` events) and compare it verbatim. Exactly one + * scenario per suite pins it; every other scenario stores and compares that + * content as `{{system}}`/`{{tools}}` tokens ({@link scrubRequestHeaders}), + * so a system prompt or tool-schema change shows up as ONE committed-fixture + * diff, not one per scenario. One pin suffices because header composition is + * suite-uniform (parent, spawn child, and fork child all compose the same + * prompt-modulo-cwd and the same tools) — and that premise is ASSERTED, not + * assumed: every non-pinning run's live headers must equal the pinned + * fixture's (normalized), so a session-dependent header (say, a restricted + * subagent toolset) fails loud until it gets its own pinning scenario. + * Defaults to false. + */ + pinsHeader?: boolean +} + +/** One suite's inputs: the agent to boot, where its fixtures live, and its scenario table. */ +export interface SnapshotSuiteOptions { + /** The agent composition every scenario boots. */ + agent: AgentUnderTest + /** Absolute path of the suite's `snapshots/` directory (one subdir per scenario). */ + snapshotsDir: string + /** The scenario table; exactly one entry must set `pinsHeader`. */ + scenarios: Scenario[] + /** + * `replay` (keyless, the default tier) or `record` (live API; re-records the + * `recorded` scenarios' fixtures and refreshes the vitest goldens under + * `--update`). The caller derives this from `$DSH_SNAPSHOT` — env reading + * stays outside this library. + */ + mode: 'replay' | 'record' +} + +/** + * The sibling child-fixture paths for a scenario (`session.1.jsonl` …). + * + * @param dir The scenario's snapshots directory (`/`). + * @param childSessions How many subagent child sessions the scenario records. + * @returns One path per child, 1-based, in fixture order. + */ +export function childFixturePaths(dir: string, childSessions: number): string[] { + return Array.from({ length: childSessions }, (_, i) => join(dir, `session.${i + 1}.jsonl`)) +} + +/** + * Derive the {@link NormalizeContext} for a `session.jsonl` fixture from its own + * header line (`{ type: 'session', id, cwd }`). A committed fixture carries the + * session id and cwd of the run that harvested it — different from the live + * replay run — so normalizing it against the live run's ctx would leave those + * recorded values unscrubbed. Reading them from the header scrubs the fixture's + * own id/cwd to the same `{{sessionId}}`/`{{cwd}}` tokens the replay output gets. + * An authored fixture whose header is already normalized (`id:'{{sessionId}}'`, + * `cwd:'{{cwd}}'`) yields those tokens as the volatile values, so scrubbing them + * is an idempotent no-op. A header with no `cwd` falls back to a sentinel that + * cannot occur in a log (NOT `''`, which `String.split` would match on every + * character boundary and corrupt the output). + * + * @param fixture The committed `session.jsonl` content. + * @returns The fixture's own volatile values, ready for {@link normalizeSessionLog}. + */ +export function fixtureContext(fixture: string): NormalizeContext { + const firstLine = fixture.split('\n').find(line => line.trim().length > 0) ?? '{}' + const header = JSON.parse(firstLine) as { id?: unknown; cwd?: unknown } + return { + sessionIds: typeof header.id === 'string' ? [header.id] : [], + cwd: typeof header.cwd === 'string' ? header.cwd : '\0no-cwd\0', + } +} + +/** + * The `data.header` payload of every `request/header` event in a session + * JSONL, in log order, with the log's volatile values scrubbed first + * ({@link normalizeSessionLog}) so headers harvested from different runs — + * each embedding its own temp cwd in the composed prompt — compare on equal + * footing. + * + * @param rawLog The session `.jsonl` content to extract headers from. + * @param ctx The volatile values of the run that produced it. + * @returns The normalized `data.header` payloads, in log order. + */ +export function normalizedHeaders(rawLog: string, ctx: NormalizeContext): unknown[] { + return normalizeSessionLog(rawLog, ctx) + .split('\n') + .filter(line => line.trim().length > 0) + .map(line => JSON.parse(line) as { type?: unknown; data?: { header?: unknown } }) + .filter(record => record.type === 'request/header') + .map(record => record.data?.header) +} + +/** + * Count the `request/header-delta` events in a session JSONL. + * + * @param rawLog The session `.jsonl` content. + * @returns How many `request/header-delta` events the log carries. + */ +export function headerDeltaCount(rawLog: string): number { + return rawLog.split('\n') + .filter(line => line.trim().length > 0) + .filter(line => (JSON.parse(line) as { type?: unknown }).type === 'request/header-delta') + .length +} + +/** + * Register the suite: one `describe` per scenario (the golden/log compares and + * the header-uniformity guard) plus the fixture guard block (no orphan + * scenario dirs, required files present, exactly one pin, non-pinning fixtures + * header-scrubbed). Must run at vitest collection time — it calls + * `describe`/`it`. Throws immediately if no scenario pins the header (the + * uniformity guard would have nothing to compare against). + * + * @param options The agent, snapshots directory, scenario table, and mode. + */ +export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { + const { agent, snapshotsDir, scenarios, mode } = options + const RECORDING = mode === 'record' + + /** The suite's single header-pinning scenario. Guarded here (and by a meta-test) so the pin cannot silently vanish. */ + const pinningScenario = scenarios.find(s => s.pinsHeader === true) + if (pinningScenario === undefined) throw new Error('acp-snapshot: no scenario pins the request-header content') + + for (const scenario of scenarios) { + describe(`snapshot: ${scenario.name}`, () => { + // In RECORD mode, only re-run the `recorded` (live-API) scenarios; the + // `authored` ones (sidecar-driven errors/cancel) are never re-recorded. + it.skipIf(RECORDING && !scenario.recorded)('matches the goldens', async () => { + const dir = join(snapshotsDir, scenario.name) + const input = JSON.parse(await readFile(join(dir, 'input.json'), 'utf8')) as InputScript + const overrideFile = join(dir, 'replay.override.json') + const workspaceDir = join(dir, 'workspace') + const childSessions = scenario.childSessions ?? 0 + const result = await runScenario(input, { + agent, + mode, + fixtureFile: join(dir, 'session.jsonl'), + ...existsSync(overrideFile) ? { overrideFile } : {}, + // In REPLAY, forward the recorded child fixtures so each subagent session + // replays from its own script. In RECORD they are harvested, not read. + ...!RECORDING && childSessions > 0 ? { childFiles: childFixturePaths(dir, childSessions) } : {}, + ...existsSync(workspaceDir) ? { workspaceDir } : {}, + }) + + // Scrub every volatile id the run produced: the ACP server-issued session + // id plus every harvested log's recorded id (a subagent child id never + // surfaces over ACP, but it appears in the child's own log header). The + // normalizer's UUID catch-all covers any we don't enumerate. + const ctx: NormalizeContext = { + sessionIds: [ + ...result.sessionId !== undefined ? [result.sessionId] : [], + ...result.sessionLogs.map(l => l.id), + ], + cwd: result.cwd, + } + + // RECORD mode (recorded model scenarios only): persist the freshly-harvested + // logs back to their fixtures — the primary to session.jsonl, each child to + // session..jsonl in harvest order. `--update` refreshes the Vitest + // goldens but NOT these fixtures, so write them here. A non-pinning + // scenario's fixtures are written header-scrubbed, so a re-record can + // never smuggle the full prompt/schema content back into every fixture. + const scrub = scenario.pinsHeader === true + ? (log: string): string => log + : scrubRequestHeaders + if (RECORDING && scenario.recorded && scenario.hasModelTurn) { + expect(result.sessionLogs.length, 'record produced no session log to harvest').toBeGreaterThan(0) + expect(result.sessionLogs.length, `expected ${childSessions + 1} session logs (parent + children)`) + .toBe(childSessions + 1) + await writeFile(join(dir, 'session.jsonl'), scrub((result.sessionLogs[0] as HarvestedLog).content)) + for (let i = 1; i < result.sessionLogs.length; i++) { + await writeFile(join(dir, `session.${i}.jsonl`), scrub((result.sessionLogs[i] as HarvestedLog).content)) + } + } + + await expect(normalizeStdout(result.rawStdout, ctx)) + .toMatchFileSnapshot(join(dir, 'stdout.golden.jsonl')) + + // A model turn always produces a log worth comparing; a hook scenario can + // produce one without a model turn (a `rejected` turn carrying `hook/*`). + const comparesLog = scenario.comparesLog ?? scenario.hasModelTurn + if (comparesLog) { + // The harvested logs (primary-first) must match their committed fixtures + // 1:1. Each side passes through normalizeSessionLog, scrubbed against ITS + // OWN volatile values — the live run's via `ctx`, the committed fixture's + // via its own header (a committed file cannot share the live run's ids). + // Unless this scenario pins the header, both sides ALSO pass through + // scrubRequestHeaders: the live log carries the real prompt/schemas, the + // fixture carries the `{{system}}`/`{{tools}}` tokens, and the scrub is + // idempotent — so the compare checks the header's presence, position, + // reason, and config, but not its bulk content (pinned once, in the + // `pinsHeader` scenario). + expect(result.sessionLogs.length, 'this scenario must persist a session log').toBe(childSessions + 1) + const fixtureFiles = ['session.jsonl', ...Array.from({ length: childSessions }, (_, i) => `session.${i + 1}.jsonl`)] + for (let i = 0; i < fixtureFiles.length; i++) { + const harvested = scrub((result.sessionLogs[i] as HarvestedLog).content) + const fixture = scrub(await readFile(join(dir, fixtureFiles[i] as string), 'utf8')) + expect(normalizeSessionLog(harvested, ctx), `${fixtureFiles[i]} mismatch`) + .toEqual(normalizeSessionLog(fixture, fixtureContext(fixture))) + } + } + + // Header-uniformity guard: the single pin is sound only while every + // session in the suite composes the SAME header and keeps it for the + // whole run. Assert both halves live. (1) Every request/header the run + // produced (parent, spawn child, fork child, initial or resume) must + // equal the pinned fixture's header after each side is normalized + // against its own volatile values. (2) No request/header-delta may + // appear at all — a mid-run header change diverges from the pin by + // construction, and its content would be invisible under the scrub. If + // either fails, either the header changed (update the pin: re-record or + // hand-edit the pinning scenario's fixture) or composition became + // session-dependent by design (give the divergent shape its own + // pinning scenario). + if (scenario.pinsHeader !== true) { + const pinnedFixture = await readFile(join(snapshotsDir, pinningScenario.name, 'session.jsonl'), 'utf8') + const pinned = normalizedHeaders(pinnedFixture, fixtureContext(pinnedFixture)) + expect(pinned.length, `the pinning fixture (${pinningScenario.name}) must carry exactly one request/header`) + .toBe(1) + for (const log of result.sessionLogs) { + expect(headerDeltaCount(log.content), `session ${log.id}: a request/header-delta in a non-pinning scenario`) + .toBe(0) + const headers = normalizedHeaders(log.content, ctx) + for (const [k, header] of headers.entries()) { + expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`) + .toEqual(pinned[0]) + } + } + } + }) + }) + } + + describe('snapshot fixtures', () => { + it('every scenario directory is registered (no orphans)', async () => { + // toMatchFileSnapshot does not prune orphaned golden/fixture files, so a + // renamed/removed scenario could leave a stale dir that nothing exercises. + // Fail loud on any snapshots/ not present in the scenario table. + const entries = await readdir(snapshotsDir, { withFileTypes: true }) + const onDisk = entries.filter(e => e.isDirectory()).map(e => e.name).sort() + const registered = scenarios.map(s => s.name).sort() + expect(onDisk).toEqual(registered) + }) + + it('every registered scenario has its required fixture files', () => { + // Every scenario has an input script and an stdout golden. EVERY scenario + // also needs `session.jsonl`: the suite boots `llm-replay` with that path + // as the replay source for ALL scenarios (the factory passes + // `fixtureFile: /session.jsonl` unconditionally), and `loadReplayScript` + // throws "fixture not found" when it is absent and no override replaces it. + // A no-model scenario ships a header-only `session.jsonl` (it derives to an + // empty script — no model call is made); a model scenario's fixture also + // doubles as the expected-log artifact the run is diffed against. The + // `replay.override.json` sidecar is matched BOTH ways against the table's + // `overridden` flag: required when set, forbidden when not — the harness + // forwards the file purely on existence, so an unregistered stray sidecar + // would silently replace the derived script. + for (const { name, overridden, childSessions } of scenarios) { + const dir = join(snapshotsDir, name) + expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true) + expect(existsSync(join(dir, 'stdout.golden.jsonl')), `${name}/stdout.golden.jsonl`).toBe(true) + expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true) + expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json presence must match \`overridden\``) + .toBe(overridden === true) + // A nested-agent scenario ships one child fixture per recorded subagent + // session (`session.1.jsonl` …), the replay source for that child session. + for (const childFixture of childFixturePaths(dir, childSessions ?? 0)) { + expect(existsSync(childFixture), childFixture).toBe(true) + } + } + }) + + it('exactly one scenario pins the request-header content', () => { + // Zero pins would drop the prompt/schema surface from the suite entirely; + // two would split it. One pin per suite is the design (pinned-header RFC); + // WHICH scenario pins is the scenario table's reviewable choice. + expect(scenarios.filter(s => s.pinsHeader === true).map(s => s.name)).toEqual([pinningScenario.name]) + }) + + it('committed fixtures carry request-header content ONLY in the pinning scenario', async () => { + // The whole point of the pin: a system-prompt or tool-schema change must + // churn exactly one committed line. A non-pinning fixture that carries the + // full header (a hand-recorded file, or a header line hand-edited out of + // its canonical JSON form) silently reopens the suite-wide churn, so fail + // loud here: every non-pinning session*.jsonl must be a fixed point of + // scrubRequestHeaders (apply the scrub to fix a violation), and the + // pinning scenario's fixtures must NOT be (their content IS the pin). + for (const scenario of scenarios) { + const dir = join(snapshotsDir, scenario.name) + const files = [ + 'session.jsonl', + ...Array.from({ length: scenario.childSessions ?? 0 }, (_, i) => `session.${i + 1}.jsonl`), + ] + for (const file of files) { + const fixture = await readFile(join(dir, file), 'utf8') + if (scenario.pinsHeader === true) { + expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} must PIN the full header content`) + .not.toEqual(fixture) + } else { + expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} carries unscrubbed header content`) + .toEqual(fixture) + } + } + } + }) + }) +} diff --git a/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts b/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts new file mode 100644 index 0000000000..cf41412046 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts @@ -0,0 +1,232 @@ +/** + * Scripted fake ACP agent bin for `dsh-acp-snapshot`'s unit specs. Speaks + * newline-delimited JSON-RPC on stdio like the real `dsh-acp-agent` bin, but + * every behavior — how prompts settle, whether session/new rejects, which + * session logs get persisted, what filesystem noise to leave — comes from a + * `behavior.json` sitting NEXT to the `$DSH_SNAPSHOT_FILE` fixture, so a spec + * scripts a whole subprocess run from data. The specs launch it through the + * REAL `runScenario` spawn path (tsx loader, temp cwd, env plumbing), so the + * harness plumbing is exercised for real; only the agent behind the protocol + * is scripted. + * + * The specs (not the golden tier) own this bin: it asserts nothing, echoes + * observable facts into `session/update` text chunks (env probe, permission + * outcome, seeded-workspace listing) for the spec to read off `rawStdout`, and + * exits 0 on stdin EOF after writing the scripted logs — mirroring the real + * bin's dispose-flush-exit shape. + */ + +import { mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { readdirSync } from 'node:fs' +import { dirname, join } from 'node:path' +import { randomUUID } from 'node:crypto' +import { createInterface } from 'node:readline' + +/** One scripted session log: a file path under the sessions root plus its JSONL lines. */ +interface ScriptedLog { + /** Path relative to `$DSH_SNAPSHOT_SESSIONS_ROOT`, e.g. `bucket/a.jsonl` (an empty dir segment is invalid). */ + file: string + /** + * The JSONL records. String templates `{{CWD}}` and `{{SID}}` are replaced + * with the run's real cwd and the ACP session id this bin issued, so a + * written log carries genuine volatile values for the normalizers to scrub. + */ + lines: unknown[] +} + +/** The whole scripted behavior for one run. Every field defaults to the least surprising choice. */ +interface Behavior { + /** Reject every `session/new` (exercises the expect-error step without extra dirs). */ + rejectNewSession?: boolean + /** Reject `session/new` only when `additionalDirectories` is non-empty (the real bridge's rule). */ + rejectExtraDirs?: boolean + /** How `session/prompt` settles: a clean response, a JSON-RPC error, or a hang until `session/cancel`. */ + prompt?: 'respond' | 'error' | 'hang-until-cancel' + /** Before responding to a prompt, send a `session/request_permission` request and echo its outcome as a chunk. */ + permissionProbe?: boolean + /** Echo the `DSH_SNAPSHOT_*` env the harness set as a chunk (spec-side env-plumbing assertions). */ + echoEnv?: boolean + /** Echo the sorted cwd listing as a chunk (spec-side workspace-seeding assertions). */ + echoWorkspace?: boolean + /** Write a line to stderr on boot (spec-side stderr-capture assertions). */ + stderrNote?: string + /** Session logs to persist on stdin EOF. */ + logs?: ScriptedLog[] + /** Leave a stray FILE directly under the sessions root (harvest must skip it). */ + strayRootFile?: boolean + /** Leave a stray non-`.jsonl` file inside a bucket (harvest must skip it). */ + strayBucketFile?: boolean + /** Delete the sessions root entirely (harvest must yield no logs). */ + deleteSessionsRoot?: boolean +} + +const sessionsRoot = process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? '' +const fixtureFile = process.env.DSH_SNAPSHOT_FILE ?? '' +const behavior: Behavior = fixtureFile === '' + ? {} + : JSON.parse(readFileSync(join(dirname(fixtureFile), 'behavior.json'), 'utf8')) as Behavior + +if (behavior.stderrNote !== undefined) process.stderr.write(`${behavior.stderrNote}\n`) + +let nextOutboundId = 1000 +let sessionId = '' +/** + * The cwd the client passed to `session/new` — used verbatim for `{{CWD}}` + * substitution, mirroring the real bin (whose persisted header carries the + * session cwd as given, NOT `process.cwd()`, which the OS realpaths — on + * macOS `/var/folders/…` vs `/private/var/folders/…`). + */ +let sessionCwd = '' +/** The parked prompt request id while `hang-until-cancel` waits for the cancel notification. */ +let parkedPromptId: number | string | null = null +/** Resolvers for permission-probe responses, keyed by outbound request id. */ +const pendingPermission = new Map void>() + +function send(frame: Record): void { + process.stdout.write(`${JSON.stringify({ jsonrpc: '2.0', ...frame })}\n`) +} + +function respond(id: number | string, result: unknown): void { + send({ id, result }) +} + +function respondError(id: number | string, message: string): void { + send({ id, error: { code: -32603, message } }) +} + +function chunk(text: string): void { + send({ + method: 'session/update', + params: { sessionId, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text } } }, + }) +} + +/** Substitute the `{{CWD}}`/`{{SID}}` templates through a scripted log record. */ +function instantiate(value: unknown): unknown { + if (typeof value === 'string') return value.split('{{CWD}}').join(sessionCwd).split('{{SID}}').join(sessionId) + if (Array.isArray(value)) return value.map(instantiate) + if (value !== null && typeof value === 'object') { + const out: Record = {} + for (const [k, v] of Object.entries(value)) out[k] = instantiate(v) + return out + } + return value +} + +async function handlePrompt(id: number | string): Promise { + if ((behavior.prompt ?? 'respond') === 'hang-until-cancel') { + // A thought chunk BEFORE any message chunk: a promptAndCancel waiter + // watches for agent_message_chunk, so this exercises its non-matching + // update path while the waiter is armed. + send({ + method: 'session/update', + params: { sessionId, update: { sessionUpdate: 'agent_thought_chunk', content: { type: 'text', text: 'mulling' } } }, + }) + } + chunk('thinking about it') + if (behavior.echoEnv === true) { + chunk(`env:${JSON.stringify({ + mode: process.env.DSH_SNAPSHOT, + override: process.env.DSH_SNAPSHOT_OVERRIDE ?? null, + childFiles: process.env.DSH_SNAPSHOT_CHILD_FILES ?? null, + })}`) + } + if (behavior.echoWorkspace === true) { + chunk(`workspace:${readdirSync(process.cwd()).sort().join(',')}`) + } + if (behavior.permissionProbe === true) { + const requestId = nextOutboundId++ + const outcome = await new Promise((resolve) => { + pendingPermission.set(requestId, resolve) + send({ + id: requestId, + method: 'session/request_permission', + params: { + sessionId, + toolCall: { toolCallId: 'call_fake_1', title: 'fake tool', kind: 'execute', status: 'pending' }, + options: [ + { optionId: 'opt-allow', name: 'Allow once', kind: 'allow_once' }, + { optionId: 'opt-reject', name: 'Reject once', kind: 'reject_once' }, + ], + }, + }) + }) + chunk(`permission:${JSON.stringify(outcome)}`) + } + switch (behavior.prompt ?? 'respond') { + case 'respond': + respond(id, { stopReason: 'end_turn' }) + return + case 'error': + respondError(id, 'model exploded') + return + case 'hang-until-cancel': + parkedPromptId = id + return + } +} + +function handleFrame(frame: Record): void { + const id = frame.id as number | string | undefined + const method = frame.method as string | undefined + const params = (frame.params ?? {}) as Record + // A response to one of OUR outbound requests (the permission probe). + if (method === undefined && id !== undefined && typeof id === 'number' && pendingPermission.has(id)) { + const resolve = pendingPermission.get(id) as (outcome: unknown) => void + pendingPermission.delete(id) + resolve((frame.result as { outcome?: unknown } | undefined)?.outcome ?? null) + return + } + switch (method) { + case 'initialize': + respond(id as number | string, { protocolVersion: 1, agentCapabilities: { loadSession: false } }) + return + case 'session/new': { + const extra = params.additionalDirectories as unknown[] | undefined + if (behavior.rejectNewSession === true || (behavior.rejectExtraDirs === true && extra !== undefined && extra.length > 0)) { + respondError(id as number | string, 'unsupported workspace scope') + return + } + sessionId = randomUUID() + sessionCwd = typeof params.cwd === 'string' ? params.cwd : process.cwd() + respond(id as number | string, { sessionId }) + return + } + case 'session/prompt': + void handlePrompt(id as number | string) + return + case 'session/cancel': + if (parkedPromptId !== null) { + const parked = parkedPromptId + parkedPromptId = null + respond(parked, { stopReason: 'cancelled' }) + } + return + default: + // Unknown method: a notification is ignored; a request gets an error so + // the SDK never waits forever on a frame this fake doesn't model. + if (id !== undefined) respondError(id, `unhandled method ${String(method)}`) + } +} + +function flushLogsAndExit(): void { + for (const log of behavior.logs ?? []) { + const target = join(sessionsRoot, log.file) + mkdirSync(dirname(target), { recursive: true }) + writeFileSync(target, log.lines.map(l => JSON.stringify(instantiate(l))).join('\n') + '\n') + } + if (behavior.strayRootFile === true) writeFileSync(join(sessionsRoot, 'stray.txt'), 'not a bucket\n') + if (behavior.strayBucketFile === true) { + mkdirSync(join(sessionsRoot, 'bucket-noise'), { recursive: true }) + writeFileSync(join(sessionsRoot, 'bucket-noise', 'notes.txt'), 'not a session log\n') + } + if (behavior.deleteSessionsRoot === true) rmSync(sessionsRoot, { recursive: true, force: true }) + process.exit(0) +} + +const rl = createInterface({ input: process.stdin }) +rl.on('line', (line) => { + if (line.trim().length === 0) return + handleFrame(JSON.parse(line) as Record) +}) +rl.on('close', () => { flushLogsAndExit() }) diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json new file mode 100644 index 0000000000..d44a3a9698 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json @@ -0,0 +1,13 @@ +{ + "prompt": "respond", + "logs": [ + { "file": "b/parent.jsonl", "lines": [ + { "type": "session", "id": "{{SID}}", "createdAt": 700, "cwd": "{{CWD}}" }, + { "type": "request/header", "seq": 0, "time": 3, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + ]}, + { "file": "b/child.jsonl", "lines": [ + { "type": "session", "id": "abababab-cdcd-4efe-8ada-badabadabada", "createdAt": 800, "cwd": "{{CWD}}", "parentSession": "{{SID}}" }, + { "type": "request/header", "seq": 0, "time": 2, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + ]} + ] +} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/input.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/input.json new file mode 100644 index 0000000000..6d3e49b830 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "rec child" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl new file mode 100644 index 0000000000..1caf2610b3 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl @@ -0,0 +1,2 @@ +{"type":"session","id":"abababab-cdcd-4efe-8ada-badabadabada","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88"} +{"type":"request/header","seq":0,"time":2,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl new file mode 100644 index 0000000000..a2beac360d --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl @@ -0,0 +1,2 @@ +{"type":"session","id":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88","createdAt":700,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW"} +{"type":"request/header","seq":0,"time":3,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/stdout.golden.jsonl new file mode 100644 index 0000000000..f173b45b77 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/stdout.golden.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json new file mode 100644 index 0000000000..a24e30d80a --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json @@ -0,0 +1,10 @@ +{ + "prompt": "respond", + "logs": [{ + "file": "b/main.jsonl", + "lines": [ + { "type": "session", "id": "{{SID}}", "createdAt": 600, "cwd": "{{CWD}}" }, + { "type": "request/header", "seq": 0, "time": 4, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + ] + }] +} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/input.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/input.json new file mode 100644 index 0000000000..9573d20b27 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "rec pin" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl new file mode 100644 index 0000000000..109a192083 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl @@ -0,0 +1,2 @@ +{"type":"session","id":"ccdc749f-56f3-4267-9750-598b5c60b7b2","createdAt":600,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-nOQ4Gy"} +{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"SYS PROMPT","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/stdout.golden.jsonl new file mode 100644 index 0000000000..f173b45b77 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-pin/stdout.golden.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/behavior.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/behavior.json new file mode 100644 index 0000000000..0967ef424b --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/behavior.json @@ -0,0 +1 @@ +{} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/input.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/input.json new file mode 100644 index 0000000000..d1e94c22eb --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/replay.override.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/replay.override.json new file mode 100644 index 0000000000..8ed00c0651 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/replay.override.json @@ -0,0 +1 @@ +[{ "kind": "hang" }] diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/session.jsonl new file mode 100644 index 0000000000..104f2a0df2 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/session.jsonl @@ -0,0 +1 @@ +{"type":"session","id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/stdout.golden.jsonl new file mode 100644 index 0000000000..d6a1d2b232 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-skip/stdout.golden.jsonl @@ -0,0 +1 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/behavior.json b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/behavior.json new file mode 100644 index 0000000000..808d9672b9 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/behavior.json @@ -0,0 +1,10 @@ +{ + "prompt": "error", + "logs": [{ + "file": "b/main.jsonl", + "lines": [ + { "type": "session", "id": "{{SID}}", "createdAt": 500, "cwd": "{{CWD}}" }, + { "type": "turn/end", "seq": 1, "time": 9, "data": { "error": "model exploded" } } + ] + }] +} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/input.json b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/input.json new file mode 100644 index 0000000000..c281971465 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "promptExpectError", "text": "boom" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/replay.override.json b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/replay.override.json new file mode 100644 index 0000000000..e868115f35 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/replay.override.json @@ -0,0 +1 @@ +[{ "kind": "throw", "chunks": [], "message": "model exploded", "code": "PROVIDER" }] diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl new file mode 100644 index 0000000000..36991a214e --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/session.jsonl @@ -0,0 +1,2 @@ +{"type":"session","id":"44444444-3333-4222-8111-000000000000","createdAt":17,"cwd":"/rec/authored-cwd"} +{"type":"turn/end","seq":1,"time":17,"data":{"error":"model exploded"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/stdout.golden.jsonl new file mode 100644 index 0000000000..2a1d69bd93 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/authored-error/stdout.golden.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}} +{"jsonrpc":"2.0","id":3,"error":{"code":-32603,"message":"model exploded"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/behavior.json b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/behavior.json new file mode 100644 index 0000000000..e0a438297d --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/behavior.json @@ -0,0 +1,10 @@ +{ + "prompt": "error", + "logs": [{ + "file": "b/main.jsonl", + "lines": [ + { "type": "session", "id": "{{SID}}", "createdAt": 400, "cwd": "{{CWD}}" }, + { "type": "hook/result", "seq": 1, "time": 8, "data": { "decision": "block", "durationMs": 37 } } + ] + }] +} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/input.json b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/input.json new file mode 100644 index 0000000000..0a711dca3c --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "promptExpectError", "text": "blocked" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl new file mode 100644 index 0000000000..6d8474812d --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/session.jsonl @@ -0,0 +1,2 @@ +{"type":"session","id":"99999999-8888-4777-8666-555555555555","createdAt":13,"cwd":"/rec/blocked-cwd"} +{"type":"hook/result","seq":1,"time":13,"data":{"decision":"block","durationMs":99}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/stdout.golden.jsonl new file mode 100644 index 0000000000..2a1d69bd93 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/blocked-log/stdout.golden.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}} +{"jsonrpc":"2.0","id":3,"error":{"code":-32603,"message":"model exploded"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/no-model/behavior.json b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/behavior.json new file mode 100644 index 0000000000..0967ef424b --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/behavior.json @@ -0,0 +1 @@ +{} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/no-model/input.json b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/input.json new file mode 100644 index 0000000000..d1e94c22eb --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/no-model/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/session.jsonl new file mode 100644 index 0000000000..104f2a0df2 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/session.jsonl @@ -0,0 +1 @@ +{"type":"session","id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/no-model/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/stdout.golden.jsonl new file mode 100644 index 0000000000..d6a1d2b232 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/no-model/stdout.golden.jsonl @@ -0,0 +1 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/behavior.json b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/behavior.json new file mode 100644 index 0000000000..422e0a17e6 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/behavior.json @@ -0,0 +1,11 @@ +{ + "prompt": "respond", + "logs": [{ + "file": "b/main.jsonl", + "lines": [ + { "type": "session", "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}" }, + { "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "type": "turn/start", "seq": 1, "time": 100, "data": { "turn": 1 } } + ] + }] +} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/input.json b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/input.json new file mode 100644 index 0000000000..b9e2d9bbc5 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "pin" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl new file mode 100644 index 0000000000..87bf09c839 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl @@ -0,0 +1,3 @@ +{"type":"session","id":"12121212-3434-4545-8686-787878787878","createdAt":7,"cwd":"/rec/pin-cwd"} +{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"SYS PROMPT","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}} +{"type":"turn/start","seq":1,"time":7,"data":{"turn":1}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/stdout.golden.jsonl new file mode 100644 index 0000000000..f173b45b77 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/stdout.golden.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/behavior.json b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/behavior.json new file mode 100644 index 0000000000..d5cbbf9d28 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/behavior.json @@ -0,0 +1,15 @@ +{ + "prompt": "respond", + "echoWorkspace": true, + "logs": [ + { "file": "b/parent.jsonl", "lines": [ + { "type": "session", "id": "{{SID}}", "createdAt": 200, "cwd": "{{CWD}}" }, + { "type": "request/header", "seq": 0, "time": 5, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "type": "assistant/chunk", "seq": 1, "time": 5, "data": { "turn": 1, "step": 1, "chunk": { "type": "text-delta", "index": 0, "text": "hi" } } } + ]}, + { "file": "b/child.jsonl", "lines": [ + { "type": "session", "id": "eeeeeeee-1111-4222-8333-444444444444", "createdAt": 300, "cwd": "{{CWD}}", "parentSession": "{{SID}}" }, + { "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + ]} + ] +} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/input.json b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/input.json new file mode 100644 index 0000000000..60b9e363b5 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/input.json @@ -0,0 +1 @@ +{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "plain" }] } diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl new file mode 100644 index 0000000000..a844f891fc --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl @@ -0,0 +1,2 @@ +{"type":"session","id":"eeeeeeee-1111-4222-8333-444444444444","createdAt":12,"cwd":"/rec/plain-cwd","parentSession":"56565656-7878-4989-8a9a-9b9b9b9b9b9b"} +{"type":"request/header","seq":0,"time":12,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl new file mode 100644 index 0000000000..744998f959 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/session.jsonl @@ -0,0 +1,3 @@ +{"type":"session","id":"56565656-7878-4989-8a9a-9b9b9b9b9b9b","createdAt":11,"cwd":"/rec/plain-cwd"} +{"type":"request/header","seq":0,"time":11,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"assistant/chunk","seq":1,"time":11,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"hi"}}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/stdout.golden.jsonl b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/stdout.golden.jsonl new file mode 100644 index 0000000000..d0242ae39f --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/stdout.golden.jsonl @@ -0,0 +1,5 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"workspace:seed.txt"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/workspace/seed.txt b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/workspace/seed.txt new file mode 100644 index 0000000000..c19e887d68 --- /dev/null +++ b/packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/workspace/seed.txt @@ -0,0 +1 @@ +seeded diff --git a/packages/support/acp-snapshot/tests/harness.spec.ts b/packages/support/acp-snapshot/tests/harness.spec.ts new file mode 100644 index 0000000000..683d2aaf80 --- /dev/null +++ b/packages/support/acp-snapshot/tests/harness.spec.ts @@ -0,0 +1,272 @@ +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { delimiter, join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterAll, describe, expect, it } from 'vitest' +import { runScenario, type AgentUnderTest, type InputStep } from '../src/harness.ts' + +/** + * Unit tests for the subprocess harness, driven through the REAL spawn path + * (tsx loader, temp cwd, env plumbing) against the scripted fake ACP bin in + * ./fixtures/fake-acp-agent.ts. Each case writes a `behavior.json` next to a + * throwaway fixture path; the fake bin echoes observable facts (env, seeded + * workspace, permission outcomes) into `agent_message_chunk` text, so the + * assertions read plain `rawStdout`. + */ + +const AGENT: AgentUnderTest = { + binScript: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)), + // The fake bin ignores its config argv; any real path documents the shape. + configPath: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)), + tsconfigPath: fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)), +} + +/** Temp scenario dirs to drop after the suite. */ +const tempDirs: string[] = [] +afterAll(async () => { + for (const dir of tempDirs) await rm(dir, { recursive: true, force: true }) +}) + +/** Write a behavior.json into a fresh temp dir; return the sibling fixture path the harness points the bin at. */ +async function scenario(behavior: object): Promise<{ dir: string; fixtureFile: string }> { + const dir = await mkdtemp(join(tmpdir(), 'acp-snap-spec-')) + tempDirs.push(dir) + await writeFile(join(dir, 'behavior.json'), JSON.stringify(behavior)) + return { dir, fixtureFile: join(dir, 'session.jsonl') } +} + +const boot: InputStep[] = [{ op: 'initialize' }, { op: 'newSession' }] + +describe('runScenario', () => { + it('drives a full turn: initialize (terminal caps), session, prompt, permission stub, harvest', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ + permissionProbe: true, + logs: [{ + file: 'bucket/main.jsonl', + lines: [ + { type: 'session', id: '{{SID}}', createdAt: 42, cwd: '{{CWD}}' }, + { type: 'turn/start', seq: 1, time: 9, data: { turn: 1 } }, + ], + }], + }) + const result = await runScenario( + { steps: [{ op: 'initialize', terminalOutput: true }, { op: 'newSession' }, { op: 'prompt', text: 'go' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.sessionId).toBeDefined() + // The harness's client answers a permission request with `cancelled`; the + // fake bin echoes the outcome it received back as a chunk. + expect(result.rawStdout).toContain('permission:{\\"outcome\\":\\"cancelled\\"}') + expect(result.sessionLogs).toHaveLength(1) + expect(result.sessionLogs[0]?.id).toBe(result.sessionId) + expect(result.sessionLogs[0]?.createdAt).toBe(42) + expect(result.sessionLogs[0]?.content).toContain('turn/start') + // The harvested log embeds the run's REAL temp cwd (template-substituted). + expect(result.sessionLogs[0]?.content).toContain(result.cwd) + }) + + it('forwards override/child fixture paths into the child env and captures stderr', { timeout: 20_000 }, async () => { + const { dir, fixtureFile } = await scenario({ echoEnv: true, stderrNote: 'fake bin booted' }) + const childFiles = [join(dir, 'session.1.jsonl'), join(dir, 'session.2.jsonl')] + const result = await runScenario( + { steps: [...boot, { op: 'prompt', text: 'env?' }] }, + { + agent: AGENT, + mode: 'replay', + fixtureFile, + overrideFile: join(dir, 'replay.override.json'), + childFiles, + // A workspaceDir that does not exist is skipped, not an error. + workspaceDir: join(dir, 'no-such-workspace'), + }, + ) + expect(result.stderr).toContain('fake bin booted') + expect(result.rawStdout).toContain('replay.override.json') + // Child paths ride one env var, joined with the platform delimiter. + expect(result.rawStdout).toContain(JSON.stringify(childFiles.join(delimiter)).slice(1, -1)) + }) + + it('seeds the workspace dir into the temp cwd before the run', { timeout: 20_000 }, async () => { + const { dir, fixtureFile } = await scenario({ echoWorkspace: true }) + const workspaceDir = join(dir, 'workspace') + await writeFile(join(dir, 'behavior.json'), JSON.stringify({ echoWorkspace: true })) + const { mkdir } = await import('node:fs/promises') + await mkdir(workspaceDir, { recursive: true }) + await writeFile(join(workspaceDir, 'seeded.txt'), 'hello') + const result = await runScenario( + { steps: [...boot, { op: 'prompt', text: 'ls' }] }, + { agent: AGENT, mode: 'replay', fixtureFile, workspaceDir }, + ) + expect(result.rawStdout).toContain('workspace:seeded.txt') + }) + + it('promptAndCancel waits for the streamed chunk, cancels, and settles the prompt', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ prompt: 'hang-until-cancel' }) + const result = await runScenario( + { steps: [...boot, { op: 'promptAndCancel', text: 'hang' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.rawStdout).toContain('"stopReason":"cancelled"') + // The streamed chunk deterministically precedes the cancelled response. + expect(result.rawStdout.indexOf('thinking about it')).toBeLessThan(result.rawStdout.indexOf('cancelled')) + }) + + it('promptExpectError swallows a model-error response as the expected outcome', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ prompt: 'error' }) + const result = await runScenario( + { steps: [...boot, { op: 'promptExpectError', text: 'boom' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.rawStdout).toContain('model exploded') + }) + + it('promptExpectError throws when the prompt unexpectedly succeeds (and teardown kills the live child)', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ prompt: 'respond' }) + await expect(runScenario( + { steps: [...boot, { op: 'promptExpectError', text: 'fine' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + )).rejects.toThrow(/expected the prompt to fail/) + }) + + it('newSessionExpectError swallows the rejection, with and without extra dirs', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ rejectExtraDirs: true }) + const result = await runScenario( + { steps: [{ op: 'initialize' }, { op: 'newSessionExpectError', additionalDirectories: ['/elsewhere'] }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + // No session was created, so no id and no logs. + expect(result.sessionId).toBeUndefined() + expect(result.sessionLogs).toHaveLength(0) + + const rejectAll = await scenario({ rejectNewSession: true }) + const second = await runScenario( + { steps: [{ op: 'initialize' }, { op: 'newSessionExpectError' }] }, + { agent: AGENT, mode: 'replay', fixtureFile: rejectAll.fixtureFile }, + ) + expect(second.rawStdout).toContain('unsupported workspace scope') + }) + + it('newSessionExpectError throws when session/new unexpectedly succeeds', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({}) + await expect(runScenario( + { steps: [{ op: 'initialize' }, { op: 'newSessionExpectError' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + )).rejects.toThrow(/expected session\/new to be rejected/) + }) + + it('a plain cancel step is forwarded (and ignored by an idle agent)', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({}) + const result = await runScenario( + { steps: [...boot, { op: 'cancel' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.sessionId).toBeDefined() + }) + + it.each([ + [{ op: 'prompt', text: 'x' }, /prompt before newSession/], + [{ op: 'promptExpectError', text: 'x' }, /promptExpectError before newSession/], + [{ op: 'promptAndCancel', text: 'x' }, /promptAndCancel before newSession/], + [{ op: 'cancel' }, /cancel before newSession/], + ] as [InputStep, RegExp][])('rejects %j before newSession', { timeout: 20_000 }, async (step, message) => { + const { fixtureFile } = await scenario({}) + await expect(runScenario( + { steps: [{ op: 'initialize' }, step] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + )).rejects.toThrow(message) + }) + + it('rejects an unknown input op', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({}) + const bogus = { op: 'reticulate' } as unknown as InputStep + await expect(runScenario( + { steps: [bogus] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + )).rejects.toThrow(/unknown input op/) + }) + + it('harvests all logs primary-first, children by createdAt then id, skipping filesystem noise', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ + strayRootFile: true, + strayBucketFile: true, + logs: [ + // File names chosen so readdir feeds the sort children-first AND + // parent-in-the-middle: the comparator then sees a parent on both + // sides of a pair, plus the same-createdAt (localeCompare) tiebreak. + { file: 'b1/aa-child-c.jsonl', lines: [{ type: 'session', id: 'cccccccc-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] }, + { file: 'b1/bb-parent.jsonl', lines: [{ type: 'session', id: '{{SID}}', createdAt: 900 }] }, + { file: 'b1/cc-child-a.jsonl', lines: [{ type: 'session', id: 'aaaaaaaa-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] }, + // Missing id/createdAt fall back to ''/0; earliest child by createdAt. + { file: 'b2/orphan-fields.jsonl', lines: [{ type: 'session', parentSession: '{{SID}}' }] }, + ], + }) + const result = await runScenario( + { steps: [...boot, { op: 'prompt', text: 'go' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.sessionLogs.map(l => [l.id, l.createdAt])).toEqual([ + [result.sessionId, 900], + ['', 0], + ['aaaaaaaa-0000-4000-8000-000000000000', 500], + ['cccccccc-0000-4000-8000-000000000000', 500], + ]) + expect(result.sessionLogs[1]?.parentSession).toBe(result.sessionId) + }) + + it('treats an empty log file as a header-less primary with default fields', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ logs: [{ file: 'b/empty.jsonl', lines: [] }] }) + const result = await runScenario( + { steps: boot }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.sessionLogs.map(l => [l.id, l.createdAt, l.parentSession])).toEqual([['', 0, undefined]]) + }) + + it('yields no logs when the sessions root vanished', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ deleteSessionsRoot: true }) + const result = await runScenario( + { steps: boot }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.sessionLogs).toHaveLength(0) + }) + + it('answers permission requests from the scripted queue by option kind, falling back to cancelled', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ permissionProbe: true }) + // Two prompts → two permission round-trips; one scripted answer, so the + // second request exercises the exhausted-queue fallback. + const result = await runScenario( + { + steps: [...boot, { op: 'prompt', text: 'one' }, { op: 'prompt', text: 'two' }], + permissionAnswers: [{ kind: 'allow_once' }], + }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + const first = result.rawStdout.indexOf('permission:{\\"outcome\\":\\"selected\\",\\"optionId\\":\\"opt-allow\\"}') + const second = result.rawStdout.indexOf('permission:{\\"outcome\\":\\"cancelled\\"}') + expect(first).toBeGreaterThanOrEqual(0) + expect(second).toBeGreaterThan(first) + }) + + it('selects a non-first offered option by kind', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ permissionProbe: true }) + const result = await runScenario( + { steps: [...boot, { op: 'prompt', text: 'deny it' }], permissionAnswers: [{ kind: 'reject_once' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.rawStdout).toContain('permission:{\\"outcome\\":\\"selected\\",\\"optionId\\":\\"opt-reject\\"}') + }) + + it('rejects the run on a scripted permission kind the agent never offered', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ permissionProbe: true }) + // The fake bin offers allow_once/reject_once; scripting allow_always is a + // scenario bug. The agent is answered `cancelled` (it must not be able to + // absorb the bug as an error-means-denial), and the RUN fails: a callback + // throw would only reach the agent as a JSON-RPC error response, letting + // a tolerant agent carry on and the scenario pass — or record. + await expect(runScenario( + { steps: [...boot, { op: 'prompt', text: 'impossible click' }], permissionAnswers: [{ kind: 'allow_always' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + )).rejects.toThrow(/allow_always not among the offered options \[allow_once, reject_once\]/) + }) +}) diff --git a/examples/acp-agent/tests/snapshot-normalize.spec.ts b/packages/support/acp-snapshot/tests/normalize.spec.ts similarity index 74% rename from examples/acp-agent/tests/snapshot-normalize.spec.ts rename to packages/support/acp-snapshot/tests/normalize.spec.ts index fa225bd659..8ebd1412b9 100644 --- a/examples/acp-agent/tests/snapshot-normalize.spec.ts +++ b/packages/support/acp-snapshot/tests/normalize.spec.ts @@ -1,9 +1,9 @@ import { describe, expect, it } from 'vitest' -import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from '../tests/snapshot-normalize.ts' +import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from '../src/normalize.ts' /** * Unit tests for the pure snapshot normalizers. Live as a *.spec.ts (runs in - * the default unit gate) and import the harness-side normalizers directly. + * the default unit gate) and import the normalizers directly. */ const ctx: NormalizeContext = { @@ -107,6 +107,17 @@ describe('normalizeSessionLog', () => { const out = normalizeSessionLog(`${header({})}\n${ev}\n`, ctx) expect(out).toContain('"durationMs":88') }) + + it('tolerates records missing the volatile fields it would zero', () => { + const bareHeader = JSON.stringify({ type: 'session', id: 's' }) + const timeless = JSON.stringify({ type: 'note', seq: 1 }) + const bareHook = JSON.stringify({ type: 'hook/result', seq: 2, time: 5, data: { decision: 'allow' } }) + const nullDataHook = JSON.stringify({ type: 'hook/result', seq: 3, time: 6, data: null }) + const out = normalizeSessionLog(`${bareHeader}\n${timeless}\n${bareHook}\n${nullDataHook}\n`, ctx) + expect(out).toContain('"type":"note","seq":1') + expect(out).toContain('"decision":"allow"') + expect(out).not.toContain('durationMs') + }) }) describe('scrubRequestHeaders', () => { @@ -135,6 +146,40 @@ describe('scrubRequestHeaders', () => { expect(out).not.toContain('{{tools}}') }) + it('scrubs a header carrying only one of system/tools, leaving the other absent', () => { + const systemOnly = scrubRequestHeaders(`${headerLine}\n${headerEvent({ system: 'secret prompt' })}\n`) + expect(systemOnly).toContain('"system":"{{system}}"') + expect(systemOnly).not.toContain('{{tools}}') + const toolsOnly = scrubRequestHeaders(`${headerLine}\n${headerEvent({ tools: [{ name: 't' }] })}\n`) + expect(toolsOnly).toContain('"tools":"{{tools}}"') + expect(toolsOnly).not.toContain('{{system}}') + }) + + it('leaves a delta with no scrubbable payload byte-identical (config-only, or non-array shapes)', () => { + const configOnly = JSON.stringify({ type: 'request/header-delta', seq: 8, time: 9, data: { config: { model: 'm2' } } }) + const oddShapes = JSON.stringify({ type: 'request/header-delta', seq: 9, time: 9, data: { system: { insert: 'not-an-array' }, tools: null } }) + const headerless = JSON.stringify({ type: 'request/header', seq: 10, time: 9, data: { reason: 'initial' } }) + const nullData = JSON.stringify({ type: 'request/header', seq: 11, time: 9, data: null }) + const raw = `${headerLine}\n${configOnly}\n${oddShapes}\n${headerless}\n${nullData}\n` + expect(scrubRequestHeaders(raw)).toBe(raw) + }) + + it('scrubs a one-sided tools delta and passes non-object schema entries through', () => { + const addedOnly = JSON.stringify({ + type: 'request/header-delta', seq: 8, time: 9, + data: { tools: { added: [null, 'weird', { name: 'x', description: 'D' }] } }, + }) + const out = scrubRequestHeaders(`${headerLine}\n${addedOnly}\n`) + // Non-object entries survive untouched; the object entry keeps only name. + expect(out).toContain('"added":[null,"weird",{"name":"x","description":"{{tools}}"}]') + const changedOnly = JSON.stringify({ + type: 'request/header-delta', seq: 8, time: 9, + data: { tools: { changed: [{ name: 'y', parameters: {} }] } }, + }) + expect(scrubRequestHeaders(`${headerLine}\n${changedOnly}\n`)) + .toContain('"changed":[{"name":"y","parameters":"{{tools}}"}]') + }) + it('scrubs a header-delta system payload but keeps its line positions and arity', () => { const delta = JSON.stringify({ type: 'request/header-delta', seq: 8, time: 9, diff --git a/packages/support/acp-snapshot/tests/suite.spec.ts b/packages/support/acp-snapshot/tests/suite.spec.ts new file mode 100644 index 0000000000..e14f525b30 --- /dev/null +++ b/packages/support/acp-snapshot/tests/suite.spec.ts @@ -0,0 +1,145 @@ +import { cpSync, mkdtempSync } from 'node:fs' +import { rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterAll, describe, expect, it } from 'vitest' +import { defineAcpSnapshotSuite, type Scenario } from '../src/index.ts' +import { childFixturePaths, fixtureContext, headerDeltaCount, normalizedHeaders } from '../src/suite.ts' + +/** + * Unit tests for the suite factory, by running it: two synthetic suites over + * the scripted fake ACP bin (./fixtures/fake-acp-agent.ts) register REAL + * describe/it trees at collection time, so every factory path — golden and log + * compares, the per-suite header pin and its uniformity guard, record-mode + * fixture write-back, skip semantics, and the fixture guard block — executes + * as an ordinary green test. The pure helpers get direct cases below. + * + * The replay suite runs against the committed fixtures in ./fixtures/suite. + * The record suite runs against a TEMP COPY of ./fixtures/record-suite + * (record mode writes session fixtures back into its snapshots dir; a run must + * never touch the committed tree). To re-bootstrap the record tree's goldens + * after changing the fake bin's output, run this spec once with + * `ACP_SNAPSHOT_SPEC_BOOTSTRAP=1` (points the record suite at the committed + * tree so vitest creates/updates the goldens and the write-back lands there), + * then commit the result. + */ + +const AGENT = { + binScript: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)), + configPath: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)), + tsconfigPath: fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)), +} + +const REPLAY_DIR = fileURLToPath(new URL('./fixtures/suite', import.meta.url)) +const RECORD_SRC = fileURLToPath(new URL('./fixtures/record-suite', import.meta.url)) + +const REPLAY_SCENARIOS: Scenario[] = [ + { name: 'pin-turn', hasModelTurn: true, recorded: true, pinsHeader: true }, + { name: 'plain-turn', hasModelTurn: true, recorded: true, childSessions: 1 }, + { name: 'no-model', hasModelTurn: false, recorded: false }, + { name: 'blocked-log', hasModelTurn: false, comparesLog: true, recorded: false }, + { name: 'authored-error', hasModelTurn: true, recorded: false, overridden: true }, +] + +const RECORD_SCENARIOS: Scenario[] = [ + { name: 'rec-pin', hasModelTurn: true, recorded: true, pinsHeader: true }, + { name: 'rec-child', hasModelTurn: true, recorded: true, childSessions: 1 }, + // recorded:false in record mode → registered but skipped (never re-recorded). + { name: 'rec-skip', hasModelTurn: true, recorded: false, overridden: true }, +] + +// Record mode mutates its snapshots dir, so run it on a throwaway copy — +// except under the documented bootstrap knob, which regenerates the committed +// fixtures/goldens in place. +const BOOTSTRAP = process.env.ACP_SNAPSHOT_SPEC_BOOTSTRAP === '1' +const recordDir = BOOTSTRAP ? RECORD_SRC : mkdtempSync(join(tmpdir(), 'acp-snap-record-suite-')) +if (!BOOTSTRAP) cpSync(RECORD_SRC, recordDir, { recursive: true }) +afterAll(async () => { + if (!BOOTSTRAP) await rm(recordDir, { recursive: true, force: true }) +}) + +describe('defineAcpSnapshotSuite: replay mode', () => { + defineAcpSnapshotSuite({ agent: AGENT, snapshotsDir: REPLAY_DIR, scenarios: REPLAY_SCENARIOS, mode: 'replay' }) +}) + +// The record suite's tests run in registration order: rec-pin re-records the +// pinned fixture FIRST, so rec-child's uniformity guard reads the fresh pin. +describe('defineAcpSnapshotSuite: record mode', () => { + defineAcpSnapshotSuite({ agent: AGENT, snapshotsDir: recordDir, scenarios: RECORD_SCENARIOS, mode: 'record' }) +}) + +describe('defineAcpSnapshotSuite: registration contract', () => { + it('throws when no scenario pins the request-header content', () => { + expect(() => { + defineAcpSnapshotSuite({ + agent: AGENT, + snapshotsDir: REPLAY_DIR, + scenarios: [{ name: 'pinless', hasModelTurn: true, recorded: true }], + mode: 'replay', + }) + }).toThrow(/no scenario pins/) + }) +}) + +describe('childFixturePaths', () => { + it('yields one sibling path per child, 1-based', () => { + expect(childFixturePaths('/snap/s', 2)).toEqual(['/snap/s/session.1.jsonl', '/snap/s/session.2.jsonl']) + }) + + it('yields nothing for a single-session scenario', () => { + expect(childFixturePaths('/snap/s', 0)).toEqual([]) + }) +}) + +describe('fixtureContext', () => { + it('reads the fixture header id and cwd', () => { + const ctx = fixtureContext('{"type":"session","id":"abc","cwd":"/rec"}\n{"type":"turn/start"}\n') + expect(ctx).toEqual({ sessionIds: ['abc'], cwd: '/rec' }) + }) + + it('yields no session ids for a header without a string id', () => { + expect(fixtureContext('{"type":"session","cwd":"/rec"}\n').sessionIds).toEqual([]) + }) + + it('falls back to an impossible sentinel cwd (never the empty string)', () => { + const ctx = fixtureContext('{"type":"session","id":"abc"}\n') + expect(ctx.cwd).toBe('\0no-cwd\0') + expect(ctx.cwd).not.toBe('') + }) + + it('treats an empty fixture as an empty header', () => { + expect(fixtureContext('')).toEqual({ sessionIds: [], cwd: '\0no-cwd\0' }) + }) +}) + +describe('normalizedHeaders', () => { + const header = (system: string): string => JSON.stringify({ + type: 'request/header', seq: 0, time: 9, data: { header: { config: { model: 'm' }, system }, reason: 'initial' }, + }) + + it('extracts every request/header payload in log order, normalized', () => { + const id = '11111111-2222-4333-8444-555555555555' + const log = `${JSON.stringify({ type: 'session', id, createdAt: 5, cwd: '/w' })}\n${header('one')}\n` + + `${JSON.stringify({ type: 'turn/start', seq: 1, time: 9, data: { turn: 1 } })}\n${header('two')}\n` + const headers = normalizedHeaders(log, { sessionIds: [id], cwd: '/w' }) + expect(headers).toEqual([ + { config: { model: 'm' }, system: 'one' }, + { config: { model: 'm' }, system: 'two' }, + ]) + }) + + it('yields nothing for a log without header events', () => { + const log = `${JSON.stringify({ type: 'session', id: 'a', createdAt: 5 })}\n` + expect(normalizedHeaders(log, { sessionIds: [], cwd: '/w' })).toEqual([]) + }) +}) + +describe('headerDeltaCount', () => { + it('counts request/header-delta events, ignoring blanks and other lines', () => { + const delta = JSON.stringify({ type: 'request/header-delta', seq: 2, time: 9, data: {} }) + const other = JSON.stringify({ type: 'request/header', seq: 0, time: 9, data: {} }) + expect(headerDeltaCount(`${other}\n\n${delta}\n${delta}\n`)).toBe(2) + expect(headerDeltaCount(`${other}\n`)).toBe(0) + }) +}) diff --git a/packages/support/acp-snapshot/tsconfig.json b/packages/support/acp-snapshot/tsconfig.json new file mode 100644 index 0000000000..749cb0208e --- /dev/null +++ b/packages/support/acp-snapshot/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [] +} diff --git a/packages/support/llm-replay/src/index.ts b/packages/support/llm-replay/src/index.ts index 3cca0a8291..94bd0cdd7e 100644 --- a/packages/support/llm-replay/src/index.ts +++ b/packages/support/llm-replay/src/index.ts @@ -128,6 +128,8 @@ export interface SessionScript { * Parse a session `.jsonl` buffer into its event list. Line 0 is the session * header (a `{type:'session',…}` record), every subsequent non-empty line is a * {@link SessionEvent}. The header is skipped; malformed lines fail loud. + * @param text - the raw `.jsonl` file contents. + * @returns every event after the header, in log order. */ export function parseSessionLog(text: string): SessionEvent[] { const lines = text.split('\n').filter(line => line.trim().length > 0) @@ -151,6 +153,8 @@ export function parseSessionLog(text: string): SessionEvent[] { * own model calls; absent ⇒ 0). A header missing a field falls back to a stable * default (`''` / `0` / `0`) rather than throwing: a no-model fixture is * header-only and still orders fine as the single (primary) script. + * @param text - the raw `.jsonl` file contents (only the header line is read). + * @returns the header's `id`, `createdAt`, and `seedLength`, defaulted when absent. */ export function parseSessionHeader(text: string): { id: string; createdAt: number; seedLength: number } { const firstLine = text.split('\n').find(line => line.trim().length > 0) ?? '{}' @@ -180,6 +184,8 @@ export function parseSessionHeader(text: string): { id: string; createdAt: numbe * sidecar with an explicit `throw` (or `hang`) entry instead. {@link * deriveReplayScript} throws, naming the offending `(turn, step)`, so a missing * override fails loud rather than silently replaying a thrown call as success. + * @param events - the recorded session's events; only `assistant/chunk` is consulted. + * @returns one `chunks` entry per recorded model call, in call order. */ export function deriveReplayScript(events: SessionEvent[]): ReplayEntry[] { const script: ReplayEntry[] = [] @@ -218,6 +224,8 @@ export function deriveReplayScript(events: SessionEvent[]): ReplayEntry[] { * Fail-loud if the JSONL fixture is missing (the scenario was never recorded) — * never silently returns an empty script, so a coverage hole can't masquerade * as a passing replay. + * @param config - the fixture paths; only `file` and `overrideFile` are consulted. + * @returns the primary session's replay entries. */ export function loadReplayScript(config: ReplayConfig): ReplayEntry[] { if (config.overrideFile !== undefined && existsSync(config.overrideFile)) { @@ -245,6 +253,8 @@ export function loadReplayScript(config: ReplayConfig): ReplayEntry[] { * the parent issues the FIRST model call (it must stream before it can delegate * in the synchronous nested cut), so binding it to the first live session is * correct regardless of a timestamp tie. + * @param config - the fixture paths: the primary log plus any recorded child logs. + * @returns the primary script first, then the child scripts in bind order. */ export function loadSessionScripts(config: ReplayConfig): SessionScript[] { const primaryEntries = loadReplayScript(config) @@ -359,6 +369,9 @@ async function* replayEntry(entry: ReplayEntry, signal: AbortSignal | undefined) * Each per-session cursor advances synchronously at listener-invocation time * (not lazily inside the generator) so call ORDER within a session, not * iteration order, fixes the mapping. + * @param ctx - the context whose `llm/stream` waterfall the listener short-circuits. + * @param config - the resolved fixture paths (env-var defaulting is `apply`'s job). + * @returns the `ctx.on` disposer that removes the listener. */ export function installLlmReplay(ctx: Context, config: ReplayConfig): () => void { const scripts = loadSessionScripts(config) @@ -412,6 +425,7 @@ export function installLlmReplay(ctx: Context, config: ReplayConfig): () => void export const name = 'llm-replay' export const inject = ['llm'] +/** Plugin config: the {@link ReplayConfig} inputs, each defaulting to its `DSH_SNAPSHOT_*` env var in `apply`. */ export interface Config { /** Override the fixture path; defaults to `$DSH_SNAPSHOT_FILE`. */ file?: string diff --git a/packages/support/subagent-mock/tests/subagent-mock.spec.ts b/packages/support/subagent-mock/tests/subagent-mock.spec.ts index f35ed884eb..ddd725da4b 100644 --- a/packages/support/subagent-mock/tests/subagent-mock.spec.ts +++ b/packages/support/subagent-mock/tests/subagent-mock.spec.ts @@ -41,13 +41,13 @@ describe('dsh-subagent-mock', () => { it('surfaces a structured result when the request carries an outputSchema', async () => { const ctx = await mount({ reply: 'r', structured: { answer: 42 } }) - const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { answer: { type: 'number' } } })) + const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { type: 'object', properties: { answer: { type: 'number' } } } })) await expect(run.result).resolves.toMatchObject({ structured: { answer: 42 } }) }) it('defaults structured output to { reply } when outputSchema is requested but no structured value is configured', async () => { const ctx = await mount({ reply: 'fallback reply' }) - const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { answer: { type: 'number' } } })) + const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { type: 'object', properties: { answer: { type: 'number' } } } })) await expect(run.result).resolves.toMatchObject({ structured: { reply: 'fallback reply' } }) }) diff --git a/packages/timeout/README.md b/packages/timeout/README.md new file mode 100644 index 0000000000..36f52abaf3 --- /dev/null +++ b/packages/timeout/README.md @@ -0,0 +1,9 @@ +# timeout/ — tool-call timeout policy + +The tool-call timeout policy plugin. A single **product** package: it is a deployment-policy consumer of the `tools/execute` around-dispatch seam (owned by [`dsh-tools`](../core/tools)) and the pure [`dsh-timeout`](../util/timeout) library — not a swappable capability with an interface/implementation split, so it needs no seam trio. + +| Package | Role | ctx key | +|---|---|---| +| `timeout-policy/` | A `tools/execute` wrapper: for each configured tool it arms a per-call deadline on `exec.signal` and returns a structured `TOOL_TIMEOUT` result when that deadline wins | (registers a `tools/execute` listener; injects nothing) | + +Timeout is split across three layers: [`dsh-timeout`](../util/timeout) owns the pure timing/classification primitive (`deadline`/`timeoutOf`), each capability owns termination (bash kills its process group, the fetch provider tears down its socket), and this package owns the *model-facing tool-call budget as deployment policy* — no model-facing timeout argument, no global default. It is the middleware the [timeout-library RFC](../../docs/rfc/implemented/architecture/2026-07-06-timeout-deadline-library.md) foresaw. `bash` and hook command execution keep their own `BASH_TIMEOUT` backend timeout and do not route through this policy. diff --git a/packages/timeout/timeout-policy/README.md b/packages/timeout/timeout-policy/README.md new file mode 100644 index 0000000000..e637a658bf --- /dev/null +++ b/packages/timeout/timeout-policy/README.md @@ -0,0 +1,34 @@ +# dsh-timeout-policy + +Tool-call timeout enforcer: a single `tools/execute` around-dispatch listener that arms a per-call cooperative deadline on `exec.signal` for a tool declaring `timeoutMs` on its `ToolDefinition` and returns a structured `TOOL_TIMEOUT` result when that deadline wins. The budget is read from the tool's own declaration (`ToolDefinition.timeoutMs`, set by the owning tool plugin), so this plugin is **zero-config**. It is the reference `tools/execute` wrapper and the enforcement home for model-facing tool-call budgets (the timeout-library RFC's foreseen middleware). + +## Plugin (namespace: `timeout-policy`) + +A function/namespace plugin (`name` / `inject` / `apply`), not a service. It registers no tool and takes no config — it consumes `ctx.tools`'s `tools/execute` waterfall (which the `dsh-tools` registry always provides) and reads each dispatched tool's declared `timeoutMs` from the registry (`ctx.tools.get(exec.name)`). + +```yaml +- id: timeout-policy + name: '@deepseek-ai/dsh-timeout-policy' +``` + +The per-tool budget is declared by the tool plugin (e.g. `dsh-tool-web`'s `fetchTimeoutMs`/`searchTimeoutMs` config, attached as `ToolDefinition.timeoutMs`); this plugin only enforces it, so a mistyped tool name is not possible. + +### Behavior + +For a tool that **declares a `timeoutMs`** the listener: + +1. Reads the budget from the tool's own declaration in the registry (`ctx.tools.get(exec.name)?.timeoutMs`) and arms `deadline(exec.signal, timeoutMs, 'TOOL_TIMEOUT')` — one signal fusing the caller's abort with this plugin's timer (`@deepseek-ai/dsh-timeout`). +2. Swaps that derived signal onto `exec` for the downstream dispatch, then restores the caller's own signal afterward (cordis `next()` ignores passed arguments, so the wrapper mutates the shared `exec` in place; restoring keeps `tools/post-execute` seeing the caller's signal). +3. After dispatch, if `timeoutOf(d.signal, 'TOOL_TIMEOUT')` matches — this plugin's own timer fired — replaces the result with a structured `TOOL_TIMEOUT` tool result: `{ isError: true, error: { name: 'ToolTimeoutError', code: 'TOOL_TIMEOUT' }, content: 'Error: tool call timed out after ms' }`. + +A tool that **declares no budget** delegates untouched (no deadline). + +The base `next()` of `tools/execute` is the registry's dispatch-with-normalization thunk, so when the timeout signal reaches a provider that throws its own upstream-abort error, dispatch first turns it into a normal error result, and this wrapper then replaces that with `TOOL_TIMEOUT`. That ordering is why the replacement is keyed off the signal (`timeoutOf`), not off the dispatched result's shape. + +### Cooperative, not a hard kill + +The derived signal only **notifies**; termination stays with the tool and the capability it forwards `exec.signal` to (the `dsh-timeout` library owns no kill). **Declaring `timeoutMs` therefore means "cooperative with `exec.signal`"**: a tool that ignores the signal will not stop on timeout. Only signal-forwarding tools should declare it — the shipped `web_fetch`/`web_search` (which forward through `ctx.web` to providers) are the reference. `TOOL_TIMEOUT` needs no session event for reconstructability: it is the final model-facing `tool/result`, already logged by the loop. + +### Composing with other `tools/execute` wrappers + +Multiple `tools/execute` listeners compose by cordis registration order. Combined with a future retry/sandbox/metrics wrapper, registration order chooses the semantics — "timeout covers the whole retry operation" (timeout registered outer) versus "timeout covers each attempt" (timeout registered inner). diff --git a/packages/timeout/timeout-policy/package.json b/packages/timeout/timeout-policy/package.json new file mode 100644 index 0000000000..9069735b86 --- /dev/null +++ b/packages/timeout/timeout-policy/package.json @@ -0,0 +1,36 @@ +{ + "name": "@deepseek-ai/dsh-timeout-policy", + "description": "Tool-call timeout policy: a tools/execute wrapper that arms a per-tool deadline on exec.signal and returns TOOL_TIMEOUT when it wins", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", + "@deepseek-ai/dsh-tools": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "@deepseek-ai/dsh-tools": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/timeout/timeout-policy/src/index.ts b/packages/timeout/timeout-policy/src/index.ts new file mode 100644 index 0000000000..319de676a9 --- /dev/null +++ b/packages/timeout/timeout-policy/src/index.ts @@ -0,0 +1,115 @@ +/** + * `@deepseek-ai/dsh-timeout-policy`: the tool-call timeout ENFORCER. It registers + * ONE `tools/execute` around-dispatch listener that, for a tool declaring a + * `timeoutMs` on its {@link ToolDefinition}, arms a per-call deadline on + * `exec.signal` and returns a structured `TOOL_TIMEOUT` result when that deadline + * wins. The budget is DECLARED by the tool (see `ToolDefinition.timeoutMs`, set + * by the owning tool plugin from its own config); this plugin only enforces it, + * so it is zero-config and there is no tool-name map to mistype. + * + * This is a COOPERATIVE deadline, not a hard kill: the derived signal only + * NOTIFIES. A tool that declares `timeoutMs` (and the capability it forwards + * `exec.signal` to) must honor that signal and reach quiescence — the plugin + * never races the tool promise or terminates work itself (see the timeout-library + * RFC's rejection of `Promise.race`). Declaring `timeoutMs` therefore MEANS "this + * tool is cooperative with `exec.signal`": a tool that ignores the signal will + * not stop on timeout, so only signal-forwarding tools should declare it (the + * shipped web tools are the reference). + * + * Ownership of the `TOOL_TIMEOUT` code is entirely here: it is both the internal + * {@link deadline} code (so {@link timeoutOf} scopes the classification to THIS + * plugin's own timer, reading a foreign/nested outer deadline as an ordinary + * cancel) and the structured `{ name, code }` on the replacement tool result. + * No new session event is needed for reconstructability: the `TOOL_TIMEOUT` + * result IS the final model-facing `tool/result`, already logged by the loop. + * + * Why a `tools/execute` around seam and not a `pre`/`post` pair: the deadline + * needs ONE lexical scope — arm on `exec.signal`, delegate to dispatch, classify + * the result, dispose the timer — which the around seam gives directly. A + * pre/post split would spread one deadline's lifetime across two independent + * waterfalls (a call-id map, cleanup on every deny/throw/dispose path). + * + * @module @deepseek-ai/dsh-timeout-policy + */ + +import type { Context } from 'cordis' +import type { CallId } from '@deepseek-ai/dsh-llm' +import { deadline, timeoutOf } from '@deepseek-ai/dsh-timeout' +import type { ToolExecutionResult } from '@deepseek-ai/dsh-tools' + +/** + * The code owned by this plugin, used BOTH as the internal {@link deadline} + * classification code AND as the structured error `code` on the replacement + * tool result. Scoping {@link timeoutOf} to it keeps a nested outer deadline + * (another `tools/execute` wrapper's timer that fired first) from being misread + * as this plugin's own timeout — it reads as an ordinary upstream cancel. + */ +export const TOOL_TIMEOUT = 'TOOL_TIMEOUT' + +/** Cordis plugin name used by loader diagnostics. */ +export const name = 'timeout-policy' + +/** The tool registry seam this plugin wraps (`tools/execute`) and reads (`get`). */ +export const inject = ['tools'] + +/** + * The structured result substituted when this plugin's deadline wins. `content` + * is the model-facing message; `error.code` is the same {@link TOOL_TIMEOUT} + * this plugin owns, so a retry/sandbox plugin (and replay) can route on it. + * + * @param callId - the timed-out call's id, carried onto the replacement result. + * @param timeoutMs - the elapsed budget, rendered into the model-facing message. + * @returns the `isError` {@link ToolExecutionResult} with a `TOOL_TIMEOUT` error. + */ +export function toolTimeoutResult(callId: CallId, timeoutMs: number): ToolExecutionResult { + return { + callId, + content: [{ type: 'text', text: `Error: tool call timed out after ${timeoutMs}ms` }], + isError: true, + error: { name: 'ToolTimeoutError', code: TOOL_TIMEOUT }, + } +} + +/** + * Register the tool-call timeout enforcer. For a tool whose {@link ToolDefinition} + * declares `timeoutMs`, the listener arms a {@link deadline} on the caller's + * `exec.signal`, swaps it onto `exec` for the downstream dispatch (cordis + * `next()` ignores passed arguments, so a wrapper mutates the shared `exec` in + * place), restores the original signal afterward so `tools/post-execute` sees the + * caller's own signal, and replaces the result with {@link toolTimeoutResult} + * when its own timer fired. A tool that declares no budget delegates untouched. + * + * The budget source is the tool's own declaration read from the registry + * (`ctx.tools.get(exec.name)?.timeoutMs`), NOT a plugin config map — `exec.name` + * is the tool being dispatched, so the lookup always resolves and there is no + * mistypable tool name and no unknown-name path to warn or throw about. + */ +export function apply(ctx: Context): void { + ctx.on('tools/execute', async (exec, next): Promise => { + const timeoutMs = ctx.tools.get(exec.name)?.timeoutMs + // A tool that declares no budget: no deadline, delegate unchanged. + if (timeoutMs === undefined) return next() + + using d = deadline(exec.signal, timeoutMs, TOOL_TIMEOUT) + // Swap the derived deadline onto exec for dispatch, then restore the + // caller's own signal so post-execute listeners never see this plugin's + // (possibly already-aborted) timeout signal. `undefined` is not assignable to + // the optional `signal` under exactOptionalPropertyTypes, so branch on it. + const upstream = exec.signal + exec.signal = d.signal + try { + const result = await next() + // If OUR timer fired (scoped by code — a nested outer deadline reads as + // undefined here), the tool/capability saw the abort and reached + // quiescence; replace whatever it returned (its own abort result) with the + // structured TOOL_TIMEOUT the model sees. + if (timeoutOf(d.signal, TOOL_TIMEOUT) !== undefined) { + return toolTimeoutResult(exec.callId, timeoutMs) + } + return result + } finally { + if (upstream === undefined) delete exec.signal + else exec.signal = upstream + } + }) +} diff --git a/packages/timeout/timeout-policy/tests/timeout-policy.spec.ts b/packages/timeout/timeout-policy/tests/timeout-policy.spec.ts new file mode 100644 index 0000000000..ef5c52030f --- /dev/null +++ b/packages/timeout/timeout-policy/tests/timeout-policy.spec.ts @@ -0,0 +1,200 @@ +/** + * Unit + real-load-path coverage for @deepseek-ai/dsh-timeout-policy. The + * timeout-wins cases drive the deadline under fake timers (deterministic — no + * wall-clock race) and use a COOPERATIVE tool that settles only when its + * `exec.signal` aborts, mirroring how a real capability forwards the signal and + * reaches quiescence. + */ + +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' +import { CallId, HarnessError } from '@deepseek-ai/dsh-llm' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry, { defineTool, type ToolExecution, type ToolExecutionResult, type PostToolDecision } from '@deepseek-ai/dsh-tools' +import * as timeoutPolicy from '@deepseek-ai/dsh-timeout-policy' +import { TOOL_TIMEOUT, toolTimeoutResult } from '@deepseek-ai/dsh-timeout-policy' + +/** Mount the registry + the zero-config timeout-policy enforcer. */ +async function setup() { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(timeoutPolicy) + return ctx +} + +/** A cooperative tool that settles ONLY when its exec.signal aborts (returns text). */ +const cooperativeTool = defineTool({ + name: 'slow', description: 'stops when aborted', parameters: {}, timeoutMs: 100, + execute(_args, exec): Promise<{ type: 'text'; text: string }[]> { + const done = [{ type: 'text' as const, text: 'stopped cooperatively' }] + if (exec.signal?.aborted) return Promise.resolve(done) + return new Promise((resolve) => { exec.signal?.addEventListener('abort', () => { resolve(done) }) }) + }, +}) + +/** A cooperative tool that THROWS its own upstream-abort error when aborted (web-provider shape). */ +const abortThrowingTool = defineTool({ + name: 'aborter', description: 'throws WEB_ABORTED when aborted', parameters: {}, timeoutMs: 100, + execute(_args, exec): Promise { + if (exec.signal?.aborted) return Promise.reject(new HarnessError('web fetch aborted', 'WEB_ABORTED')) + return new Promise((_resolve, reject) => { exec.signal?.addEventListener('abort', () => { reject(new HarnessError('web fetch aborted', 'WEB_ABORTED')) }) }) + }, +}) + +describe('timeout-policy delegation (unconfigured / fast)', () => { + it('delegates a tool with NO declared budget unchanged and does not touch exec.signal', async () => { + const ctx = await setup() + let seenSignal: AbortSignal | undefined + ctx.tools.register(defineTool({ name: 'probe', description: 'd', parameters: {}, + async execute(_a, exec) { seenSignal = exec.signal; return [{ type: 'text' as const, text: 'ok' }] } })) + const upstream = new AbortController().signal + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'probe', arguments: {}, signal: upstream }) + expect(result.isError).toBe(false) + expect(seenSignal).toBe(upstream) + }) + + it('a tool with a budget that returns fast keeps its own result (no timeout)', async () => { + const ctx = await setup() + ctx.tools.register(defineTool({ name: 'fast', description: 'd', parameters: {}, timeoutMs: 10_000, + async execute() { return [{ type: 'text' as const, text: 'ok' }] } })) + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'fast', arguments: {} }) + expect(result).toEqual({ callId: CallId('c1'), content: [{ type: 'text', text: 'ok' }], isError: false }) + }) + + it('a budgeted tool receives the DERIVED deadline signal (not the caller signal) during dispatch', async () => { + const ctx = await setup() + let seenSignal: AbortSignal | undefined + ctx.tools.register(defineTool({ name: 'probe', description: 'd', parameters: {}, timeoutMs: 10_000, + async execute(_a, exec) { seenSignal = exec.signal; return [{ type: 'text' as const, text: 'ok' }] } })) + const upstream = new AbortController().signal + await ctx.tools.execute({ callId: CallId('c1'), name: 'probe', arguments: {}, signal: upstream }) + expect(seenSignal).toBeDefined() + expect(seenSignal).not.toBe(upstream) + }) +}) + +describe('timeout-policy signal restoration', () => { + it('restores the caller signal for post-execute after wrapping', async () => { + const ctx = await setup() + ctx.tools.register(defineTool({ name: 'fast', description: 'd', parameters: {}, timeoutMs: 10_000, + async execute() { return [{ type: 'text' as const, text: 'ok' }] } })) + let postSignal: AbortSignal | undefined | 'unset' = 'unset' + ctx.on('tools/post-execute', async (exec, _result, next): Promise => { postSignal = exec.signal; return next() }) + const upstream = new AbortController().signal + await ctx.tools.execute({ callId: CallId('c1'), name: 'fast', arguments: {}, signal: upstream }) + expect(postSignal).toBe(upstream) + }) + + it('deletes exec.signal again when the caller passed none', async () => { + const ctx = await setup() + ctx.tools.register(defineTool({ name: 'fast', description: 'd', parameters: {}, timeoutMs: 10_000, + async execute() { return [{ type: 'text' as const, text: 'ok' }] } })) + let hadSignal: boolean | undefined + ctx.on('tools/post-execute', async (exec, _result, next): Promise => { hadSignal = 'signal' in exec && exec.signal !== undefined; return next() }) + await ctx.tools.execute({ callId: CallId('c1'), name: 'fast', arguments: {} }) + expect(hadSignal).toBe(false) + }) +}) + +describe('timeout-policy TOOL_TIMEOUT replacement (deadline wins)', () => { + beforeEach(() => { vi.useFakeTimers() }) + afterEach(() => { vi.useRealTimers() }) + + it('replaces a cooperative tool result with TOOL_TIMEOUT when its own deadline fires', async () => { + const ctx = await setup() + ctx.tools.register(cooperativeTool) + const pending = ctx.tools.execute({ callId: CallId('c1'), name: 'slow', arguments: {} }) + await vi.advanceTimersByTimeAsync(150) + const result = await pending + expect(result).toEqual({ + callId: CallId('c1'), + content: [{ type: 'text', text: 'Error: tool call timed out after 100ms' }], + isError: true, + error: { name: 'ToolTimeoutError', code: 'TOOL_TIMEOUT' }, + }) + }) + + it('replaces a provider-owned abort ERROR result with TOOL_TIMEOUT when the signal was ours', async () => { + const ctx = await setup() + ctx.tools.register(abortThrowingTool) + const pending = ctx.tools.execute({ callId: CallId('c1'), name: 'aborter', arguments: {} }) + await vi.advanceTimersByTimeAsync(150) + const result = await pending + expect(result.isError).toBe(true) + expect(result.error).toEqual({ name: 'ToolTimeoutError', code: 'TOOL_TIMEOUT' }) + expect(result.content[0]).toMatchObject({ text: 'Error: tool call timed out after 100ms' }) + }) + + it('does NOT replace when the caller aborts first (upstream cancel, not our timeout)', async () => { + const ctx = await setup() + ctx.tools.register(cooperativeTool) + const upstream = new AbortController() + const pending = ctx.tools.execute({ callId: CallId('c1'), name: 'slow', arguments: {}, signal: upstream.signal }) + upstream.abort('user cancelled') + await vi.advanceTimersByTimeAsync(0) + const result = await pending + expect(result.isError).toBe(false) + expect(result.content[0]).toMatchObject({ text: 'stopped cooperatively' }) + }) +}) + +describe('toolTimeoutResult', () => { + it('builds the structured TOOL_TIMEOUT result', () => { + expect(toolTimeoutResult(CallId('c9'), 250)).toEqual({ + callId: CallId('c9'), + content: [{ type: 'text', text: 'Error: tool call timed out after 250ms' }], + isError: true, + error: { name: 'ToolTimeoutError', code: 'TOOL_TIMEOUT' }, + } satisfies ToolExecutionResult) + }) + + it('exposes the owned code constant', () => { + expect(TOOL_TIMEOUT).toBe('TOOL_TIMEOUT') + }) +}) + +describe('timeout-policy disposal (HMR safety)', () => { + it('removes its tools/execute listener when the plugin fiber disposes', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + let seenSignal: AbortSignal | undefined + ctx.tools.register(defineTool({ name: 'probe', description: 'd', parameters: {}, timeoutMs: 10_000, + async execute(_a, exec) { seenSignal = exec.signal; return [{ type: 'text' as const, text: 'ok' }] } })) + const fiber = await ctx.plugin(timeoutPolicy) + const upstream = new AbortController().signal + await ctx.tools.execute({ callId: CallId('c1'), name: 'probe', arguments: {}, signal: upstream }) + expect(seenSignal).not.toBe(upstream) + await fiber.dispose() + await ctx.tools.execute({ callId: CallId('c2'), name: 'probe', arguments: {}, signal: upstream }) + expect(seenSignal).toBe(upstream) + }) +}) + +describe('dsh-timeout-policy real-load-path guard', () => { + it('has no default export and keeps name/inject through unwrapExports', () => { + expect('default' in timeoutPolicy).toBe(false) + const loader = Object.create(Loader.prototype) as Loader + const unwrapped = loader.unwrapExports(timeoutPolicy) as Record + expect(unwrapped).toBe(timeoutPolicy) + expect(unwrapped.name).toBe('timeout-policy') + expect(unwrapped.inject).toEqual(['tools']) + expect(typeof unwrapped.apply).toBe('function') + }) + + it('boots over ctx.tools through the unwrapped module and wraps a budgeted tool', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + ctx.tools.register(defineTool({ name: 'fast', description: 'd', parameters: {}, timeoutMs: 5_000, + async execute() { return [{ type: 'text' as const, text: 'ok' }] } })) + const loader = Object.create(Loader.prototype) as Loader + const unwrapped = loader.unwrapExports(timeoutPolicy) as Parameters[0] + const fiber = await ctx.plugin(unwrapped) + const result = await ctx.tools.execute({ callId: CallId('c1'), name: 'fast', arguments: {} } satisfies ToolExecution) + expect(result.isError).toBe(false) + await fiber.dispose() + }) +}) diff --git a/packages/timeout/timeout-policy/tsconfig.json b/packages/timeout/timeout-policy/tsconfig.json new file mode 100644 index 0000000000..8c0b47716e --- /dev/null +++ b/packages/timeout/timeout-policy/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { "path": "../../../vendor/cosmokit" }, + { "path": "../../../vendor/cordis" }, + { "path": "../../../vendor/schemastery" }, + { "path": "../../llm/llm" }, + { "path": "../../util/timeout" }, + { "path": "../../core/tools" } + ] +} diff --git a/packages/ui/acp-agent/README.md b/packages/ui/acp-agent/README.md index 4a114619a6..36e794f92c 100644 --- a/packages/ui/acp-agent/README.md +++ b/packages/ui/acp-agent/README.md @@ -24,6 +24,7 @@ Because the package wires no logger entry, an ACP leaf has **nothing to get wron |---|---|---| | `model` | (required) | the per-session agent template the bridge creates agents from | | `persona` | — | the deployment persona template (may reference `{{model}}`/`{{cwd}}`), routed to `dsh-system-prompt` | +| `toolOrder` | — | explicit model-facing tool order (a name list with one `''` rest entry; absent — lexicographic; an unregistered name fails each turn at prompt assembly), routed to `dsh-system-prompt` | | `persistenceRoot` | `./.sessions` | the JSONL backend's root directory | The leaf supplies the swappable backends: an LLM adapter (`llm-deepseek` for the real model, `llm-replay` for keyless snapshot replay) and a bash executor (`bash-local`). diff --git a/packages/ui/acp-agent/package.json b/packages/ui/acp-agent/package.json index 71f8fe0171..e8b4f3723b 100644 --- a/packages/ui/acp-agent/package.json +++ b/packages/ui/acp-agent/package.json @@ -45,6 +45,7 @@ "@deepseek-ai/dsh-app-boot": "workspace:^", "@deepseek-ai/dsh-acp": "workspace:^", "@deepseek-ai/dsh-agent-core": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", "cordis": "^4.0.0-rc.6", "schemastery": "^3.17.0" diff --git a/packages/ui/acp-agent/src/index.ts b/packages/ui/acp-agent/src/index.ts index 3bd498dc63..898ec9a509 100644 --- a/packages/ui/acp-agent/src/index.ts +++ b/packages/ui/acp-agent/src/index.ts @@ -42,7 +42,8 @@ export const name = 'acp-agent' * App config: the swappable per-deployment values. `model` configures the * agent template the ACP bridge creates each session's agent from (NOT a * pre-created agent — ACP creates agents at `session/new`); `persona` is the - * deployment persona (forwarded to the system-prompt plugin); + * deployment persona (forwarded to the system-prompt plugin); `toolOrder` is + * the explicit model-facing tool order (forwarded to the system-prompt plugin); * `persistenceRoot` is the JSONL backend's directory. */ export interface Config { @@ -50,6 +51,8 @@ export interface Config { model: string /** Deployment persona (the system-prompt plugin's `persona` config). */ persona?: string + /** Explicit model-facing tool order (the system-prompt plugin's `toolOrder` config; see dsh-system-prompt). */ + toolOrder?: string[] /** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */ persistenceRoot?: string } @@ -57,6 +60,10 @@ export interface Config { export const Config: z = z.object({ model: z.string().required(), persona: z.string(), + // The array default is forced to undefined: ABSENT means "lexicographic + // order" (the owning dsh-system-prompt schema does the same), while + // schemastery's native [] default would read as an invalid configured list. + toolOrder: z.array(z.string()).default(undefined as unknown as string[]), persistenceRoot: z.string().default('./.sessions'), }) @@ -70,6 +77,7 @@ export const Config: z = z.object({ export function apply(ctx: Context, config: Config): void { ctx.plugin(agentCore, { ...config.persona !== undefined ? { persona: config.persona } : {}, + ...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {}, }) ctx.plugin(SessionPersistenceJsonl, { root: config.persistenceRoot ?? './.sessions' }) ctx.plugin(acp, { model: config.model }) diff --git a/packages/ui/acp-agent/tests/acp-agent.spec.ts b/packages/ui/acp-agent/tests/acp-agent.spec.ts index 87cf670107..7e1de1226f 100644 --- a/packages/ui/acp-agent/tests/acp-agent.spec.ts +++ b/packages/ui/acp-agent/tests/acp-agent.spec.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' +import { TOOL_ORDER_REST } from '@deepseek-ai/dsh-system-prompt' import * as acpAgent from '../src/index.ts' /** @@ -52,6 +53,27 @@ describe('dsh-acp-agent composition', () => { expect(acpAgent.Config).toBeDefined() }) + it('forwards toolOrder through agent-core to the system-prompt assembly', async () => { + const ctx = await mount({ + model: 'mock', + toolOrder: ['zulu', TOOL_ORDER_REST], + persistenceRoot: '/tmp/dsh-acp-agent-test-tool-order', + }) + // The bundle's own bash tools pend on the absent `ctx.bash` executor in + // this providerless mount, so register two plain tools to order. + for (const name of ['alpha', 'zulu']) { + ctx.get('tools')!.register({ + name, + description: name, + parameters: {}, + execute: async () => [], + }) + } + const assembly = await ctx.get('systemPrompt')!.assemble() + expect(assembly.tools.map(tool => tool.name)).toEqual(['zulu', 'alpha']) + await ctx.fiber.dispose() + }) + it('has the namespace-plugin export shape (no stray default) so the Loader keeps name/Config/apply', () => { // Postmortem 0001 guard: a stray `export default apply` makes the Loader's // `unwrapExports` (`exports.default ?? exports`) collapse the module to the diff --git a/packages/ui/acp/src/codec.ts b/packages/ui/acp/src/codec.ts index 444e71545c..3830dba676 100644 --- a/packages/ui/acp/src/codec.ts +++ b/packages/ui/acp/src/codec.ts @@ -39,6 +39,8 @@ import type { ContentBlock as AcpContentBlock, StopReason } from '@agentclientpr * hook before any step ran — ACP has no "rejected" reason, and a * blocked prompt is, from the client's view, the prompt not being * carried out; `cancelled` is the closest legal wire reason) + * @param reason - the harness turn-end reason to translate. + * @returns the legal ACP wire value per the mapping above. */ export function turnEndToStopReason(reason: TurnEndReason): StopReason { switch (reason.kind) { @@ -71,6 +73,8 @@ export function turnEndToStopReason(reason: TurnEndReason): StopReason { * `reasoning` is surfaced via `agent_thought_chunk` * streaming rather than as a message block, and `tool-call`/`tool-result` * are handled by the tool-call update path. + * @param block - the harness content block to translate. + * @returns the ACP block, or `undefined` for a kind with no message-content mapping. */ export function harnessBlockToAcpContent(block: ContentBlock): AcpContentBlock | undefined { switch (block.type) { @@ -89,6 +93,8 @@ export function harnessBlockToAcpContent(block: ContentBlock): AcpContentBlock | * concatenated verbatim; resource links become explicit textual references so * baseline ACP clients can point at files without the bridge silently dropping * that context. + * @param prompt - the ACP prompt blocks to flatten. + * @returns the concatenated text, with resource links rendered as bracketed references. */ export function acpPromptToText(prompt: readonly AcpContentBlock[]): string { return prompt @@ -109,6 +115,8 @@ export function acpPromptToText(prompt: readonly AcpContentBlock[]): string { * Whether an ACP prompt contains content the bridge cannot accept. Baseline ACP * requires `text` and `resource_link`; richer inline payloads (`resource`, * image, audio, …) are rejected rather than silently dropped. + * @param prompt - the ACP prompt blocks to inspect. + * @returns `true` when any block is neither `text` nor `resource_link`. */ export function promptHasUnsupportedContent(prompt: readonly AcpContentBlock[]): boolean { return prompt.some(block => block.type !== 'text' && block.type !== 'resource_link') diff --git a/packages/ui/acp/src/index.ts b/packages/ui/acp/src/index.ts index a819e607d0..b4fa277311 100644 --- a/packages/ui/acp/src/index.ts +++ b/packages/ui/acp/src/index.ts @@ -701,6 +701,8 @@ export function apply(ctx: Context, config: AcpConfig): void { * Build per-agent options from the plugin config, omitting absent fields * (exactOptionalPropertyTypes: never assign `undefined` to an optional key). * Exported for unit coverage of both the present and absent branches. + * @param config - the plugin config carrying the optional model name. + * @returns the per-agent options, with `model` present only when configured. */ export function agentOptions(config: AcpConfig): { model?: string } { return { @@ -764,6 +766,16 @@ function validateMcpServers(params: { mcpServers?: unknown[] }): void { * * Other event types (turn/step boundaries, context/message, …) produce * no client update. + * @param sessionId - the ACP session id stamped on every emitted notification. + * @param event - the harness session event to translate. + * @param notify - sink for each produced `session/update` notification; called + * zero or more times per event (best-effort UI feed, never load-bearing). + * @param presenter - resolves tool-owned render intent for tool events; + * defaults to the generic-fallback {@link nullToolPresenter}. + * @param terminal - the connection's terminal-rendering context; defaults to + * disabled (the plain-text console-block fallback). + * @param options - `includeUserMessages` (default `true`): live streaming + * passes `false` so a prompt the client just sent is not echoed back. */ export function streamSessionEventUpdate( sessionId: SessionId, @@ -825,6 +837,8 @@ export function streamSessionEventUpdate( * harness status triple IS `PlanEntryStatus`). The ACP client REPLACES its whole * plan on each `plan` update, matching the harness's whole-list-replace * semantics, so no per-entry diffing is needed. + * @param todos - the harness todo list (the whole list, not a diff). + * @returns the ACP plan body, one entry per todo. */ export function todosToPlan(todos: TodoItem[]): Plan { return { entries: todos.map((todo): PlanEntry => ({ content: todo.content, priority: 'medium', status: todo.status })) } @@ -885,7 +899,16 @@ export class ToolPresenter { private readonly onError: (message: string) => void = () => {}, ) {} - /** Pending-state render intent for a `tool/call`; remembers `(name, args, card)` for the matching result. */ + /** + * Pending-state render intent for a `tool/call`; remembers `(name, args, card)` + * for the matching result. + * @param callId - the call id the matching `tool/result` will look up. + * @param name - the tool name, resolved against the registry for `presentCall`. + * @param argsJson - the raw arguments JSON from the event; parsed for the view + * (a non-JSON string is surfaced raw). + * @returns the tool-owned view, or the generic fallback (title = tool name, + * kind `other`, parsed args as raw input) when the tool defines none or threw. + */ call(callId: CallId, name: string, argsJson: string): ToolCallView { const args = parseToolArguments(argsJson) let present: ToolCallView | undefined @@ -905,7 +928,18 @@ export class ToolPresenter { return view } - /** Completed-state render intent for a `tool/result`; consumes the remembered `(name, args, card)`. */ + /** + * Completed-state render intent for a `tool/result`; consumes the remembered + * `(name, args, card)`. + * @param callId - the id of the matching `tool/call`; an unknown or late id + * falls back to the raw content. + * @param content - the result's content blocks (the fallback and fill-in body). + * @param isError - whether the result is an error, forwarded to `presentResult`. + * @param meta - the result's machine-readable meta, forwarded when present. + * @returns the tool-owned view — an orphaned `terminal` result (no terminal + * call side) and a content-less `generic` are normalized — or the raw-content + * generic card when the tool defines no `presentResult` or threw. + */ result(callId: CallId, content: ContentBlock[], isError: boolean, meta?: unknown): ToolResultView { const call = this.pending.get(callId) this.pending.delete(callId) diff --git a/packages/ui/app-boot/src/index.ts b/packages/ui/app-boot/src/index.ts index 5515501188..8f3392e913 100644 --- a/packages/ui/app-boot/src/index.ts +++ b/packages/ui/app-boot/src/index.ts @@ -36,6 +36,10 @@ import Loader from '@cordisjs/plugin-loader' * the SAME directory (the keyless replay tree). Other modes — including no * snapshot mode at all — use the path as-is. Returns an absolute path resolved * from `cwd`. + * @param configPath - the requested config path (absolute, or relative to `cwd`). + * @param snapshotMode - the bin's `$DSH_SNAPSHOT` value; only `'replay'` swaps the basename. + * @param cwd - the base a relative `configPath` resolves against. + * @returns the absolute path of the config to boot. */ export function resolveConfigPath( configPath: string, snapshotMode: string | undefined, cwd: string = process.cwd(), @@ -54,6 +58,9 @@ export function resolveConfigPath( * them via the `!!js` tag. A present-but-unreadable `.env` is a real * misconfiguration: surface it via `warn` (one line, default stderr) rather * than silently running with the wrong environment. + * @param binName - the diagnostic prefix on the warn line. + * @param dir - the directory whose `.env` to load. + * @param warn - sink for the one-line misconfiguration diagnostic. */ export function loadEnv( binName: string, dir: string = process.cwd(), @@ -90,6 +97,9 @@ export interface FailLoudProcess { * STDERR (never stdout — for the ACP bin that channel carries JSON-RPC) and * guarantees `exit(1)`. Install before `boot()`. Returns the uninstaller * (tests use it; the bins run until exit and never do). + * @param binName - the diagnostic prefix on the fatal-failure line. + * @param proc - the process slice to register on; tests inject a fake. + * @returns the uninstaller that removes the rejection handler. */ export function installFailLoud(binName: string, proc: FailLoudProcess = process): () => void { const handler = (err: unknown): void => { @@ -108,6 +118,8 @@ export function installFailLoud(binName: string, proc: FailLoudProcess = process * entry is the one legitimate fiber-less state: `Entry.refresh()` deliberately * skips `init()` for it — a valid "plugin turned off" config, not a failed * import — so it is excluded. + * @param ctx - the settled context whose loader entries to audit. + * @param binName - the diagnostic prefix on the thrown error. */ export function assertEntriesLoaded(ctx: Context, binName: string): void { const failed = [...ctx.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled) @@ -139,6 +151,10 @@ export function assertEntriesLoaded(ctx: Context, binName: string): void { * active under `node --expose-internals`; a consumer running a built bin must * pass that flag (or install the plugins where node hoists them). Relative * specifiers resolve against the config directory with no flag. + * @param binName - the diagnostic prefix for load-failure errors. + * @param absoluteConfigPath - the config to include; must already be absolute + * (see {@link resolveConfigPath}). + * @returns the root context once every entry has started. */ export async function boot(binName: string, absoluteConfigPath: string): Promise { const ctx = new Context() diff --git a/packages/ui/stdio-agent/README.md b/packages/ui/stdio-agent/README.md index fd608b7888..5fb0bbe78c 100644 --- a/packages/ui/stdio-agent/README.md +++ b/packages/ui/stdio-agent/README.md @@ -25,6 +25,7 @@ The leaf `cordis.yml` supplies only the **swappable backends** — an LLM adapte |---|---|---| | `model` | (required) | the pre-created `main` agent's model | | `persona` | — | the deployment persona template (may reference `{{model}}`), routed to `dsh-system-prompt` | +| `toolOrder` | — | explicit model-facing tool order (a name list with one `''` rest entry; absent — lexicographic; an unregistered name fails each turn at prompt assembly), routed to `dsh-system-prompt` | | `persistenceRoot` | `./.sessions` | the JSONL backend's root directory | | `welcome` | `ready.` | the stdin-chat banner | | `resumeSessionId` | — | resume a persisted session id instead of starting fresh (sourced from an env var in the leaf) | diff --git a/packages/ui/stdio-agent/package.json b/packages/ui/stdio-agent/package.json index 0dffe49211..b8cf84cac4 100644 --- a/packages/ui/stdio-agent/package.json +++ b/packages/ui/stdio-agent/package.json @@ -50,6 +50,7 @@ "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-agent-core": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", "cordis": "^4.0.0-rc.6", diff --git a/packages/ui/stdio-agent/src/index.ts b/packages/ui/stdio-agent/src/index.ts index fe63e02643..7d36db373e 100644 --- a/packages/ui/stdio-agent/src/index.ts +++ b/packages/ui/stdio-agent/src/index.ts @@ -53,7 +53,8 @@ export const name = 'stdio-agent' * App config: the swappable per-demo values, each routed to where the app wires * it. `model`/`resumeSessionId` configure the pre-created `main` agent (through * {@link @deepseek-ai/dsh-agent-core}'s forwarded `agents` list); `persona` is - * the deployment persona (forwarded to the system-prompt plugin); + * the deployment persona (forwarded to the system-prompt plugin); `toolOrder` + * is the explicit model-facing tool order (forwarded to the system-prompt plugin); * `persistenceRoot` is the JSONL backend's directory; `welcome` is the UI banner. */ export interface Config { @@ -61,6 +62,8 @@ export interface Config { model: string /** Deployment persona (the system-prompt plugin's `persona` config). */ persona?: string + /** Explicit model-facing tool order (the system-prompt plugin's `toolOrder` config; see dsh-system-prompt). */ + toolOrder?: string[] /** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */ persistenceRoot?: string /** stdin-chat banner printed once on start. Defaults to `'ready.'`. */ @@ -76,6 +79,10 @@ export interface Config { export const Config: z = z.object({ model: z.string().required(), persona: z.string(), + // The array default is forced to undefined: ABSENT means "lexicographic + // order" (the owning dsh-system-prompt schema does the same), while + // schemastery's native [] default would read as an invalid configured list. + toolOrder: z.array(z.string()).default(undefined as unknown as string[]), persistenceRoot: z.string().default('./.sessions'), welcome: z.string().default('ready.'), resumeSessionId: z.string(), @@ -92,6 +99,7 @@ export function apply(ctx: Context, config: Config): void { ctx.plugin(ConsoleExporter) ctx.plugin(agentCore, { ...config.persona !== undefined ? { persona: config.persona } : {}, + ...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {}, agents: [{ id: AgentId('main'), model: config.model, diff --git a/packages/ui/stdio-agent/src/stdio-chat.ts b/packages/ui/stdio-agent/src/stdio-chat.ts index 2996e11e89..78b0a5ae5b 100644 --- a/packages/ui/stdio-agent/src/stdio-chat.ts +++ b/packages/ui/stdio-agent/src/stdio-chat.ts @@ -63,6 +63,10 @@ function isTTYPair(input: Readable, output: Writable): boolean { * directly with fakes. Returns nothing — all registration is via `ctx.on`/ * `ctx.effect`, so fiber disposal tears every listener and the readline * interface down. + * @param ctx - the context supplying the `agents` service and the event feeds. + * @param config - the plugin config; defaults are re-applied here for direct + * callers that bypass Loader validation. + * @param runtime - the process-I/O seam (line source, render sink, exit hook). */ export function createStdioChat(ctx: Context, config: Config, runtime: StdioRuntime): void { // Default here too (not just via schemastery's `.default()`): this helper is diff --git a/packages/ui/stdio-agent/tests/built-bin.e2e.ts b/packages/ui/stdio-agent/tests/built-bin.e2e.ts index cda8b41b1f..7b84fad65b 100644 --- a/packages/ui/stdio-agent/tests/built-bin.e2e.ts +++ b/packages/ui/stdio-agent/tests/built-bin.e2e.ts @@ -76,9 +76,10 @@ async function makeConsumer(welcome: string, disabledBrokenEntry = false): Promi await mkdir(dirname(target), { recursive: true }) await symlink(abs, target) } - // The example's mock model + echo tool are example-local TS plugins (Node 24+ - // strips types natively, so plain `node` loads them); they import the workspace - // packages the symlinked node_modules now provides. + // The example's mock model + echo tool are example-local TS plugins (Node + // 22.19+ — the engines floor — strips types natively, so plain `node` loads + // them); they import the workspace packages the symlinked node_modules now + // provides. await cp(join(repoRoot, 'examples/echo-agent/src'), join(dir, 'src'), { recursive: true }) await writeFile(join(dir, 'cordis.yml'), [ '- id: mock-llm', diff --git a/packages/ui/stdio-agent/tests/stdio-agent.spec.ts b/packages/ui/stdio-agent/tests/stdio-agent.spec.ts index 09fbf8987d..34ba201040 100644 --- a/packages/ui/stdio-agent/tests/stdio-agent.spec.ts +++ b/packages/ui/stdio-agent/tests/stdio-agent.spec.ts @@ -2,6 +2,7 @@ import { describe, it, expect } from 'vitest' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import { AgentId } from '@deepseek-ai/dsh-agent' +import { TOOL_ORDER_REST } from '@deepseek-ai/dsh-system-prompt' import * as stdioAgent from '../src/index.ts' /** @@ -74,6 +75,27 @@ describe('dsh-stdio-agent app', () => { expect(stdioAgent.Config).toBeDefined() }) + it('forwards toolOrder through agent-core to the system-prompt assembly', async () => { + const ctx = await mount({ + model: 'mock', + toolOrder: ['zulu', TOOL_ORDER_REST], + persistenceRoot: '/tmp/dsh-stdio-agent-spec-tool-order', + }) + // The bundle's own bash tools pend on the absent `ctx.bash` executor in + // this providerless mount, so register two plain tools to order. + for (const name of ['alpha', 'zulu']) { + ctx.get('tools')!.register({ + name, + description: name, + parameters: {}, + execute: async () => [], + }) + } + const assembly = await ctx.get('systemPrompt')!.assemble() + expect(assembly.tools.map(tool => tool.name)).toEqual(['zulu', 'alpha']) + await ctx.fiber.dispose() + }) + it('has the namespace-plugin export shape (no stray default) so the Loader keeps name/Config/apply', () => { // Postmortem 0001 guard: a stray `export default apply` makes the Loader's // `unwrapExports` (`exports.default ?? exports`) collapse the module to the diff --git a/packages/util/README.md b/packages/util/README.md index ae73c8125f..45afe7b0a9 100644 --- a/packages/util/README.md +++ b/packages/util/README.md @@ -5,5 +5,8 @@ Zero-dependency primitives shared across the other groups. A package lands here | Package | Role | |---|---| | `brand/` | The type-only `Branded` nominal-typing primitive (no runtime code, no harness deps) | +| `timeout/` | The timing/classification half of a timeout — `clampTimeout`/`deadline`/`timeoutOf`/`TimeoutReason` (pure functions, no harness deps); termination stays in each capability | `dsh-brand` is the canonical case: it owns ONLY the `Branded` helper, so a capability package can brand the ids it owns (`dsh-bash`'s `BashTaskId`/`OwnerToken`, `dsh-session`'s `SessionId`, …) by depending on `dsh-brand` alone, without pulling in an unrelated package just to reach `Branded`. + +`dsh-timeout` follows the same shape for the timeout family: `dsh-bash` and `dsh-web-fetch-local` each fuse a caller's cancellation with a deadline and later classify "timed out" vs "cancelled" by depending on `dsh-timeout` alone. It deliberately owns only the timing/classification half — the *termination* (SIGKILL a process group, tear down a fetch socket) stays in each capability, because no shared layer can own every capability's kill (see [the timeout-library RFC](../../docs/rfc/implemented/architecture/2026-07-06-timeout-deadline-library.md)). diff --git a/packages/util/timeout/README.md b/packages/util/timeout/README.md new file mode 100644 index 0000000000..db2b06ba53 --- /dev/null +++ b/packages/util/timeout/README.md @@ -0,0 +1,42 @@ +# dsh-timeout + +The **timing-and-classification** half of a timeout — a zero-dependency library of pure functions (no runtime harness deps) shared by every capability that clamps a caller's timeout hint, arms a deadline, and later has to tell "timed out" apart from "cancelled". + +It owns **no termination**. The signal it hands out only *notifies*; actually stopping the work stays in each capability, because that mechanism differs — bash SIGKILLs an OS process group, web tears down a `fetch` socket — and no shared layer can own all of them. This is the boundary the [RFC](../../../docs/rfc/implemented/architecture/2026-07-06-timeout-deadline-library.md) draws: share the timing/classification, keep the hard kill local. + +It is a **library, not a service or plugin**: no `ctx`, registers nothing, holds no state, emits no events. A "timeout service" would have to understand how to stop every capability's work — exactly the knowledge a microkernel keeps out of shared layers. + +## Surface + +```ts +import { clampTimeout, deadline, timeoutOf, TimeoutReason } from '@deepseek-ai/dsh-timeout' +``` + +| Export | Role | +|---|---| +| `clampTimeout(requested, def, max, name?)` | Validate the caller's optional positive-finite hint, fill from `def`, cap at `max`. Throws (with `name`) on a non-positive/non-finite hint. | +| `deadline(upstream, timeoutMs, code)` | Fuse `upstream` cancellation with a timeout into one `AbortSignal` (`AbortSignal.any`); the timeout carries a `TimeoutReason`. `[Symbol.dispose]` clears the timer. | +| `timeoutOf(signal \| { reason }, code?)` | Recover the `TimeoutReason` from an aborted signal/error, else `undefined` — the timeout-vs-cancel classifier. Pass `code` to match only THIS deadline's timer (see nesting below). | +| `TimeoutReason` | The internal reason (`code` + `timeoutMs`) stamped on a timeout abort. Not a public error — providers translate it into their own error/field. | + +## The `timeoutMs <= 0` sentinel + +`0` is the **internal** "no timeout" value for backend-owned background work (bash `start()`): `deadline()` arms no timer and forwards only `upstream`; with no upstream either, it returns a never-aborting signal plus a no-op disposer, so every caller keeps one call shape. External request hints validate as **positive finite** via `clampTimeout` before they reach `deadline`, so `0` is never a model-/plugin-facing "disable timeout" value. + +## Usage shape + +```ts ignore-check +// Scope-lifetime consumer (foreground bash, one fetch): `using` disposes the timer. +using d = deadline(upstream, timeoutMs, 'BASH_TIMEOUT') +const outcome = await runWork({ signal: d.signal }) // work listens on d.signal and terminates itself +const timedOut = timeoutOf(d.signal, 'BASH_TIMEOUT') !== undefined // classify the first abort, scoped to OUR code +const aborted = d.signal.aborted && !timedOut // mutually exclusive: timeout won, or cancel did +``` + +The signal only *notifies* — the caller MUST attach its own termination (`d.signal.addEventListener('abort', kill)`, or hand `d.signal` to `fetch`). Racing a promise against a timer would resolve the tool-call while the child process or socket leaks on; handing out a signal forces a real termination path to exist. + +Pass your own `code` to `timeoutOf` so classification composes under nesting: when the `upstream` you were handed is *itself* a deadline signal (a future `tools/execute` middleware arming a per-call deadline), `AbortSignal.any` preserves the outer `TimeoutReason` if the outer timer fires first. Scoping to your `code` makes a foreign timeout read as an ordinary upstream cancel — the correct classification from your capability's view — instead of your own timeout firing when your local timer never expired. + +## What does NOT get a timeout + +Local file `read`/`write`/`edit` take no `timeoutMs`: a syscall is best-effort-abortable at most, a timeout could not force `fsync`/`rename` to stop, and adding one would be an implicit default that violates explicit-over-implicit. See [`fs/`](../../fs/README.md). diff --git a/packages/util/timeout/package.json b/packages/util/timeout/package.json new file mode 100644 index 0000000000..150a155324 --- /dev/null +++ b/packages/util/timeout/package.json @@ -0,0 +1,30 @@ +{ + "name": "@deepseek-ai/dsh-timeout", + "description": "Zero-dependency timeout/deadline primitive: clampTimeout, deadline, timeoutOf, TimeoutReason (timing + classification only, no termination)", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/util/timeout/src/index.ts b/packages/util/timeout/src/index.ts new file mode 100644 index 0000000000..ed95a877d3 --- /dev/null +++ b/packages/util/timeout/src/index.ts @@ -0,0 +1,162 @@ +/** + * The timing-and-classification half of a timeout — a zero-dependency library + * of pure functions shared by every capability that clamps a caller's timeout + * hint, arms a deadline, and later has to tell "timed out" apart from + * "cancelled". It owns NO termination: the returned {@link deadline} signal only + * NOTIFIES; actually stopping the work (SIGKILL a process group, tear down a + * fetch socket, …) stays in each capability's implementation, because that + * mechanism differs per capability and no shared layer can own all of them. + * + * This is deliberately a library, not a cordis service or plugin: it takes no + * `ctx`, registers nothing, holds no cross-call state, and emits no events. A + * "timeout service" would have to understand how to stop every capability's + * work — exactly the knowledge a microkernel keeps out of shared layers. + * + * The four exports and their division of labor: + * - {@link clampTimeout} — validate a caller's optional positive hint, fill the + * backend default, cap at the backend max (pure arithmetic + the shared + * positive-finite request contract). + * - {@link deadline} — fuse upstream cancellation with a timeout into one + * `AbortSignal`, the timeout carrying an identifiable {@link TimeoutReason}; + * `[Symbol.dispose]` clears the timer. + * - {@link timeoutOf} — classify an aborted signal (or error): a + * {@link TimeoutReason} means the timeout fired, anything else (or nothing) + * means it did not. + * - {@link TimeoutReason} — the internal classification reason; providers + * translate it into their own public error/result shape before returning. + * + * @module @deepseek-ai/dsh-timeout + */ + +/** + * The internal reason attached to a timeout abort so consumers can classify it + * after the fact. It carries the failing `code` (each capability's own string — + * `BASH_TIMEOUT`, `WEB_FETCH_TIMEOUT`, …) and the `timeoutMs` that elapsed. + * + * It is an INTERNAL classification reason, not a public error: providers + * translate it into their seam-specific error code or result field (via + * {@link timeoutOf}) before returning to callers. Native `AbortSignal.timeout()` + * yields a fixed `TimeoutError` indistinguishable across timeout kinds; this + * type is identifiable and carries the code/duration. + */ +export class TimeoutReason extends Error { + override name = 'TimeoutReason' + + /** + * @param code Capability-owned timeout code (e.g. `BASH_TIMEOUT`). + * @param timeoutMs The deadline that elapsed, in milliseconds. + */ + constructor(readonly code: string, readonly timeoutMs: number) { + super(`${code} after ${timeoutMs}ms`) + } +} + +/** + * Validate a caller's optional timeout hint, fill it from the backend default, + * then cap at the backend max. The shared positive-finite request contract: + * a supplied `requested` must be a positive finite number or this throws — + * `0` is NOT a caller-facing "disable timeout" value (that sentinel is internal + * to {@link deadline}). A missing `requested` falls back to `def`. + * + * @param requested The caller's optional hint; validated when present. + * @param def The backend default applied when `requested` is absent. + * @param max The backend upper bound the result is capped to. + * @param name Field name used in the thrown message (so the caller sees which input was bad). + * @returns The effective timeout in milliseconds: `min(requested ?? def, max)`. + */ +export function clampTimeout( + requested: number | undefined, + def: number, + max: number, + name = 'timeoutMs', +): number { + if (requested !== undefined && (!Number.isFinite(requested) || requested <= 0)) { + throw new Error(`${name} must be a positive finite number`) + } + return Math.min(requested ?? def, max) +} + +/** A deadline signal plus the cleanup that clears its timer (dispose-once). */ +export interface Deadline { + /** Aborts on upstream cancellation OR on timeout (the timeout carries a {@link TimeoutReason}). */ + readonly signal: AbortSignal + /** Clear the timer. Safe to call once; `using` calls it at scope exit. */ + [Symbol.dispose](): void +} + +/** + * Build a deadline signal that aborts on upstream cancellation OR on timeout, + * with the timeout carrying an identifiable {@link TimeoutReason} (unlike + * native `AbortSignal.timeout()`, whose fixed `TimeoutError` is opaque). It is + * `AbortSignal.any([upstream, ])` — the single primitive that fuses + * two abort sources — with the reason and a disposable timer added on top. + * + * `timeoutMs <= 0` is the INTERNAL "no timeout" sentinel for backend-owned + * background work: arm no timer and forward only the upstream signal; with no + * upstream either, return a never-aborting signal so callers keep one call + * shape. External request hints validate as positive finite via + * {@link clampTimeout} before reaching here, so `0` never arrives from a model + * or plugin. + * + * The returned object's `[Symbol.dispose]` clears the timer — use `using` for a + * scope-lifetime consumer, or call it manually for an event-lifetime one. The + * signal only NOTIFIES; the caller must attach its own termination (kill the + * process group, abort the fetch, …). + * + * @param upstream The caller's cancellation signal, if any, fused into the result. + * @param timeoutMs Deadline in milliseconds; `<= 0` means "no timeout" (arm no timer). + * @param code Capability-owned code stamped onto the timeout's {@link TimeoutReason}. + * @returns The fused {@link Deadline} (signal + timer cleanup). + */ +export function deadline( + upstream: AbortSignal | undefined, + timeoutMs: number, + code: string, +): Deadline { + if (timeoutMs <= 0) { + // No timeout (background work): forward only the upstream signal, or a + // never-aborting one when there is no upstream. No timer, so dispose is a + // no-op — the empty method keeps the one call shape for every caller. + return { signal: upstream ?? new AbortController().signal, [Symbol.dispose]() {} } + } + + const timer = new AbortController() + const id = setTimeout(() => { timer.abort(new TimeoutReason(code, timeoutMs)) }, timeoutMs) + return { + // AbortSignal.any adopts the reason of whichever source aborts FIRST, so a + // race resolves to a single cause: timeoutOf() reads TimeoutReason only + // when the timeout won, and upstream-wins leaves an ordinary abort reason. + signal: upstream !== undefined ? AbortSignal.any([upstream, timer.signal]) : timer.signal, + [Symbol.dispose]() { clearTimeout(id) }, + } +} + +/** + * Recover the {@link TimeoutReason} from an aborted signal (or any object with a + * `reason`), else `undefined`. This is the classification half: a provider + * calls it on the deadline signal after an abort to decide whether the cause + * was its timeout (translate to the capability's timeout error/field) or an + * ordinary upstream cancellation (`undefined` → the cancel path). + * + * Pass `code` to scope the match to THIS deadline's timer. It matters under + * nesting: when the `upstream` handed to {@link deadline} is itself a deadline + * signal (e.g. a future `tools/execute` middleware arming a per-call deadline), + * `AbortSignal.any` preserves the OUTER `TimeoutReason` if the outer timer fires + * first. Without `code`, the inner capability would misclassify that outer + * timeout as its own (`timedOut:true` / `WEB_FETCH_TIMEOUT`) though its local + * timer never expired; with `code`, a foreign timeout reads as `undefined` and + * falls through to the upstream-cancel path, which is the correct classification + * from the inner capability's view. Omit `code` only to ask "was this ANY + * timeout" (a generic middleware that owns no single code). + * + * @param x An {@link AbortSignal} or any `{ reason }` carrier (e.g. a caught abort error). + * @param code When provided, only a {@link TimeoutReason} with this exact `code` matches. + * @returns The matching {@link TimeoutReason}, else `undefined`. + */ +export function timeoutOf(x: AbortSignal | { reason?: unknown }, code?: string): TimeoutReason | undefined { + // AbortSignal.reason is typed `any`; pin it to `unknown` so no `any` leaks and + // the instanceof narrows cleanly for both a signal and a bare reason carrier. + const reason: unknown = x.reason + if (!(reason instanceof TimeoutReason)) return undefined + return code === undefined || reason.code === code ? reason : undefined +} diff --git a/packages/util/timeout/tests/timeout.spec.ts b/packages/util/timeout/tests/timeout.spec.ts new file mode 100644 index 0000000000..588317f48d --- /dev/null +++ b/packages/util/timeout/tests/timeout.spec.ts @@ -0,0 +1,185 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { clampTimeout, deadline, timeoutOf, TimeoutReason } from '@deepseek-ai/dsh-timeout' + +describe('TimeoutReason', () => { + it('is an Error carrying the code and elapsed ms', () => { + const reason = new TimeoutReason('BASH_TIMEOUT', 100) + expect(reason).toBeInstanceOf(Error) + expect(reason.name).toBe('TimeoutReason') + expect(reason.code).toBe('BASH_TIMEOUT') + expect(reason.timeoutMs).toBe(100) + expect(reason.message).toBe('BASH_TIMEOUT after 100ms') + }) +}) + +describe('clampTimeout', () => { + it('fills the default when the hint is absent', () => { + expect(clampTimeout(undefined, 120_000, 600_000)).toBe(120_000) + }) + + it('caps the hint at max', () => { + expect(clampTimeout(999_999, 120_000, 600_000)).toBe(600_000) + }) + + it('keeps a valid hint under the cap', () => { + expect(clampTimeout(5_000, 120_000, 600_000)).toBe(5_000) + }) + + it('caps the default itself when the default exceeds max', () => { + // min(def, max) applies even with no hint — a misconfigured backend never + // exceeds its own cap. + expect(clampTimeout(undefined, 900_000, 600_000)).toBe(600_000) + }) + + it('rejects a non-finite hint with the caller-provided name', () => { + expect(() => clampTimeout(Number.NaN, 100, 200, 'bash-local: request.timeoutMs')) + .toThrow(/bash-local: request\.timeoutMs must be a positive finite number/) + expect(() => clampTimeout(Number.POSITIVE_INFINITY, 100, 200)) + .toThrow(/timeoutMs must be a positive finite number/) + }) + + it('rejects a non-positive hint', () => { + expect(() => clampTimeout(0, 100, 200)).toThrow(/must be a positive finite number/) + expect(() => clampTimeout(-1, 100, 200)).toThrow(/must be a positive finite number/) + }) +}) + +describe('deadline — timeout arm', () => { + afterEach(() => { vi.useRealTimers() }) + + it('aborts on timeout with a TimeoutReason after the elapsed ms', () => { + vi.useFakeTimers() + using d = deadline(undefined, 100, 'BASH_TIMEOUT') + expect(d.signal.aborted).toBe(false) + vi.advanceTimersByTime(100) + expect(d.signal.aborted).toBe(true) + const reason = timeoutOf(d.signal) + expect(reason).toBeInstanceOf(TimeoutReason) + expect(reason?.code).toBe('BASH_TIMEOUT') + expect(reason?.timeoutMs).toBe(100) + }) + + it('[Symbol.dispose] clears the timer so no abort fires afterward', () => { + vi.useFakeTimers() + const d = deadline(undefined, 100, 'BASH_TIMEOUT') + d[Symbol.dispose]() + vi.advanceTimersByTime(1_000) + expect(d.signal.aborted).toBe(false) + expect(timeoutOf(d.signal)).toBeUndefined() + }) +}) + +describe('deadline — fuse with upstream', () => { + it('aborts on upstream cancellation, classified as NOT a timeout', () => { + const upstream = new AbortController() + using d = deadline(upstream.signal, 60_000, 'BASH_TIMEOUT') + upstream.abort('user cancelled') + expect(d.signal.aborted).toBe(true) + expect(timeoutOf(d.signal)).toBeUndefined() + }) + + it('cancel wins when it fires before the timeout', () => { + vi.useFakeTimers() + try { + const upstream = new AbortController() + using d = deadline(upstream.signal, 100, 'BASH_TIMEOUT') + upstream.abort('user cancelled') // fires first, before the 100ms timer + vi.advanceTimersByTime(200) + expect(d.signal.aborted).toBe(true) + // AbortSignal.any adopts the FIRST source's reason: cancel won, so no + // TimeoutReason even though the timer later elapsed. + expect(timeoutOf(d.signal)).toBeUndefined() + } finally { + vi.useRealTimers() + } + }) + + it('timeout wins when it fires before upstream cancellation', () => { + vi.useFakeTimers() + try { + const upstream = new AbortController() + using d = deadline(upstream.signal, 100, 'WEB_FETCH_TIMEOUT') + vi.advanceTimersByTime(150) // past the 100ms deadline: the timer fires first + expect(d.signal.aborted).toBe(true) + expect(timeoutOf(d.signal)?.code).toBe('WEB_FETCH_TIMEOUT') + // A later upstream abort is a no-op on the already-aborted fused signal: + // AbortSignal.any keeps the FIRST cause, so the timeout classification stands. + upstream.abort('too late') + expect(timeoutOf(d.signal)?.code).toBe('WEB_FETCH_TIMEOUT') + } finally { + vi.useRealTimers() + } + }) + + it('forwards a pre-aborted upstream signal immediately', () => { + const upstream = new AbortController() + upstream.abort('already gone') + using d = deadline(upstream.signal, 60_000, 'BASH_TIMEOUT') + expect(d.signal.aborted).toBe(true) + expect(timeoutOf(d.signal)).toBeUndefined() + }) +}) + +describe('deadline — timeoutMs <= 0 (no-timeout sentinel)', () => { + afterEach(() => { vi.useRealTimers() }) + + it('arms no timer and forwards only the upstream signal', () => { + vi.useFakeTimers() + const upstream = new AbortController() + using d = deadline(upstream.signal, 0, 'BASH_TIMEOUT') + vi.advanceTimersByTime(1_000_000) + expect(d.signal.aborted).toBe(false) // no timer ever armed + upstream.abort('kill') + expect(d.signal.aborted).toBe(true) + expect(timeoutOf(d.signal)).toBeUndefined() // never a timeout + }) + + it('returns a never-aborting signal with a no-op disposer when there is no upstream', () => { + vi.useFakeTimers() + const d = deadline(undefined, 0, 'BASH_TIMEOUT') + expect(() => { d[Symbol.dispose]() }).not.toThrow() + vi.advanceTimersByTime(1_000_000) + expect(d.signal.aborted).toBe(false) + expect(timeoutOf(d.signal)).toBeUndefined() + }) + + it('treats a negative timeout the same as zero', () => { + const d = deadline(undefined, -5, 'BASH_TIMEOUT') + expect(d.signal.aborted).toBe(false) + d[Symbol.dispose]() + }) +}) + +describe('timeoutOf', () => { + it('classifies a bare reason carrier that holds a TimeoutReason', () => { + const reason = new TimeoutReason('WEB_FETCH_TIMEOUT', 50) + expect(timeoutOf({ reason })).toBe(reason) + }) + + it('returns undefined for a non-timeout reason', () => { + expect(timeoutOf({ reason: new Error('other') })).toBeUndefined() + expect(timeoutOf({ reason: 'user cancelled' })).toBeUndefined() + expect(timeoutOf({})).toBeUndefined() + }) + + it('matches only the requested code when one is given', () => { + const reason = new TimeoutReason('BASH_TIMEOUT', 100) + expect(timeoutOf({ reason }, 'BASH_TIMEOUT')).toBe(reason) + expect(timeoutOf({ reason }, 'WEB_FETCH_TIMEOUT')).toBeUndefined() + }) +}) + +describe('deadline — nested deadlines', () => { + it("does not misclassify an outer deadline's timeout as the inner code", () => { + // The upstream handed to the inner deadline is ITSELF a deadline that has + // already timed out (outer). AbortSignal.any preserves the outer reason; + // scoping timeoutOf to the inner code keeps the inner capability from + // reporting the outer timeout as its own — it reads as an upstream cancel. + const outer = new AbortController() + outer.abort(new TimeoutReason('OUTER_TIMEOUT', 30)) + using inner = deadline(outer.signal, 60_000, 'BASH_TIMEOUT') + expect(inner.signal.aborted).toBe(true) + expect(timeoutOf(inner.signal, 'BASH_TIMEOUT')).toBeUndefined() // not ours → upstream-cancel path + expect(timeoutOf(inner.signal)?.code).toBe('OUTER_TIMEOUT') // but IS a timeout, unscoped + }) +}) diff --git a/packages/util/timeout/tsconfig.json b/packages/util/timeout/tsconfig.json new file mode 100644 index 0000000000..749cb0208e --- /dev/null +++ b/packages/util/timeout/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [] +} diff --git a/packages/web/tool-web/README.md b/packages/web/tool-web/README.md index d8a2e266a9..ab1326a21e 100644 --- a/packages/web/tool-web/README.md +++ b/packages/web/tool-web/README.md @@ -1,6 +1,6 @@ # @deepseek-ai/dsh-tool-web -The model-facing web tool suite — `web_search` and `web_fetch` — over the [web capability seam](../web/README.md) (`ctx.web`). It owns model-facing concerns only: tool names, JSON schemas, snake_case argument names, prompt sections, the result-count bound, result formatting, HTML→markdown presentation, and `presentCall`. All web access goes through `ctx.web`; this package never imports a concrete provider. +The model-facing web tool suite — `web_search` and `web_fetch` — over the [web capability seam](../web/README.md) (`ctx.web`). It owns model-facing concerns only: tool names, JSON schemas, snake_case argument names, prompt sections, the result-count bound, result formatting, HTML→markdown presentation, and `presentCall`. All web access goes through `ctx.web`; this package never imports a concrete provider. Neither tool exposes a model-facing timeout — each tool's cooperative tool-call budget is declared here via config (`fetchTimeoutMs`/`searchTimeoutMs`, attached as `ToolDefinition.timeoutMs`) and enforced by [`@deepseek-ai/dsh-timeout-policy`](../../timeout/timeout-policy/README.md) (a `tools/execute` wrapper); each tool just forwards `exec.signal` to the seam. Each tool is registered independently; a product that wants only one disables the other via config (`{ search: false }` / `{ fetch: false }`). @@ -9,7 +9,7 @@ Each tool is registered independently; a product that wants only one disables th | Tool | Args | Behavior | |---|---|---| | `web_search` | `query` (string) | Discovery. Returns an optional answer plus source URLs. `max_results` is **not** model-facing — the tool sets the bound (the `searchMaxResults` config, default 8) and passes it to the seam. | -| `web_fetch` | `url` (string), `timeout_ms` (number, optional) | Retrieves a specific URL. HTML bodies are rendered to markdown-ish text; text bodies pass through. A non-2xx status is reported, not an error. | +| `web_fetch` | `url` (string) | Retrieves a specific URL. HTML bodies are rendered to markdown-ish text; text bodies pass through. A non-2xx status is reported, not an error. The tool-call timeout is deployment policy (`dsh-timeout-policy`), not a model argument. | ## Config @@ -18,6 +18,10 @@ Each tool is registered independently; a product that wants only one disables th | `search` | `true` | Register `web_search`. | | `fetch` | `true` | Register `web_fetch`. | | `searchMaxResults` | `8` | Upper bound on sources returned by one `web_search` call (the seam truncates a longer provider list and flags it). | +| `fetchTimeoutMs` | `30000` | Cooperative tool-call timeout budget (ms) for `web_fetch`. | +| `searchTimeoutMs` | `30000` | Cooperative tool-call timeout budget (ms) for `web_search`. | + +`fetchTimeoutMs`/`searchTimeoutMs` declare each tool's cooperative timeout budget (attached as `ToolDefinition.timeoutMs`), enforced by [`@deepseek-ai/dsh-timeout-policy`](../../timeout/timeout-policy/README.md); the model-facing schema exposes no timeout argument. ```yaml - id: tool-web diff --git a/packages/web/tool-web/package.json b/packages/web/tool-web/package.json index 8c22afa9a8..80fd69dbc3 100644 --- a/packages/web/tool-web/package.json +++ b/packages/web/tool-web/package.json @@ -37,6 +37,7 @@ "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-tools": "workspace:^", + "@deepseek-ai/dsh-timeout-policy": "workspace:^", "@deepseek-ai/dsh-web": "workspace:^", "@deepseek-ai/dsh-web-fetch-local": "workspace:^", "@deepseek-ai/dsh-web-search-exa": "workspace:^", diff --git a/packages/web/tool-web/src/fetch.ts b/packages/web/tool-web/src/fetch.ts index 5f7334d952..571ce00797 100644 --- a/packages/web/tool-web/src/fetch.ts +++ b/packages/web/tool-web/src/fetch.ts @@ -3,6 +3,13 @@ * Execution goes through `ctx.web` — this module owns the model-facing schema, * argument validation, and PRESENTATION (HTML→markdown, truncation formatting), * while the fetch provider owns safe retrieval (transport, redirects, caps). + * + * The model-facing schema exposes NO timeout knob: the tool-call budget is + * deployment policy DECLARED via this package's `fetchTimeoutMs` config (attached + * as `ToolDefinition.timeoutMs`) and ENFORCED by `@deepseek-ai/dsh-timeout-policy` + * (a `tools/execute` wrapper), matching the reference-agent `WebFetch` shape. This + * tool just forwards the (possibly deadline-derived) `exec.signal` to `ctx.web`; + * the provider keeps its own timeout only as a resource backstop for direct callers. */ import type { Context } from 'cordis' @@ -14,16 +21,27 @@ import { assertNever } from '@deepseek-ai/dsh-llm' import type {} from '@deepseek-ai/dsh-system-prompt' import { htmlToMarkdown } from './html.ts' -/** Validate value constraints the schema DSL can't express. */ -export function parseFetchArgs(args: { url: string; timeout_ms?: number }): { url: string; timeoutMs?: number } { +/** + * Validate value constraints the schema DSL can't express: a non-blank `url`. + * Throws a plain `Error` otherwise. No timeout parameter — the tool-call budget + * is deployment policy declared via `fetchTimeoutMs` config and enforced by + * `@deepseek-ai/dsh-timeout-policy`, not a model argument. + * + * @param args - the schema-validated `web_fetch` arguments. + * @returns the arguments as the seam's request fields. + */ +export function parseFetchArgs(args: { url: string }): { url: string } { if (args.url.trim().length === 0) throw new Error('url must be a non-empty string') - if (args.timeout_ms !== undefined && (!Number.isFinite(args.timeout_ms) || args.timeout_ms <= 0)) { - throw new Error('timeout_ms must be a positive number') - } - return { url: args.url, ...args.timeout_ms !== undefined ? { timeoutMs: args.timeout_ms } : {} } + return { url: args.url } } -/** Render a fetched body to model-facing markdown text. */ +/** + * Render a fetched body to model-facing markdown text. + * + * @param body - the decoded body; `html` is converted via + * {@link htmlToMarkdown}, `text` passes through verbatim. + * @returns the text for the tool's output block. + */ export function renderBody(body: WebFetchBody): string { switch (body.kind) { case 'html': @@ -36,20 +54,38 @@ export function renderBody(body: WebFetchBody): string { } } -/** Format a fetch result as one model-facing text block. */ +/** + * Format a fetch result as one model-facing text block. + * + * @param result - the seam's fetch outcome. + * @returns a `Fetched (HTTP )` header, the rendered body, and a + * fetch-something-narrower notice when the provider truncated the content. + */ export function formatFetchOutput(result: WebFetchResult): string { const header = `Fetched ${result.url} (HTTP ${result.statusCode})` const footer = result.truncated ? '\n\n(Content truncated. Fetch a more specific URL or section for the full text.)' : '' return `${header}\n\n${renderBody(result.body)}${footer}` } -/** Pending-call presentation: a fetch card titled by the URL. */ -export function presentFetchCall(args: { url: string; timeout_ms?: number }): GenericCallView { +/** + * Pending-call presentation: a fetch card titled by the URL. + * + * @param args - the raw tool arguments; only `url` feeds the view. + * @returns the generic card view (`kind: 'fetch'`) shown while the call runs. + */ +export function presentFetchCall(args: { url: string }): GenericCallView { return { card: 'generic', title: args.url, kind: 'fetch', rawInput: args.url } } -/** Register the `web_fetch` tool and its system-prompt guidance. */ -export function applyWebFetchTool(ctx: Context): void { +/** + * Register the `web_fetch` tool and its system-prompt guidance. + * + * @param ctx - context whose `tools` and `systemPrompt` registries receive the + * registrations; both are effect-scoped and unregister on plugin dispose. + * @param timeoutMs - the cooperative tool-call budget (ms) attached as the tool's + * `ToolDefinition.timeoutMs` for `@deepseek-ai/dsh-timeout-policy` to enforce. + */ +export function applyWebFetchTool(ctx: Context, timeoutMs: number): void { ctx.systemPrompt.section({ name: 'tool:web_fetch', order: 111, @@ -61,12 +97,12 @@ export function applyWebFetchTool(ctx: Context): void { description: 'Fetch the content of a specific HTTP(S) URL and return it decoded to text.', parameters: { url: { type: 'string', required: true, description: 'The HTTP(S) URL to fetch.' }, - timeout_ms: { type: 'number', description: 'Optional fetch timeout in milliseconds (capped by the provider).' }, }, + timeoutMs, async execute(args, exec): Promise { const input = parseFetchArgs(args) const result = await ctx.web.fetch( - { url: input.url, ...input.timeoutMs !== undefined ? { timeoutMs: input.timeoutMs } : {} }, + { url: input.url }, exec.signal ? { signal: exec.signal } : undefined, ) return [{ type: 'text', text: formatFetchOutput(result) }] diff --git a/packages/web/tool-web/src/html.ts b/packages/web/tool-web/src/html.ts index 622be86fd5..d848ad6225 100644 --- a/packages/web/tool-web/src/html.ts +++ b/packages/web/tool-web/src/html.ts @@ -44,6 +44,10 @@ function safeFromCodePoint(code: number, fallback: string): string { * Convert an HTML document to a readable markdown-ish text approximation. * Best-effort and lossy by design — fidelity is the job of a future heavier * converter, not this fallback. + * + * @param html - the raw HTML source. + * @returns plain text with markdown headings, list bullets, and links; + * whitespace collapsed to at most one blank line and trimmed. */ export function htmlToMarkdown(html: string): string { let text = html diff --git a/packages/web/tool-web/src/index.ts b/packages/web/tool-web/src/index.ts index c0191c023b..0f948eacb6 100644 --- a/packages/web/tool-web/src/index.ts +++ b/packages/web/tool-web/src/index.ts @@ -33,6 +33,10 @@ export const name = 'tool-web' /** Services required by the web tool suite. */ export const inject = ['tools', 'web', 'systemPrompt'] +/** Default cooperative tool-call timeout budget (ms) for the web tools. */ +export const DEFAULT_WEB_TOOL_TIMEOUT_MS = 30_000 + +/** Plugin config: which web tools to register, the source cap, and per-tool budgets. */ export interface Config { /** Register `web_search`. Defaults to true. */ search?: boolean @@ -40,12 +44,18 @@ export interface Config { fetch?: boolean /** Upper bound on sources returned by one `web_search` call. */ searchMaxResults?: number + /** Cooperative timeout budget (ms) for `web_fetch`. Defaults to 30000. */ + fetchTimeoutMs?: number + /** Cooperative timeout budget (ms) for `web_search`. Defaults to 30000. */ + searchTimeoutMs?: number } export const Config: z = z.object({ search: z.boolean().default(true), fetch: z.boolean().default(true), searchMaxResults: z.number().default(WEB_SEARCH_MAX_RESULTS), + fetchTimeoutMs: z.number().default(DEFAULT_WEB_TOOL_TIMEOUT_MS), + searchTimeoutMs: z.number().default(DEFAULT_WEB_TOOL_TIMEOUT_MS), }) /** The shape after schemastery applies its defaults to every field. */ @@ -60,7 +70,10 @@ function assertPositiveInteger(name: string, value: number): void { /** * Register the enabled web tools. `search`/`fetch` default to true; a product - * that wants only one disables the other in config. The tools' disposers are + * that wants only one disables the other in config. Each tool's cooperative + * timeout budget (`fetchTimeoutMs`/`searchTimeoutMs`, default 30000) is resolved + * here and attached to the tool as `ToolDefinition.timeoutMs` for + * `@deepseek-ai/dsh-timeout-policy` to enforce. The tools' disposers are * fiber-scoped (the effect-based registries clean up on dispose), so no manual * teardown is needed. */ @@ -68,6 +81,8 @@ export function apply(ctx: Context, config: Config): void { // schemastery (Config) has already filled every defaulted field. const resolved = config as ResolvedConfig assertPositiveInteger('searchMaxResults', resolved.searchMaxResults) - if (resolved.search) applyWebSearchTool(ctx, resolved.searchMaxResults) - if (resolved.fetch) applyWebFetchTool(ctx) + assertPositiveInteger('fetchTimeoutMs', resolved.fetchTimeoutMs) + assertPositiveInteger('searchTimeoutMs', resolved.searchTimeoutMs) + if (resolved.search) applyWebSearchTool(ctx, resolved.searchMaxResults, resolved.searchTimeoutMs) + if (resolved.fetch) applyWebFetchTool(ctx, resolved.fetchTimeoutMs) } diff --git a/packages/web/tool-web/src/search.ts b/packages/web/tool-web/src/search.ts index 28e4e9a2e9..a7587d328b 100644 --- a/packages/web/tool-web/src/search.ts +++ b/packages/web/tool-web/src/search.ts @@ -20,7 +20,13 @@ import type {} from '@deepseek-ai/dsh-system-prompt' */ export const WEB_SEARCH_MAX_RESULTS = 8 -/** Validate value constraints the schema DSL can't express. */ +/** + * Validate value constraints the schema DSL can't express: a non-blank + * `query`. Throws a plain `Error` otherwise. + * + * @param args - the schema-validated `web_search` arguments. + * @returns the accepted arguments, passed through unchanged. + */ export function parseSearchArgs(args: { query: string }): { query: string } { if (args.query.trim().length === 0) throw new Error('query must be a non-empty string') return { query: args.query } @@ -38,7 +44,14 @@ function sourceLabel(url: string, title: string | undefined): string { } } -/** Format a search result as one model-facing text block. */ +/** + * Format a search result as one model-facing text block. + * + * @param result - the seam's search outcome. + * @returns the provider answer (when any), a markdown source list with snippet + * and date metadata (or `No results found.`), a refine-the-query note when + * truncated, and a standing cite-your-sources instruction. + */ export function formatSearchOutput(result: WebSearchResult): string { const parts: string[] = [] if (result.content !== undefined && result.content.length > 0) parts.push(result.content) @@ -62,13 +75,27 @@ export function formatSearchOutput(result: WebSearchResult): string { return parts.join('\n\n') } -/** Pending-call presentation: a search card titled by the query. */ +/** + * Pending-call presentation: a search card titled by the query. + * + * @param args - the raw tool arguments; only `query` feeds the view. + * @returns the generic card view (`kind: 'search'`) shown while the call runs. + */ export function presentSearchCall(args: { query: string }): GenericCallView { return { card: 'generic', title: args.query, kind: 'search', rawInput: args.query } } -/** Register the `web_search` tool and its system-prompt guidance. `maxResults` is the deployment's source cap. */ -export function applyWebSearchTool(ctx: Context, maxResults: number): void { +/** + * Register the `web_search` tool and its system-prompt guidance. + * + * @param ctx - context whose `tools` and `systemPrompt` registries receive the + * registrations; both are effect-scoped and unregister on plugin dispose. + * @param maxResults - the deployment's source cap, sent as every seam + * request's `maxResults`. + * @param timeoutMs - the cooperative tool-call budget (ms) attached as the tool's + * `ToolDefinition.timeoutMs` for `@deepseek-ai/dsh-timeout-policy` to enforce. + */ +export function applyWebSearchTool(ctx: Context, maxResults: number, timeoutMs: number): void { ctx.systemPrompt.section({ name: 'tool:web_search', order: 110, @@ -81,6 +108,7 @@ export function applyWebSearchTool(ctx: Context, maxResults: number): void { parameters: { query: { type: 'string', required: true, description: 'The search query.' }, }, + timeoutMs, async execute(args, exec): Promise { const input = parseSearchArgs(args) const result = await ctx.web.search( diff --git a/packages/web/tool-web/tests/integration.spec.ts b/packages/web/tool-web/tests/integration.spec.ts index 50ae6c5624..de804e2bcd 100644 --- a/packages/web/tool-web/tests/integration.spec.ts +++ b/packages/web/tool-web/tests/integration.spec.ts @@ -1,10 +1,11 @@ /** * Integration: the real fetch backend (`dsh-web-fetch-local`) + a real search * provider (`dsh-web-search-exa`) + the real seam (`dsh-web`) + the model tool - * (`dsh-tool-web`), exercised through `ctx.tools.execute()` — nothing bypasses - * the tool registry. Fetch hits a real loopback HTTP server (verifying the - * WORLD); search runs the real Exa provider over a stubbed global `fetch` (the - * network is the one boundary we mock). + * (`dsh-tool-web`) + the tool-call timeout policy (`dsh-timeout-policy`), + * exercised through `ctx.tools.execute()` — nothing bypasses the tool registry. + * Fetch hits a real loopback HTTP server (verifying the WORLD); search runs the + * real Exa provider over a stubbed global `fetch` (the network is the one + * boundary we mock). */ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' @@ -18,6 +19,7 @@ import WebService from '@deepseek-ai/dsh-web' import * as WebFetchLocal from '@deepseek-ai/dsh-web-fetch-local' import * as WebSearchExa from '@deepseek-ai/dsh-web-search-exa' import * as ToolWeb from '@deepseek-ai/dsh-tool-web' +import * as TimeoutPolicy from '@deepseek-ai/dsh-timeout-policy' type Handler = (req: IncomingMessage, res: ServerResponse) => void @@ -39,6 +41,11 @@ beforeEach(async () => { await ctx.plugin(WebService, { searchProvider: WebSearchExa.EXA_PROVIDER_ID, fetchProvider: WebFetchLocal.LOCAL_FETCH_PROVIDER_ID }) await ctx.plugin(WebFetchLocal, {}) await ctx.plugin(WebSearchExa, { apiKey: 'exa-key', baseURL: 'https://api.exa.test' }) + // The shipped deployment shape: the tool-call budget is declared by tool-web + // config (default 30s, attached as ToolDefinition.timeoutMs) and enforced by + // the zero-config timeout-policy plugin, set above the provider backstop so the + // policy normally wins. + await ctx.plugin(TimeoutPolicy) fiber = await ctx.plugin(ToolWeb) }) @@ -96,3 +103,70 @@ describe('web_search integration over the real Exa provider', () => { expect(out.content.map(b => b.text).join('')).toContain('[Result](https://result.test)') }) }) + +describe('tool-call timeout policy over the migrated web tools', () => { + it('neither model schema exposes a timeout parameter after the migration', () => { + const byName = new Map(ctx.tools.schemas().map(s => [s.name, s])) + const fetchParams = byName.get('web_fetch')!.parameters as { properties: Record } + const searchParams = byName.get('web_search')!.parameters as { properties: Record } + expect(Object.keys(fetchParams.properties)).toEqual(['url']) + expect('timeout_ms' in fetchParams.properties).toBe(false) + expect(Object.keys(searchParams.properties)).toEqual(['query']) + }) +}) + +describe('tool-call timeout returns TOOL_TIMEOUT (deadline wins over a slow fetch)', () => { + let slowServer: Server + let slowBase: string + let openSockets: ServerResponse[] + let tctx: Context + let tfiber: Awaited> + + beforeEach(async () => { + // A server that never responds: it holds the connection open until the + // client aborts. The cooperative deadline (via exec.signal → the fetch + // provider → undici) is what ends the call. + openSockets = [] + slowServer = createServer((_req, res) => { openSockets.push(res) }) + await new Promise(resolve => slowServer.listen(0, '127.0.0.1', resolve)) + slowBase = `http://127.0.0.1:${(slowServer.address() as AddressInfo).port}` + + tctx = new Context() + await tctx.plugin(SystemPrompt) + await tctx.plugin(ToolRegistry) + await tctx.plugin(WebService, { fetchProvider: WebFetchLocal.LOCAL_FETCH_PROVIDER_ID }) + // Provider backstop well ABOVE the tool-call budget, so the policy wins. + await tctx.plugin(WebFetchLocal, { timeoutMs: 30_000, maxTimeoutMs: 60_000 }) + await tctx.plugin(TimeoutPolicy) + // The tool-call budget is declared by tool-web config, enforced by the policy. + tfiber = await tctx.plugin(ToolWeb, { fetchTimeoutMs: 50 }) + }) + + afterEach(async () => { + for (const res of openSockets) res.destroy() + await tfiber.dispose() + await new Promise(resolve => slowServer.close(() => { resolve() })) + }) + + it('returns a structured TOOL_TIMEOUT (not the provider WEB_FETCH_TIMEOUT) when the tool-call budget wins', async () => { + const out = await tctx.tools.execute({ callId: CallId('slow-1'), name: 'web_fetch', arguments: { url: slowBase } }) + expect(out.isError).toBe(true) + // The outer tool-call deadline won: TOOL_TIMEOUT, owned by dsh-timeout-policy, + // NOT the provider's own WEB_FETCH_TIMEOUT (its 30s backstop never fired). + expect(out.error?.code).toBe('TOOL_TIMEOUT') + const text = out.content.map(b => (b.type === 'text' ? b.text : '')).join('') + expect(text).toContain('timed out after 50ms') + }) + + it('the provider backstop still protects a DIRECT ctx.web.fetch() call (no tool-call policy in that path)', async () => { + // A direct seam caller does not go through tools/execute, so the tool-call + // policy never applies; the provider's OWN timeout is the only budget. A + // short per-request hint proves the provider backstop is intact and classifies + // as WEB_FETCH_TIMEOUT (the provider-owned code), never TOOL_TIMEOUT. + const err = await tctx.web.fetch({ url: slowBase, timeoutMs: 50 }).then( + () => undefined, + (e: unknown) => e as { code?: string }, + ) + expect(err?.code).toBe('WEB_FETCH_TIMEOUT') + }) +}) diff --git a/packages/web/tool-web/tests/tool-web.spec.ts b/packages/web/tool-web/tests/tool-web.spec.ts index 924e0aaeb2..4bb2728df7 100644 --- a/packages/web/tool-web/tests/tool-web.spec.ts +++ b/packages/web/tool-web/tests/tool-web.spec.ts @@ -110,10 +110,9 @@ describe('fetch formatting', () => { expect(renderBody({ kind: 'html', content: '

y

' })).toBe('y') }) - it('validates url and timeout', () => { + it('validates url (non-empty), no timeout parameter', () => { expect(() => parseFetchArgs({ url: ' ' })).toThrow('non-empty') - expect(() => parseFetchArgs({ url: 'https://a.test', timeout_ms: -1 })).toThrow('positive') - expect(parseFetchArgs({ url: 'https://a.test', timeout_ms: 5 })).toEqual({ url: 'https://a.test', timeoutMs: 5 }) + expect(parseFetchArgs({ url: 'https://a.test' })).toEqual({ url: 'https://a.test' }) }) it('presents a fetch call as a fetch-kind card titled by the url', () => { @@ -249,7 +248,7 @@ describe('tool-web execution through the real registry', () => { expect('default' in ToolWeb).toBe(false) }) - it('executes web_fetch, forwarding timeout_ms and the abort signal to the seam', async () => { + it('executes web_fetch, forwarding the url (no timeout param) and the abort signal to the seam', async () => { const seen: { request?: { url: string; timeoutMs?: number }; signal?: AbortSignal | undefined } = {} const fetchProvider = { id: 'stub-fetch', @@ -262,13 +261,35 @@ describe('tool-web execution through the real registry', () => { } const { ctx, fiber } = await mountTools({ webConfig: { fetchProvider: 'stub-fetch' }, fetchProvider }) const controller = new AbortController() - const out = await ctx.tools.execute({ callId: CallId('fetch-1'), name: 'web_fetch', arguments: { url: 'https://a.test', timeout_ms: 1234 }, signal: controller.signal }) + const out = await ctx.tools.execute({ callId: CallId('fetch-1'), name: 'web_fetch', arguments: { url: 'https://a.test' }, signal: controller.signal }) expect(out.isError).toBe(false) - expect(seen.request).toEqual({ url: 'https://a.test', timeoutMs: 1234 }) + // The model schema exposes no timeout: the tool forwards only the url; the + // tool-call budget is owned by dsh-timeout-policy over exec.signal. + expect(seen.request).toEqual({ url: 'https://a.test' }) expect(seen.signal).toBe(controller.signal) await fiber.dispose() }) + it('executes web_fetch with no caller signal (forwards undefined to the seam)', async () => { + const seen: { signal?: AbortSignal | undefined; passedExec?: boolean } = {} + const fetchProvider = { + id: 'stub-fetch', + status: () => available, + fetch: (request: { url: string }, exec?: { signal?: AbortSignal }) => { + seen.passedExec = exec !== undefined + seen.signal = exec?.signal + return Promise.resolve({ providerId: 'stub-fetch', url: request.url, statusCode: 200, body: { kind: 'text' as const, content: 'ok' }, truncated: false }) + }, + } + const { ctx, fiber } = await mountTools({ webConfig: { fetchProvider: 'stub-fetch' }, fetchProvider }) + // No signal on the execution: the tool passes `undefined` (not `{ signal: undefined }`). + const out = await ctx.tools.execute({ callId: CallId('fetch-2'), name: 'web_fetch', arguments: { url: 'https://a.test' } }) + expect(out.isError).toBe(false) + expect(seen.passedExec).toBe(false) + expect(seen.signal).toBeUndefined() + await fiber.dispose() + }) + it('executes web_search, forwarding the abort signal to the seam', async () => { const seen: { signal?: AbortSignal | undefined } = {} const provider: WebSearchProvider = { @@ -328,3 +349,31 @@ describe('searchMaxResults is plugin config', () => { .rejects.toThrow(/tool-web: searchMaxResults must be a positive integer/) }) }) + +describe('tool-call timeout budget is plugin config', () => { + it('attaches the default 30s budget to web_fetch and web_search', async () => { + const { fiber, ctx } = await mountTools() + expect(ctx.tools.get('web_fetch')?.timeoutMs).toBe(30_000) + expect(ctx.tools.get('web_search')?.timeoutMs).toBe(30_000) + await fiber.dispose() + }) + + it('honors per-tool timeout overrides from config', async () => { + const { fiber, ctx } = await mountTools({ config: { fetchTimeoutMs: 60_000, searchTimeoutMs: 10_000 } }) + expect(ctx.tools.get('web_fetch')?.timeoutMs).toBe(60_000) + expect(ctx.tools.get('web_search')?.timeoutMs).toBe(10_000) + await fiber.dispose() + }) + + it.each([ + ['fetchTimeoutMs', { fetchTimeoutMs: 0 }], + ['searchTimeoutMs', { searchTimeoutMs: -5 }], + ])('rejects a non-positive-integer %s at load', async (key, config) => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(WebService, {}) + await expect(ctx.plugin(ToolWeb, config)) + .rejects.toThrow(new RegExp(`tool-web: ${key} must be a positive integer`)) + }) +}) diff --git a/packages/web/tool-web/tsconfig.json b/packages/web/tool-web/tsconfig.json index 463a18dee9..5226425ec6 100644 --- a/packages/web/tool-web/tsconfig.json +++ b/packages/web/tool-web/tsconfig.json @@ -12,6 +12,7 @@ { "path": "../../llm/llm" }, { "path": "../../core/tools" }, { "path": "../../core/system-prompt" }, + { "path": "../../timeout/timeout-policy" }, { "path": "../web" } ] } diff --git a/packages/web/web-fetch-local/README.md b/packages/web/web-fetch-local/README.md index 58db557581..9c2ef0030f 100644 --- a/packages/web/web-fetch-local/README.md +++ b/packages/web/web-fetch-local/README.md @@ -6,7 +6,9 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i ## Responsibility split -The provider owns **safe resource retrieval**: URL validation, HTTP transport, redirect policy, timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `@deepseek-ai/dsh-tool-web` owns **presentation** (HTML→markdown, truncation formatting). A non-2xx HTTP response is a *result* (status code + decoded body), not an error; `WebError` is reserved for failures to safely retrieve or represent the resource. +The provider owns **safe resource retrieval**: URL validation, HTTP transport, redirect policy, a resource-backstop timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `@deepseek-ai/dsh-tool-web` owns **presentation** (HTML→markdown, truncation formatting). A non-2xx HTTP response is a *result* (status code + decoded body), not an error; `WebError` is reserved for failures to safely retrieve or represent the resource. + +The provider's `timeoutMs`/`maxTimeoutMs` is a **resource backstop** for direct `ctx.web.fetch()` callers and misconfigured deployments — it is NOT the model-facing tool-call budget. The tool-call budget for `web_fetch` is deployment policy owned by [`@deepseek-ai/dsh-timeout-policy`](../../timeout/timeout-policy/README.md), which arms a per-call deadline on `exec.signal`. A shipped web-tool deployment sets the provider backstop **above** the `tool-timeout` budget, so the tool-call policy normally wins for model calls (returning `TOOL_TIMEOUT`); when the outer deadline signal reaches this provider first, it classifies as `WEB_ABORTED` and the outer wrapper replaces the result with `TOOL_TIMEOUT`. The provider's own `WEB_FETCH_TIMEOUT` only fires for a direct seam caller whose own budget elapsed. ## Transport hygiene @@ -24,8 +26,8 @@ The provider owns **safe resource retrieval**: URL validation, HTTP transport, r | `maxUrlLength` | `2048` | Maximum accepted request URL length. | | `maxResponseBytes` | `5_000_000` | Maximum response body size in bytes. | | `maxBodyChars` | `100_000` | Maximum decoded body length in characters. | -| `timeoutMs` | `30_000` | Default fetch timeout. | -| `maxTimeoutMs` | `120_000` | Upper bound for a per-request timeout override. | +| `timeoutMs` | `30_000` | Default fetch timeout — a resource backstop for direct `ctx.web.fetch()` callers, not the model-facing tool-call budget (that is `dsh-timeout-policy`). | +| `maxTimeoutMs` | `120_000` | Upper bound for a per-request timeout override (direct callers). | | `maxRedirects` | `5` | Maximum same-origin redirect hops (`0` follows none). | | `userAgent` | `deepseek-harness/…` | `User-Agent` header. | diff --git a/packages/web/web-fetch-local/package.json b/packages/web/web-fetch-local/package.json index 8d9a599a52..9b847db6f3 100644 --- a/packages/web/web-fetch-local/package.json +++ b/packages/web/web-fetch-local/package.json @@ -22,6 +22,7 @@ ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-timeout": "^0.0.1", "@deepseek-ai/dsh-web": "^0.0.1", "cordis": "^4.0.0-rc.6" }, @@ -29,6 +30,7 @@ "schemastery": "^3.18.0" }, "devDependencies": { + "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-web": "workspace:^", "cordis": "^4.0.0-rc.6" } diff --git a/packages/web/web-fetch-local/src/index.ts b/packages/web/web-fetch-local/src/index.ts index 7eb614f39a..b6f97bf0d9 100644 --- a/packages/web/web-fetch-local/src/index.ts +++ b/packages/web/web-fetch-local/src/index.ts @@ -30,6 +30,7 @@ export const name = 'web-fetch-local' /** The web seam this provider registers into. */ export const inject = ['web'] +/** Plugin config: the provider's transport and size limits plus its `User-Agent` (all defaulted). */ export interface Config { /** Maximum accepted request URL length. */ maxUrlLength?: number diff --git a/packages/web/web-fetch-local/src/policy.ts b/packages/web/web-fetch-local/src/policy.ts index 7a76bd1af1..f37697765a 100644 --- a/packages/web/web-fetch-local/src/policy.ts +++ b/packages/web/web-fetch-local/src/policy.ts @@ -16,6 +16,10 @@ export type FetchableKind = 'html' | 'text' * enforces before any network access: http(s) only, no embedded credentials, * bounded length. Returns the parsed `URL`. Throws {@link WebError} otherwise. * (SSRF / private-network blocking is deferred — see the package RFC.) + * + * @param input - the raw URL string from the fetch request. + * @param maxUrlLength - inclusive upper bound on `input`'s length. + * @returns the parsed `URL`. */ export function validateFetchUrl(input: string, maxUrlLength: number): URL { if (input.length > maxUrlLength) { @@ -40,6 +44,10 @@ export function validateFetchUrl(input: string, maxUrlLength: number): URL { * Two URLs are same-origin when scheme, hostname, and port match. A redirect * that crosses origins is refused so each new origin requires a fresh tool call * (and thus a fresh provider/permission decision). + * + * @param a - one of the two URLs to compare. + * @param b - the other URL to compare. + * @returns true when `a` and `b` share scheme, hostname, and port. */ export function isSameOrigin(a: URL, b: URL): boolean { return a.protocol === b.protocol && a.hostname === b.hostname && a.port === b.port @@ -49,6 +57,10 @@ export function isSameOrigin(a: URL, b: URL): boolean { * Classify a response `Content-Type` into a decodable body kind, or `undefined` * for an unsupported (e.g. binary) type. `text/html` and `application/xhtml+xml` * are `html`; other `text/*` plus a few structured text types are `text`. + * + * @param contentType - the raw `Content-Type` header, or `null` when the + * response carries none (unsupported). + * @returns the decodable kind, or `undefined` for an unsupported type. */ export function classifyContentType(contentType: string | null): FetchableKind | undefined { const mime = (contentType ?? '').replace(/;.*$/s, '').trim().toLowerCase() @@ -63,6 +75,10 @@ export function classifyContentType(contentType: string | null): FetchableKind | * or `undefined` when absent. The provider feeds this label to `TextDecoder` * so a non-UTF-8 response is decoded with its declared encoding rather than * silently mangled into replacement characters. + * + * @param contentType - the raw `Content-Type` header, or `null` when the + * response carries none. + * @returns the lower-cased charset label, or `undefined` when none is declared. */ export function parseCharset(contentType: string | null): string | undefined { const match = /;\s*charset\s*=\s*"?([^";]+)"?/i.exec(contentType ?? '') @@ -74,6 +90,10 @@ export function parseCharset(contentType: string | null): string | undefined { * none is declared. Throws {@link WebError} `WEB_UNSUPPORTED_CONTENT_TYPE` when * the label is present but not a charset `TextDecoder` recognizes — better to * fail loudly than return mojibake. + * + * @param charset - the declared charset label (from {@link parseCharset}), or + * `undefined` to default to UTF-8. + * @returns a decoder for the declared (or defaulted) encoding. */ export function decoderForCharset(charset: string | undefined): TextDecoder { if (charset === undefined) return new TextDecoder('utf-8') diff --git a/packages/web/web-fetch-local/src/provider.ts b/packages/web/web-fetch-local/src/provider.ts index 29b183b710..ed332c4508 100644 --- a/packages/web/web-fetch-local/src/provider.ts +++ b/packages/web/web-fetch-local/src/provider.ts @@ -1,6 +1,6 @@ /** * `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public - * HTTP(S) URL with the platform-native `fetch` (Node 24) and returns a status + * HTTP(S) URL with platform-native `fetch` at the repo's Node floor and returns a status * code plus bounded decoded content. It owns SAFE RESOURCE RETRIEVAL — URL * validation, redirect policy, timeout, abort, byte caps, charset decoding, * content-type classification, binary rejection — but NOT presentation @@ -21,6 +21,7 @@ import { WebError } from '@deepseek-ai/dsh-web' import type { WebFetchBody, WebFetchProvider, WebFetchRequest, WebFetchResult, WebProviderStatus } from '@deepseek-ai/dsh-web' +import { clampTimeout, deadline, timeoutOf } from '@deepseek-ai/dsh-timeout' import { classifyContentType, decoderForCharset, isSameOrigin, parseCharset, validateFetchUrl } from './policy.ts' /** Resolved provider limits (the plugin's schemastery Config supplies defaults). */ @@ -56,35 +57,25 @@ export class LocalFetchProvider implements WebFetchProvider { } async fetch(request: WebFetchRequest, exec?: { readonly signal?: AbortSignal }): Promise { - const timeoutMs = request.timeoutMs !== undefined - ? Math.min(request.timeoutMs, this.limits.maxTimeoutMs) - : this.limits.timeoutMs + if (exec?.signal?.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED') + const timeoutMs = clampTimeout(request.timeoutMs, this.limits.timeoutMs, this.limits.maxTimeoutMs) - // One controller drives both the caller's abort and our own timeout, so the - // network request and the streaming read both stop on either. - const controller = new AbortController() - const onAbort = (): void => { controller.abort() } - if (exec?.signal !== undefined) { - if (exec.signal.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED') - exec.signal.addEventListener('abort', onAbort, { once: true }) - } - const timer = setTimeout(() => { controller.abort(new WebError('web fetch timed out', 'WEB_FETCH_TIMEOUT')) }, timeoutMs) - - try { - return await this.followAndRead(request.url, controller) - } finally { - clearTimeout(timer) - if (exec?.signal !== undefined) exec.signal.removeEventListener('abort', onAbort) - } + // One deadline signal fuses the caller's abort with our own timeout, so the + // network request and the streaming read both stop on either. The timeout + // abort carries a TimeoutReason we recover afterward to classify the cause + // (translateAbortOrNetwork), instead of hand-rolling a controller + timer + + // reason-recovery dance. + using d = deadline(exec?.signal, timeoutMs, 'WEB_FETCH_TIMEOUT') + return await this.followAndRead(request.url, d.signal) } /** Follow same-origin redirects up to the hop cap, then read the final response. */ - private async followAndRead(initialUrl: string, controller: AbortController): Promise { + private async followAndRead(initialUrl: string, signal: AbortSignal): Promise { let currentUrl = validateFetchUrl(initialUrl, this.limits.maxUrlLength) let redirectsFollowed = 0 for (;;) { - const response = await this.requestOnce(currentUrl, controller) + const response = await this.requestOnce(currentUrl, signal) if (isRedirectStatus(response.status)) { // The redirect budget is enforced BEFORE this hop's target is resolved @@ -127,20 +118,20 @@ export class LocalFetchProvider implements WebFetchProvider { continue } - return await this.readBody(response, currentUrl, controller.signal) + return await this.readBody(response, currentUrl, signal) } } - private async requestOnce(url: URL, controller: AbortController): Promise { + private async requestOnce(url: URL, signal: AbortSignal): Promise { try { return await fetch(url, { method: 'GET', redirect: 'manual', headers: { 'user-agent': this.limits.userAgent, 'accept': 'text/html,application/xhtml+xml,text/*;q=0.9,application/json;q=0.8' }, - signal: controller.signal, + signal, }) } catch (error: unknown) { - throw translateAbortOrNetwork(error, controller.signal) + throw translateAbortOrNetwork(error, signal) } } @@ -255,24 +246,18 @@ function resolveRedirect(location: string, base: URL): URL { } /** - * Translate a thrown fetch/stream error into a `WebError`. Our own - * `WEB_FETCH_TIMEOUT` (passed to `controller.abort(reason)`) and any other - * already-typed `WebError` pass through; an `AbortError` becomes `WEB_ABORTED`, - * UNLESS the abort was our timeout — the body-read reader surfaces a generic - * `AbortError` rather than the abort reason, so we recover the timeout's - * `WebError` from `signal.reason`; anything else is a transport/network failure - * (`WEB_PROVIDER_ERROR`). + * Translate a thrown fetch/stream error into a `WebError`, classified by the + * deadline signal rather than the error's shape (which differs by phase: the + * request-phase `fetch` rejects with the abort reason, while the read-phase + * reader surfaces a bare `AbortError`). `timeoutOf(signal, 'WEB_FETCH_TIMEOUT')` + * recovering OUR reason means our timeout fired (`WEB_FETCH_TIMEOUT`); any other + * abort — an upstream cancel, or a foreign/outer deadline's timeout under + * nesting — is `WEB_ABORTED`; a throw with the signal NOT aborted is a + * transport/network failure (`WEB_PROVIDER_ERROR`). */ -function translateAbortOrNetwork(error: unknown, signal?: AbortSignal): WebError { - if (error instanceof WebError) return error - if (error instanceof DOMException && error.name === 'AbortError') { - // A timeout abort carries its WebError as the signal reason; honor the - // WEB_FETCH_TIMEOUT contract instead of reporting a generic cancellation. - // (Node rejects WITH the reason — the WebError branch above — so this only - // fires on a runtime that surfaces a bare AbortError while reason is set.) - /* v8 ignore next */ - if (signal?.reason instanceof WebError) return signal.reason - return new WebError('web fetch aborted', 'WEB_ABORTED', { cause: error }) - } +function translateAbortOrNetwork(error: unknown, signal: AbortSignal): WebError { + const timeout = timeoutOf(signal, 'WEB_FETCH_TIMEOUT') + if (timeout !== undefined) return new WebError('web fetch timed out', 'WEB_FETCH_TIMEOUT', { cause: timeout }) + if (signal.aborted) return new WebError('web fetch aborted', 'WEB_ABORTED', { cause: error }) return new WebError(`web fetch failed: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error }) } diff --git a/packages/web/web-fetch-local/tsconfig.json b/packages/web/web-fetch-local/tsconfig.json index aa7c949fec..c6fb75a5c1 100644 --- a/packages/web/web-fetch-local/tsconfig.json +++ b/packages/web/web-fetch-local/tsconfig.json @@ -17,6 +17,9 @@ { "path": "../../../vendor/schemastery" }, + { + "path": "../../util/timeout" + }, { "path": "../web" } diff --git a/packages/web/web-search-deepseek/src/index.ts b/packages/web/web-search-deepseek/src/index.ts index c993fa8808..2c3f0ede3b 100644 --- a/packages/web/web-search-deepseek/src/index.ts +++ b/packages/web/web-search-deepseek/src/index.ts @@ -44,6 +44,7 @@ export const name = 'web-search-deepseek' /** The web seam this provider registers into. */ export const inject = ['web'] +/** Plugin config (all optional — `apply` fills env-var and constant defaults). */ export interface Config { /** DeepSeek API key. Falls back to `$DEEPSEEK_API_KEY`. Empty → unavailable. */ apiKey?: string diff --git a/packages/web/web-search-deepseek/src/provider.ts b/packages/web/web-search-deepseek/src/provider.ts index 40566b4f75..fca8620e2a 100644 --- a/packages/web/web-search-deepseek/src/provider.ts +++ b/packages/web/web-search-deepseek/src/provider.ts @@ -12,7 +12,7 @@ * `web_search_tool_result` block (native search did not trigger), it throws * `WEB_PROVIDER_ERROR` rather than degrading to prose-scraping. * - * Network requests use platform-native `fetch` (Node 24), mirroring + * Network requests use platform-native `fetch` at the repo's Node floor, mirroring * `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service. * The Anthropic wire shape is a provider-private detail and does NOT make this * provider depend on `ctx.llm`. @@ -62,6 +62,7 @@ export const DEEPSEEK_DEFAULT_MAX_USES = 5 /** Attribution header sent on every request. Bump with the package version. */ const USER_AGENT = 'deepseek-harness/0.0.1' +/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */ export interface DeepSeekSearchProviderOptions { /** DeepSeek API key. Empty/absent → `status()` reports `missing-credential`. */ apiKey: string @@ -82,6 +83,9 @@ export interface DeepSeekSearchProviderOptions { * is the snippet surface: Anthropic `web_search_result` items carry * `url`/`title`/`page_age` but typically NO inline snippet — the excerpt lives * in a separate `text` block's citation, keyed by `url` (first occurrence wins). + * + * @param blocks - the response's content blocks; non-`text` blocks are skipped. + * @returns the `url → cited_text` map (empty when no citations are present). */ export function citationSnippets(blocks: readonly ContentBlock[]): Map { const map = new Map() @@ -106,6 +110,10 @@ export function citationSnippets(blocks: readonly ContentBlock[]): Map highlight.trim().length > 0) @@ -66,7 +71,14 @@ export function mapExaResult(result: ExaResult): WebSearchSource | undefined { } } -/** Map an Exa response envelope to a normalized search result. */ +/** + * Map an Exa response envelope to a normalized search result. + * + * @param query - the original request query, echoed on the result. + * @param response - the parsed `POST /search` response body. + * @returns the normalized result; snippet-less entries are dropped + * ({@link mapExaResult}). + */ export function mapExaResponse(query: string, response: ExaSearchResponse): WebSearchResult { const sources = (response.results ?? []) .map(mapExaResult) diff --git a/packages/web/web-search-perplexity/src/index.ts b/packages/web/web-search-perplexity/src/index.ts index 3d375eaabb..b71d5052d6 100644 --- a/packages/web/web-search-perplexity/src/index.ts +++ b/packages/web/web-search-perplexity/src/index.ts @@ -29,6 +29,7 @@ export const name = 'web-search-perplexity' /** The web seam this provider registers into. */ export const inject = ['web'] +/** Plugin config (all optional — `apply` fills env-var and constant defaults). */ export interface Config { /** Perplexity API key. Falls back to `$PERPLEXITY_API_KEY`. Empty → unavailable. */ apiKey?: string diff --git a/packages/web/web-search-perplexity/src/provider.ts b/packages/web/web-search-perplexity/src/provider.ts index ed72ea82c3..a008438c9a 100644 --- a/packages/web/web-search-perplexity/src/provider.ts +++ b/packages/web/web-search-perplexity/src/provider.ts @@ -5,7 +5,7 @@ * structured `search_results[]` for `sources[]`, falling back to the URL-only * `citations[]` when `search_results` is absent. * - * Network requests use platform-native `fetch` (Node 24), mirroring + * Network requests use platform-native `fetch` at the repo's Node floor, mirroring * `@deepseek-ai/dsh-llm-deepseek`'s adapter. The OpenAI-compatible request shape * is a provider-private detail and does NOT make this provider depend on * `ctx.llm`. @@ -41,6 +41,7 @@ export type PerplexityRecency = 'day' | 'week' | 'month' | 'year' /** Attribution header sent on every request. Bump with the package version. */ const USER_AGENT = 'deepseek-harness/0.0.1' +/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */ export interface PerplexitySearchProviderOptions { /** Perplexity API key. Empty/absent → `status()` reports `missing-credential`. */ apiKey: string @@ -54,7 +55,12 @@ export interface PerplexitySearchProviderOptions { searchRecency?: PerplexityRecency } -/** Map one structured Perplexity search result to a normalized source. */ +/** + * Map one structured Perplexity search result to a normalized source. + * + * @param result - one entry of the response's `search_results[]`. + * @returns the normalized source; blank fields are omitted rather than set empty. + */ export function mapPerplexityResult(result: PerplexitySearchResult): WebSearchSource { return { url: result.url, @@ -68,6 +74,10 @@ export function mapPerplexityResult(result: PerplexitySearchResult): WebSearchSo * Map a Perplexity response envelope to a normalized search result. Prefers * structured `search_results[]`; falls back to URL-only `citations[]` (those * sources carry just a `url`) only when `search_results` is absent. + * + * @param query - the original request query, echoed on the result. + * @param response - the parsed chat-completions response body. + * @returns the normalized result; `content` is omitted when the answer is empty. */ export function mapPerplexityResponse(query: string, response: PerplexityResponse): WebSearchResult { const content = response.choices?.[0]?.message?.content diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d5fb68c746..b538e82908 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -21,8 +21,8 @@ importers: specifier: ^4.0.4 version: 4.0.4 '@types/node': - specifier: ^25.3.5 - version: 25.9.3 + specifier: ^22.20.0 + version: 22.20.0 '@vitest/coverage-v8': specifier: ^4.1.8 version: 4.1.8(vitest@4.1.8) @@ -70,10 +70,10 @@ importers: version: 8.61.0(eslint@10.5.0(jiti@2.7.0))(typescript@6.0.3) vite-tsconfig-paths: specifier: ^6.1.1 - version: 6.1.1(typescript@6.0.3)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + version: 6.1.1(typescript@6.0.3)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) vitest: specifier: ^4.1.8 - version: 4.1.8(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + version: 4.1.8(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) packages/bash/bash: devDependencies: @@ -93,6 +93,9 @@ importers: '@deepseek-ai/dsh-bash': specifier: workspace:^ version: link:../bash + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) @@ -127,6 +130,25 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/code-runtime/code-runtime: + devDependencies: + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/code-runtime/code-runtime-worker: + dependencies: + schemastery: + specifier: ^3.18.0 + version: 3.18.0 + devDependencies: + '@deepseek-ai/dsh-code-runtime': + specifier: workspace:^ + version: link:../code-runtime + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/compact/compact: devDependencies: '@deepseek-ai/dsh-llm': @@ -381,6 +403,34 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/guard/repeat-tool-guard: + dependencies: + schemastery: + specifier: ^3.18.0 + version: 3.18.0 + devDependencies: + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:^ + version: link:../../core/agent-loop + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/hooks/hook-protocol: devDependencies: '@deepseek-ai/dsh-bash': @@ -748,6 +798,22 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/support/acp-snapshot: + dependencies: + '@agentclientprotocol/sdk': + specifier: 0.25.1 + version: 0.25.1(zod@4.4.3) + tsx: + specifier: ^4.22.4 + version: 4.22.4 + vitest: + specifier: ^4.1.8 + version: 4.1.8(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + devDependencies: + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/support/invariants: devDependencies: '@deepseek-ai/dsh-agent': @@ -797,6 +863,21 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/timeout/timeout-policy: + devDependencies: + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/todo/tool-todo: devDependencies: '@deepseek-ai/dsh-agent': @@ -899,6 +980,9 @@ importers: '@deepseek-ai/dsh-session-persistence-jsonl': specifier: workspace:^ version: link:../../session-persistence/session-persistence-jsonl + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader) @@ -947,6 +1031,9 @@ importers: '@deepseek-ai/dsh-session-persistence-jsonl': specifier: workspace:^ version: link:../../session-persistence/session-persistence-jsonl + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader) @@ -960,6 +1047,12 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/util/timeout: + devDependencies: + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/web/tool-web: dependencies: schemastery: @@ -978,6 +1071,9 @@ importers: '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt + '@deepseek-ai/dsh-timeout-policy': + specifier: workspace:^ + version: link:../../timeout/timeout-policy '@deepseek-ai/dsh-tools': specifier: workspace:^ version: link:../../core/tools @@ -1013,6 +1109,9 @@ importers: specifier: ^3.18.0 version: 3.18.0 devDependencies: + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout '@deepseek-ai/dsh-web': specifier: workspace:^ version: link:../web @@ -2348,6 +2447,9 @@ packages: '@types/ms@2.1.0': resolution: {integrity: sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA==} + '@types/node@22.20.0': + resolution: {integrity: sha512-QWlFW2wf3nTjC13/DqRnBpR4ZO36VJH/JVBkA/vcnmbTBNQIlnObqyqZE1tUR7+Ni23Lda8R1BxMfbXRpCUx5g==} + '@types/node@25.9.3': resolution: {integrity: sha512-603BddQMv3pUcr4U2dhujk83N2tTDVr/34wII2B6bJy6g+8WD6yUb11jszNs0gdi4PesVWl7ABt8nYMVpnLUcg==} @@ -3814,6 +3916,9 @@ packages: unconfig-core@7.5.0: resolution: {integrity: sha512-Su3FauozOGP44ZmKdHy2oE6LPjk51M/TRRjHv2HNCWiDvfvCoxC2lno6jevMA91MYAdCdwP05QnWdWpSbncX/w==} + undici-types@6.21.0: + resolution: {integrity: sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==} + undici-types@7.24.6: resolution: {integrity: sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg==} @@ -5083,6 +5188,10 @@ snapshots: '@types/ms@2.1.0': {} + '@types/node@22.20.0': + dependencies: + undici-types: 6.21.0 + '@types/node@25.9.3': dependencies: undici-types: 7.24.6 @@ -5206,7 +5315,7 @@ snapshots: obug: 2.1.3 std-env: 4.1.0 tinyrainbow: 3.1.0 - vitest: 4.1.8(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + vitest: 4.1.8(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) '@vitest/expect@4.1.8': dependencies: @@ -5217,6 +5326,14 @@ snapshots: chai: 6.2.2 tinyrainbow: 3.1.0 + '@vitest/mocker@4.1.8(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))': + dependencies: + '@vitest/spy': 4.1.8 + estree-walker: 3.0.3 + magic-string: 0.30.21 + optionalDependencies: + vite: 8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0) + '@vitest/mocker@4.1.8(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))': dependencies: '@vitest/spy': 4.1.8 @@ -6819,6 +6936,8 @@ snapshots: '@quansync/fs': 1.0.0 quansync: 1.0.0 + undici-types@6.21.0: {} + undici-types@7.24.6: {} undici@7.28.0: {} @@ -6848,16 +6967,31 @@ snapshots: uuid@14.0.1: {} - vite-tsconfig-paths@6.1.1(typescript@6.0.3)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): + vite-tsconfig-paths@6.1.1(typescript@6.0.3)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): dependencies: debug: 4.4.3 globrex: 0.1.2 tsconfck: 3.1.6(typescript@6.0.3) - vite: 8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0) + vite: 8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0) transitivePeerDependencies: - supports-color - typescript + vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0): + dependencies: + lightningcss: 1.32.0 + picomatch: 4.0.4 + postcss: 8.5.15 + rolldown: 1.0.3 + tinyglobby: 0.2.17 + optionalDependencies: + '@types/node': 22.20.0 + esbuild: 0.28.1 + fsevents: 2.3.3 + jiti: 2.7.0 + tsx: 4.22.4 + yaml: 2.9.0 + vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0): dependencies: lightningcss: 1.32.0 @@ -6873,6 +7007,35 @@ snapshots: tsx: 4.22.4 yaml: 2.9.0 + vitest@4.1.8(@types/node@22.20.0)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): + dependencies: + '@vitest/expect': 4.1.8 + '@vitest/mocker': 4.1.8(vite@8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) + '@vitest/pretty-format': 4.1.8 + '@vitest/runner': 4.1.8 + '@vitest/snapshot': 4.1.8 + '@vitest/spy': 4.1.8 + '@vitest/utils': 4.1.8 + es-module-lexer: 2.1.0 + expect-type: 1.3.0 + magic-string: 0.30.21 + obug: 2.1.3 + pathe: 2.0.3 + picomatch: 4.0.4 + std-env: 4.1.0 + tinybench: 2.9.0 + tinyexec: 1.2.4 + tinyglobby: 0.2.17 + tinyrainbow: 3.1.0 + vite: 8.0.16(@types/node@22.20.0)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0) + why-is-node-running: 2.3.0 + optionalDependencies: + '@types/node': 22.20.0 + '@vitest/coverage-v8': 4.1.8(vitest@4.1.8) + jsdom: 29.1.1 + transitivePeerDependencies: + - msw + vitest@4.1.8(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(jsdom@29.1.1)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)): dependencies: '@vitest/expect': 4.1.8 diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index f579169ab0..4d7d0802c7 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -105,11 +105,26 @@ const dshBinPackageFiles = [ 'src', ] as const +// Packages that ship a worker-thread entry as a sibling runtime bundle +// (lib/worker.js, its own tsdown entry): the bootstrap is loaded via +// `new Worker(new URL('./worker.js', import.meta.url))`, so it cannot live +// inside the index bundle and must be published alongside it. +const workerEntryPackages = new Set(['@deepseek-ai/dsh-code-runtime-worker']) + +const dshWorkerPackageFiles = [ + 'lib/index.js', + 'lib/worker.js', + 'lib/types/**/*.d.ts', + 'lib/types/**/*.d.ts.map', + 'src', +] as const + function sameStringList(actual: readonly string[] | undefined, expected: readonly string[]): boolean { return !!actual && actual.length === expected.length && actual.every((value, index) => value === expected[index]) } function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { + if (manifest.name && workerEntryPackages.has(manifest.name)) return dshWorkerPackageFiles return manifest.bin ? dshBinPackageFiles : dshPackageFiles } diff --git a/scripts/doc-budgets.manifest.json b/scripts/doc-budgets.manifest.json index 353a3acd28..337fc57763 100644 --- a/scripts/doc-budgets.manifest.json +++ b/scripts/doc-budgets.manifest.json @@ -1,11 +1,11 @@ { - "AGENTS.md": 1660, + "AGENTS.md": 1691, "docs/AGENTS.md": 1315, - "docs/architecture.md": 1630, + "docs/architecture.md": 1640, "docs/cordis-primer.md": 550, "docs/defensive-patterns.md": 550, "docs/testing.md": 800, "examples/AGENTS.md": 610, "packages/AGENTS.md": 450, - "packages/README.md": 605 + "packages/README.md": 610 } diff --git a/scripts/doc-typecheck.ts b/scripts/doc-typecheck.ts index 6f40cccd0a..e57f3710ee 100644 --- a/scripts/doc-typecheck.ts +++ b/scripts/doc-typecheck.ts @@ -25,9 +25,8 @@ */ import { execFileSync } from 'node:child_process' -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { globSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import { join, relative, resolve } from 'node:path' -import { glob } from 'node:fs/promises' import ts from 'typescript' const root = resolve(import.meta.dirname, '..') @@ -139,7 +138,7 @@ const markdownGlobs = ['README.md', 'docs/**/*.md', 'packages/*/*.md', 'packages const files: string[] = [] for (const pattern of markdownGlobs) { - for await (const match of glob(pattern, { cwd: root })) files.push(resolve(root, match)) + for (const match of globSync(pattern, { cwd: root })) files.push(resolve(root, match)) } files.sort() diff --git a/scripts/gen-config-catalog.ts b/scripts/gen-config-catalog.ts index 1594f5b62b..eca36286ae 100644 --- a/scripts/gen-config-catalog.ts +++ b/scripts/gen-config-catalog.ts @@ -68,7 +68,8 @@ import { globSync, readFileSync, writeFileSync } from 'node:fs' import { dirname, resolve } from 'node:path' import ts from 'typescript' -import { LINK_MAP, parseJsDoc, pointer, rawJsDoc } from './gen-cordis-catalog.ts' +import { LINK_MAP } from './gen-cordis-catalog.ts' +import { parseJsDoc, pointer, rawJsDoc } from './jsdoc.ts' const root = resolve(import.meta.dirname, '..') const OUT = 'docs/config-catalog.md' diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index a64d9bbba4..6220006ede 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -40,7 +40,9 @@ * a stale `@param` naming no real parameter errors. Violations aggregate into * ONE error listing every offender. The tags are enforcement-only: parseJsDoc * stops prose at the first block tag, so they never change the rendered - * catalog. The INHERITED + * catalog. The parsing + check helpers live in `scripts/jsdoc.ts`, shared with + * the whole-export-surface gate (`scripts/verify-export-jsdoc.ts`) so + * "documented" means the same thing on both surfaces. The INHERITED * tier (cordis core + loader/hmr/timer) is pinned vendor source a plugin author * also sees; it is rendered tersely (name + one-line + source pointer) from a * curated table in this script, NOT elevated to the harness tier's prominence. @@ -53,6 +55,7 @@ import { globSync, readFileSync, writeFileSync } from 'node:fs' import { resolve } from 'node:path' import ts from 'typescript' +import { checkParams, checkReturns, parseJsDoc, parseTags, pointer, rawJsDoc, reportViolations, type Mode } from './jsdoc.ts' const root = resolve(import.meta.dirname, '..') const OUT_EVENTS = 'docs/cordis-catalog/events.md' @@ -62,9 +65,6 @@ const OUT_SERVICES = 'docs/cordis-catalog/services.md' * doc-typecheck, since a bare signature fragment is not standalone-compilable). */ const FENCE = 'ts cordis-catalog' -/** A dispatch mode, rendered as the badge after an event name. */ -type Mode = 'emit' | 'waterfall' | 'parallel' | 'serial' - /** * Cross-link map: a type name that appears in a signature → the * core-data-structures page that documents it (path relative to the catalogs' @@ -97,6 +97,8 @@ export const LINK_MAP: Record = { BashRunResult: 'bash.md', BashTask: 'bash.md', BashTaskRead: 'bash.md', + CodeRunRequest: 'code-runtime.md', + CodeRunResult: 'code-runtime.md', FsEditOutcome: 'filesystem.md', FsEditRequest: 'filesystem.md', FsInfo: 'filesystem.md', @@ -148,135 +150,6 @@ interface InheritedEntry { source: string } -/** Repo-relative source pointer `file:line` for a node's first character. - * Shared with `gen-config-catalog.ts`. */ -export function pointer(rel: string, sf: ts.SourceFile, node: ts.Node): string { - const { line } = sf.getLineAndCharacterOfPosition(node.getStart(sf)) - return `${rel}:${line + 1}` -} - -/** The raw `/** … *​/` JSDoc block immediately preceding a node, or '' if none. - * Shared with `gen-config-catalog.ts`. */ -export function rawJsDoc(text: string, node: ts.Node): string { - const ranges = ts.getLeadingCommentRanges(text, node.getFullStart()) ?? [] - const jsdoc = ranges.filter(r => text.slice(r.pos, r.pos + 3) === '/**').at(-1) - return jsdoc ? text.slice(jsdoc.pos, jsdoc.end) : '' -} - -/** - * Parse a raw JSDoc block into description prose + the `@mode` tag (when - * present). Output obeys the repo's markdown conventions so the generated file - * passes verify-md-wrap: each prose paragraph collapses to ONE physical line, - * and a `-` bullet list is preserved with each item on its own single line - * (continuation lines folded in). `{@link Foo}` unwraps to `Foo`. Description - * prose ends at the FIRST block tag (standard JSDoc semantics): tag lines and - * their continuation lines are never prose, so `@param`/`@returns` blocks are - * invisible to the rendered catalog. Shared with `gen-config-catalog.ts` - * (which uses only the prose-presence half). - */ -export function parseJsDoc(raw: string): { doc: string; mode: Mode | null } { - const inner = raw - .replace(/^\/\*\*/, '') - .replace(/\*\/$/, '') - .split('\n') - .map(l => l.replace(/^\s*\*?\s?/, '').replace(/\s+$/, '')) - let mode: Mode | null = null - let inTags = false - const blocks: string[] = [] - let para: string[] = [] - let list: string[] = [] - let item: string[] = [] - const join = (parts: string[]): string => parts.join(' ').replace(/\s+/g, ' ').trim() - const flushItem = (): void => { - if (item.length) list.push(join(item)) - item = [] - } - const flushList = (): void => { - flushItem() - if (list.length) blocks.push(list.join('\n')) // one block, items on own lines - list = [] - } - const flushPara = (): void => { - flushList() - if (para.length) blocks.push(join(para)) - para = [] - } - for (const line of inner) { - const m = /^@mode\s+(emit|waterfall|parallel|serial)\s*$/.exec(line) - if (m) { mode = m[1] as Mode; flushPara(); inTags = true; continue } - if (line.startsWith('@')) { flushPara(); inTags = true; continue } - if (inTags) continue // block-tag territory: continuations are never prose - if (line.trim() === '') { flushPara(); continue } - if (/^-\s+/.test(line)) { - // A list item starts: a pending paragraph (e.g. an intro line directly - // above the list, no blank between) flushes FIRST so it renders above. - flushItem() - if (para.length) { blocks.push(join(para)); para = [] } - item.push(line) - continue - } - if (item.length) { item.push(line); continue } // continuation of current item - para.push(line) - } - flushPara() - const doc = blocks.join('\n\n').replace(/\{@link\s+([^}]+)\}/g, '$1').trim() - return { doc, mode } -} - -/** - * Parse the block tags of a raw JSDoc comment for the completeness checks: - * every `@param name — description` entry plus the `@returns` description. - * Standard JSDoc block-tag semantics — a tag's description runs across - * continuation lines until the next tag or a blank line, and the `-`/`—` - * separator after a param name is optional. `[name]` optional-brackets unwrap - * to `name`. Rendering never sees these: parseJsDoc stops prose at the first - * block tag. - */ -function parseTags(raw: string): { params: Map; returns: string | null } { - const inner = raw - .replace(/^\/\*\*/, '') - .replace(/\*\/$/, '') - .split('\n') - .map(l => l.replace(/^\s*\*?\s?/, '').replace(/\s+$/, '')) - const params = new Map() - let returns: string | null = null - let sink: ((text: string) => void) | null = null - for (const line of inner) { - const param = /^@param\s+(\[?[\w$]+\]?)\s*(?:[-—–]\s*)?(.*)$/.exec(line) - if (param) { - const name = (param[1] ?? '').replace(/^\[|\]$/g, '') - let acc = param[2] ?? '' - params.set(name, acc) - sink = (t) => { acc = acc ? `${acc} ${t}` : t; params.set(name, acc) } - continue - } - const ret = /^@returns?(?:\s+[-—–]?\s*(.*))?$/.exec(line) - if (ret) { - let acc = ret[1] ?? '' - returns = acc - sink = (t) => { acc = acc ? `${acc} ${t}` : t; returns = acc } - continue - } - if (line.startsWith('@') || line.trim() === '') { sink = null; continue } - sink?.(line.trim()) - } - return { params, returns } -} - -/** - * Throw one aggregate error for every completeness violation a walk collected. - * Aggregation (vs the fail-fast the @mode check used to do) is deliberate: a - * remediation pass sees the whole list at once instead of replaying the gate - * once per offender. - */ -function reportViolations(violations: string[]): void { - if (violations.length === 0) return - throw new Error( - `gen-cordis-catalog: ${violations.length} JSDoc completeness violation(s) (see AGENTS.md):\n` - + violations.map(v => ` ${v}`).join('\n'), - ) -} - /** Find the `declare module 'cordis'` body in a source file, or null. */ function cordisModuleBody(sf: ts.SourceFile): ts.ModuleBlock | null { for (const stmt of sf.statements) { @@ -339,27 +212,13 @@ export function collectEvents(scanRoot: string = root): EventEntry[] { // (mode machinery, documented once by @mode semantics). Documenting an // exempt parameter anyway is allowed — only absence is checked. const { params } = parseTags(raw) - for (const p of member.parameters) { - if (!ts.isIdentifier(p.name)) { - violations.push(`${where}: parameter '${p.name.getText(sf)}' is a binding pattern; the event surface needs simple identifier parameters so @param can name them.`) - continue - } - const pname = p.name.text - if (pname === 'this' || (hasNext && p === last)) continue - const desc = params.get(pname) - if (desc === undefined) violations.push(`${where} is missing @param ${pname}.`) - else if (!desc.trim()) violations.push(`${where}: @param ${pname} has an empty description.`) - } - for (const tag of params.keys()) { - if (!member.parameters.some(p => ts.isIdentifier(p.name) && p.name.text === tag)) { - violations.push(`${where}: @param ${tag} does not match any parameter (stale tag?).`) - } - } + checkParams(where, 'event', member.parameters, params, sf, + p => (ts.isIdentifier(p.name) && p.name.text === 'this') || (hasNext && p === last), violations) if (mode) entries.push({ name, scope: name.split('/')[0] ?? name, signature, mode, doc, source: src }) } } } - reportViolations(violations) + reportViolations('gen-cordis-catalog', violations) return entries } @@ -420,35 +279,12 @@ export function collectServices(scanRoot: string = root): ServiceEntry[] { if (!raw) { violations.push(`${where} has no JSDoc.`); continue } if (!parseJsDoc(raw).doc) violations.push(`${where} has no description prose above its block tags.`) const { params, returns } = parseTags(raw) - // Every parameter needs a non-empty @param; a `this` receiver - // annotation is not payload and is exempt. - for (const p of member.parameters) { - if (!ts.isIdentifier(p.name)) { - violations.push(`${where}: parameter '${p.name.getText(sf)}' is a binding pattern; the service surface needs simple identifier parameters so @param can name them.`) - continue - } - const pname = p.name.text - if (pname === 'this') continue - const desc = params.get(pname) - if (desc === undefined) violations.push(`${where} is missing @param ${pname}.`) - else if (!desc.trim()) violations.push(`${where}: @param ${pname} has an empty description.`) - } - for (const tag of params.keys()) { - if (!member.parameters.some(p => ts.isIdentifier(p.name) && p.name.text === tag)) { - violations.push(`${where}: @param ${tag} does not match any parameter (stale tag?).`) - } - } - // A non-void result needs a non-empty @returns. The return type must be - // ANNOTATED: a pure-AST walk cannot classify an inferred return. On a - // `void`/`Promise` method @returns stays optional (resolution - // timing can be worth documenting), never required. - const rt = member.type?.getText(sf).replace(/\s+/g, ' ') - if (rt === undefined) { - violations.push(`${where} has no return type annotation; annotate it explicitly so the gate can classify the result.`) - } else if (!/^(void|Promise)$/.test(rt)) { - if (returns === null) violations.push(`${where} is missing @returns (return type: ${rt}).`) - else if (!returns.trim()) violations.push(`${where}: @returns has an empty description.`) - } + // Every parameter needs a non-empty @param (`this` receiver exempt), + // and a non-void ANNOTATED result needs a non-empty @returns — the + // shared checkers carry the exact contract. + checkParams(where, 'service', member.parameters, params, sf, + p => ts.isIdentifier(p.name) && p.name.text === 'this', violations) + checkReturns(where, member.type, returns, sf, violations) } entries.push({ key, @@ -460,7 +296,7 @@ export function collectServices(scanRoot: string = root): ServiceEntry[] { }) } } - reportViolations(violations) + reportViolations('gen-cordis-catalog', violations) return entries.sort((a, b) => a.key.localeCompare(b.key)) } diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index 66cb102b00..14396a6645 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -149,6 +149,15 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['tool-bash', 'hooks-claude', 'hooks-codex'], note: 'The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors can replace bash-local.', }, + { + key: 'codeRuntime', + pkg: 'code-runtime', + title: 'Code-execution seam', + mode: 'seam', + implementations: ['code-runtime-worker'], + consumers: [], + note: 'Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the Code Mode RFC specifies the worker-thread backend and the tool-registry consumer).', + }, { key: 'fs', pkg: 'fs', @@ -621,7 +630,7 @@ function renderToolPipeline(): string { const maintenance = 'curated Mermaid flow; exact tool schemas and event signatures live in generated catalogs' return [ ...generatedHeader('Tool Execution Pipeline'), - 'This graph shows where policy, hooks, sandboxing, filesystem guards, result rewriting, and UI rendering fit without changing the loop. The key extension points are the `tools/pre-execute` and `tools/post-execute` waterfalls.', + 'This graph shows where policy, hooks, sandboxing, filesystem guards, result rewriting, and UI rendering fit without changing the loop. The key extension points are the `tools/pre-execute`, `tools/execute`, and `tools/post-execute` waterfalls.', '', '```mermaid', 'flowchart TD', @@ -630,6 +639,7 @@ function renderToolPipeline(): string { ' presentCall["UI pending card
presentCall(args)"]', ` pre["${mermaidCode('tools/pre-execute')} waterfall
hooks, permission, sandbox"]`, ' denied["deny or ask
tool body skipped"]', + ` around["${mermaidCode('tools/execute')} waterfall
timeout, retry, metrics (around dispatch)"]`, ' toolBody["Registered tool execute() body"]', ` fsGate["${mermaidCode('fs/write-intent')} or ${mermaidCode('fs/edit-intent')}
tool-fs mutations only"]`, ` owned["Tool-owned session events
${mermaidCode('todo/write')}, ${mermaidCode('fs/observed')}, ${mermaidCode('hook/invoked')}, ${mermaidCode('hook/result')}"]`, @@ -640,19 +650,21 @@ function renderToolPipeline(): string { ' model --> toolCall', ' toolCall --> presentCall', ' toolCall --> pre', - ' pre -->|allow| toolBody', + ' pre -->|allow| around', + ' around --> toolBody', ' pre -->|deny or ask| denied', ' denied --> post', ' toolBody --> fsGate', ' fsGate --> toolBody', ' toolBody --> owned', - ' toolBody --> post', + ' toolBody --> around', + ' around --> post', ' post --> context', ' post --> toolResult', ' toolResult --> presentResult', '```', '', - 'Filesystem read-before-edit checks live below `tool-fs` on the `fs/*` event gate, while hook bridges and future permission prompts live on the generic tool waterfalls. That split lets the same hooks observe bash, fs, web, todo, and subagent calls without coupling those tools to one policy service.', + 'Filesystem read-before-edit checks live below `tool-fs` on the `fs/*` event gate; hook bridges and future permission prompts live on the generic pre/post tool waterfalls; and around-dispatch concerns like the tool-call timeout policy (`@deepseek-ai/dsh-timeout-policy`) wrap core dispatch on `tools/execute`. That split lets the same hooks observe bash, fs, web, todo, and subagent calls without coupling those tools to one policy service.', '', ...maintenanceFooter(maintenance), ].join('\n') diff --git a/scripts/gen-module-graph.ts b/scripts/gen-module-graph.ts index b84701c819..fe39d325f8 100644 --- a/scripts/gen-module-graph.ts +++ b/scripts/gen-module-graph.ts @@ -45,6 +45,7 @@ const GROUP_ORDER = [ 'compact', 'subagent', 'web', + 'timeout', 'todo', 'hooks', 'session-persistence', diff --git a/scripts/jsdoc.ts b/scripts/jsdoc.ts new file mode 100644 index 0000000000..d0e5771882 --- /dev/null +++ b/scripts/jsdoc.ts @@ -0,0 +1,218 @@ +/** + * Shared JSDoc parsing and completeness-check helpers for the documentation + * gates: the cordis catalog generator (`scripts/gen-cordis-catalog.ts` — the + * events + `ctx.` service surface), the plugin config catalog generator + * (`scripts/gen-config-catalog.ts`, which renders the parsed prose), and the + * export-surface gate (`scripts/verify-export-jsdoc.ts` — every module-level + * export). One home for the mechanics so "documented" means the same thing on + * every gated surface: description prose ends at the first block tag; every + * checkable parameter needs a non-empty `@param`; a non-void ANNOTATED return + * needs a non-empty `@returns`; a stale `@param` naming no real parameter + * errors. + */ + +import ts from 'typescript' + +/** Repo-relative source pointer `file:line` for a node's first character. */ +export function pointer(rel: string, sf: ts.SourceFile, node: ts.Node): string { + const { line } = sf.getLineAndCharacterOfPosition(node.getStart(sf)) + return `${rel}:${line + 1}` +} + +/** The raw `/** … *​/` JSDoc block immediately preceding a node, or '' if none. */ +export function rawJsDoc(text: string, node: ts.Node): string { + const ranges = ts.getLeadingCommentRanges(text, node.getFullStart()) ?? [] + const jsdoc = ranges.filter(r => text.slice(r.pos, r.pos + 3) === '/**').at(-1) + return jsdoc ? text.slice(jsdoc.pos, jsdoc.end) : '' +} + +/** A dispatch mode, rendered as the badge after an event name in the catalog. */ +export type Mode = 'emit' | 'waterfall' | 'parallel' | 'serial' + +/** + * Parse a raw JSDoc block into description prose + the `@mode` tag (when + * present). Output obeys the repo's markdown conventions so the generated + * catalog passes verify-md-wrap: each prose paragraph collapses to ONE physical + * line, and a `-` bullet list is preserved with each item on its own single + * line (continuation lines folded in). `{@link Foo}` unwraps to `Foo`. + * Description prose ends at the FIRST block tag (standard JSDoc semantics): + * tag lines and their continuation lines are never prose, so `@param` / + * `@returns` blocks are invisible to the rendered catalog. + * @param raw - the raw comment text including the JSDoc delimiters. + * @returns the collapsed description prose plus the parsed `@mode` (or null). + */ +export function parseJsDoc(raw: string): { doc: string; mode: Mode | null } { + const inner = raw + .replace(/^\/\*\*/, '') + .replace(/\*\/$/, '') + .split('\n') + .map(l => l.replace(/^\s*\*?\s?/, '').replace(/\s+$/, '')) + let mode: Mode | null = null + let inTags = false + const blocks: string[] = [] + let para: string[] = [] + let list: string[] = [] + let item: string[] = [] + const join = (parts: string[]): string => parts.join(' ').replace(/\s+/g, ' ').trim() + const flushItem = (): void => { + if (item.length) list.push(join(item)) + item = [] + } + const flushList = (): void => { + flushItem() + if (list.length) blocks.push(list.join('\n')) // one block, items on own lines + list = [] + } + const flushPara = (): void => { + flushList() + if (para.length) blocks.push(join(para)) + para = [] + } + for (const line of inner) { + const m = /^@mode\s+(emit|waterfall|parallel|serial)\s*$/.exec(line) + if (m) { mode = m[1] as Mode; flushPara(); inTags = true; continue } + if (line.startsWith('@')) { flushPara(); inTags = true; continue } + if (inTags) continue // block-tag territory: continuations are never prose + if (line.trim() === '') { flushPara(); continue } + if (/^-\s+/.test(line)) { + // A list item starts: a pending paragraph (e.g. an intro line directly + // above the list, no blank between) flushes FIRST so it renders above. + flushItem() + if (para.length) { blocks.push(join(para)); para = [] } + item.push(line) + continue + } + if (item.length) { item.push(line); continue } // continuation of current item + para.push(line) + } + flushPara() + const doc = blocks.join('\n\n').replace(/\{@link\s+([^}]+)\}/g, '$1').trim() + return { doc, mode } +} + +/** + * Parse the block tags of a raw JSDoc comment for the completeness checks: + * every `@param name — description` entry plus the `@returns` description. + * Standard JSDoc block-tag semantics — a tag's description runs across + * continuation lines until the next tag or a blank line, and the `-`/`—` + * separator after a param name is optional. `[name]` optional-brackets unwrap + * to `name`. Rendering never sees these: parseJsDoc stops prose at the first + * block tag. + * @param raw - the raw comment text including the JSDoc delimiters. + * @returns the `@param` name→description map plus the `@returns` description + * (null when the tag is absent, '' when present but empty). + */ +export function parseTags(raw: string): { params: Map; returns: string | null } { + const inner = raw + .replace(/^\/\*\*/, '') + .replace(/\*\/$/, '') + .split('\n') + .map(l => l.replace(/^\s*\*?\s?/, '').replace(/\s+$/, '')) + const params = new Map() + let returns: string | null = null + let sink: ((text: string) => void) | null = null + for (const line of inner) { + const param = /^@param\s+(\[?[\w$]+\]?)\s*(?:[-—–]\s*)?(.*)$/.exec(line) + if (param) { + const name = (param[1] ?? '').replace(/^\[|\]$/g, '') + let acc = param[2] ?? '' + params.set(name, acc) + sink = (t) => { acc = acc ? `${acc} ${t}` : t; params.set(name, acc) } + continue + } + const ret = /^@returns?(?:\s+[-—–]?\s*(.*))?$/.exec(line) + if (ret) { + let acc = ret[1] ?? '' + returns = acc + sink = (t) => { acc = acc ? `${acc} ${t}` : t; returns = acc } + continue + } + if (line.startsWith('@') || line.trim() === '') { sink = null; continue } + sink?.(line.trim()) + } + return { params, returns } +} + +/** + * Check the `@param` half of the completeness contract for one function-like + * declaration: every checkable parameter carries a non-empty `@param`, and no + * `@param` is stale. A binding-pattern parameter is a violation (it has no name + * for `@param` to match); an exempt parameter may be documented but its absence + * is never checked. Violations append to `violations` in place. + * @param where - the offender label violations open with, e.g. `event 'x' (file:1)`. + * @param surface - the surface noun for the binding-pattern message ("event", "service", "export"). + * @param parameters - the declaration's parameter list. + * @param tags - the parsed `@param` name→description map from parseTags. + * @param sf - the source file (for rendering a binding pattern's text). + * @param isExempt - which parameters need no `@param` (e.g. `this`, a waterfall's trailing `next`). + * @param violations - the aggregate list violations append to. + */ +export function checkParams( + where: string, + surface: string, + parameters: readonly ts.ParameterDeclaration[], + tags: Map, + sf: ts.SourceFile, + isExempt: (p: ts.ParameterDeclaration) => boolean, + violations: string[], +): void { + for (const p of parameters) { + if (!ts.isIdentifier(p.name)) { + violations.push(`${where}: parameter '${p.name.getText(sf)}' is a binding pattern; the ${surface} surface needs simple identifier parameters so @param can name them.`) + continue + } + if (isExempt(p)) continue + const desc = tags.get(p.name.text) + if (desc === undefined) violations.push(`${where} is missing @param ${p.name.text}.`) + else if (!desc.trim()) violations.push(`${where}: @param ${p.name.text} has an empty description.`) + } + for (const tag of tags.keys()) { + if (!parameters.some(p => ts.isIdentifier(p.name) && p.name.text === tag)) { + violations.push(`${where}: @param ${tag} does not match any parameter (stale tag?).`) + } + } +} + +/** + * Check the `@returns` half of the completeness contract: a non-`void` / + * `Promise` return needs a non-empty `@returns`, and the return type must + * be ANNOTATED — a pure-AST walk cannot classify an inferred return. On a void + * declaration `@returns` stays optional (resolution timing can be worth + * documenting), never required. Violations append to `violations` in place. + * @param where - the offender label violations open with. + * @param typeNode - the declared return type annotation, or undefined when inferred. + * @param returns - the parsed `@returns` description from parseTags (null when absent). + * @param sf - the source file (for rendering the annotation's text). + * @param violations - the aggregate list violations append to. + */ +export function checkReturns( + where: string, + typeNode: ts.TypeNode | undefined, + returns: string | null, + sf: ts.SourceFile, + violations: string[], +): void { + if (typeNode === undefined) { + violations.push(`${where} has no return type annotation; annotate it explicitly so the gate can classify the result.`) + return + } + const rt = typeNode.getText(sf).replace(/\s+/g, ' ') + if (/^(void|Promise)$/.test(rt)) return + if (returns === null) violations.push(`${where} is missing @returns (return type: ${rt}).`) + else if (!returns.trim()) violations.push(`${where}: @returns has an empty description.`) +} + +/** + * Throw one aggregate error for every completeness violation a walk collected. + * Aggregation (vs failing fast) is deliberate: a remediation pass sees the + * whole list at once instead of replaying the gate once per offender. + * @param gate - the reporting gate's name, prefixed to the error message. + * @param violations - the collected violation lines; no-op when empty. + */ +export function reportViolations(gate: string, violations: string[]): void { + if (violations.length === 0) return + throw new Error( + `${gate}: ${violations.length} JSDoc completeness violation(s) (see AGENTS.md):\n` + + violations.map(v => ` ${v}`).join('\n'), + ) +} diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 195e188858..9ab71c6b61 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -257,7 +257,9 @@ function docSyncLeafGates(): Gate[] { return [ pnpmScript('doc-typecheck', 'doc-typecheck'), pnpmScript('cordis-catalog', 'verify-cordis-catalog', { label: 'cordis catalog' }), + pnpmScript('export-jsdoc', 'verify-export-jsdoc', { label: 'export jsdoc' }), pnpmScript('tool-catalog', 'verify-tool-catalog', { label: 'tool catalog' }), + pnpmScript('config-catalog', 'verify-config-catalog', { label: 'config catalog' }), pnpmScript('persistence-catalog', 'verify-persistence-catalog', { label: 'persistence catalog' }), pnpmScript('doc-graphs', 'verify-doc-graphs', { label: 'doc graphs' }), pnpmScript('markdown-wrap', 'verify-md-wrap', { label: 'markdown wrap' }), @@ -308,6 +310,7 @@ function builtBinSmokeGate(): Gate { 'vitest.e2e.config.ts', 'packages/ui/stdio-agent/tests/built-bin.e2e.ts', 'packages/ui/acp-agent/tests/built-bin.e2e.ts', + 'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts', ], { label: 'built-bin smoke', needs: ['build'], diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index b5c648527e..613280ee1b 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -43,6 +43,10 @@ { "doc": "docs/core-data-structures/tools.md", "symbol": "ToolExecutionResult", "source": "packages/core/tools/src/index.ts" }, { "doc": "docs/core-data-structures/tools.md", "symbol": "PreToolDecision", "source": "packages/core/tools/src/index.ts" }, { "doc": "docs/core-data-structures/tools.md", "symbol": "PostToolDecision", "source": "packages/core/tools/src/index.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "StructuredScalar", "source": "packages/core/tools/src/json-schema.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "StructuredSchemaType", "source": "packages/core/tools/src/json-schema.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "StructuredSchemaNode", "source": "packages/core/tools/src/json-schema.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "StructuredOutputSchema", "source": "packages/core/tools/src/json-schema.ts" }, { "doc": "docs/core-data-structures/bash.md", "symbol": "BashExecRequest", "source": "packages/bash/bash/src/types.ts" }, { "doc": "docs/core-data-structures/bash.md", "symbol": "BashExecSpec", "source": "packages/bash/bash/src/types.ts" }, @@ -51,6 +55,13 @@ { "doc": "docs/core-data-structures/bash.md", "symbol": "BashTask", "source": "packages/bash/bash/src/types.ts" }, { "doc": "docs/core-data-structures/bash.md", "symbol": "BashTaskRead", "source": "packages/bash/bash/src/types.ts" }, + { "doc": "docs/core-data-structures/code-runtime.md", "symbol": "CodeRunRequest", "source": "packages/code-runtime/code-runtime/src/types.ts" }, + { "doc": "docs/core-data-structures/code-runtime.md", "symbol": "CodeRunResult", "source": "packages/code-runtime/code-runtime/src/types.ts" }, + { "doc": "docs/core-data-structures/code-runtime.md", "symbol": "CodeBindingNamespace", "source": "packages/code-runtime/code-runtime/src/types.ts" }, + { "doc": "docs/core-data-structures/code-runtime.md", "symbol": "CodeBindingFunction", "source": "packages/code-runtime/code-runtime/src/types.ts" }, + { "doc": "docs/core-data-structures/code-runtime.md", "symbol": "CodeLogEntry", "source": "packages/code-runtime/code-runtime/src/types.ts" }, + { "doc": "docs/core-data-structures/code-runtime.md", "symbol": "CodeRunFailure", "source": "packages/code-runtime/code-runtime/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsTarget", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsTargetKey", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsVersion", "source": "packages/fs/fs/src/types.ts" }, diff --git a/scripts/verify-doc-refs.ts b/scripts/verify-doc-refs.ts index a53399a272..56be5140c8 100644 --- a/scripts/verify-doc-refs.ts +++ b/scripts/verify-doc-refs.ts @@ -26,9 +26,8 @@ * Run: `tsx scripts/verify-doc-refs.ts`. */ -import { existsSync, readFileSync } from 'node:fs' +import { existsSync, globSync, readFileSync } from 'node:fs' import { relative, resolve } from 'node:path' -import { glob } from 'node:fs/promises' const root = resolve(import.meta.dirname, '..') @@ -77,7 +76,7 @@ function findViolations(absPath: string): Violation[] { const all: Violation[] = [] let checked = 0 for (const pattern of PATTERNS) { - for await (const match of glob(pattern, { cwd: root })) { + for (const match of globSync(pattern, { cwd: root })) { if (isExcluded(match)) continue checked++ all.push(...findViolations(resolve(root, match))) diff --git a/scripts/verify-export-jsdoc.ts b/scripts/verify-export-jsdoc.ts new file mode 100644 index 0000000000..779888ab90 --- /dev/null +++ b/scripts/verify-export-jsdoc.ts @@ -0,0 +1,643 @@ +/** + * Verify JSDoc completeness for EVERY module-level exported name of every + * non-vendored package (each `packages///src/` tree). This is the + * mechanical form of the AGENTS.md rule "every export has a JSDoc explaining + * semantics", generalizing the cordis-surface gate (`gen-cordis-catalog.ts`, + * which owns `interface Events` members and `ctx.` service classes) to + * the whole export surface; the parsing + check helpers are shared via + * `scripts/jsdoc.ts` so "documented" means the same thing on both. + * + * `tsx scripts/verify-export-jsdoc.ts` → exit 1 listing every offender + * + * The contract, per exported declaration kind: + * + * - Every exported name needs JSDoc with non-empty description prose (prose + * ends at the first block tag, standard JSDoc semantics). + * - A function-like export (function declaration, a const with a function + * initializer or an INLINE callable annotation, or a non-identifier + * function default export) additionally needs a non-empty `@param` per + * parameter (`this` receiver annotations exempt; a stale `@param` errors) + * and a non-empty `@returns` unless the return type is `void` / + * `Promise`. Wrapper expressions (parentheses, `as` / `satisfies` + * casts, non-null assertions) are peeled before classifying. The walk + * classifies returns syntactically, so the return type must be ANNOTATED — + * except a const whose declarator is annotated with a NAMED type (e.g. + * `export const f: Handler = …`), where that type's own declaration owns + * the signature contract and `@returns` stays optional; an inline + * `(x: T) => U` annotation or single-call-signature literal is the surface + * signature itself and gets the full contract, and a literal mixing + * call/construct signatures with anything else is refused (extract a named + * type). + * - An exported class needs class-level JSDoc; its public methods (static + * included — they are reachable on the exported name) follow the function + * contract, and public properties and accessors need description prose (on + * a get/set pair the getter's doc covers both). A member declared by an + * `extends`/`implements` heritage type is EXEMPT — the seam declaration is + * the doc's one home, the IDE inherits it, and re-documenting every + * implementation invites drift — UNLESS the override grows surface the + * base never documented: a protected-only base member does not exempt a + * public override, parameters the base never names keep their `@param` + * duty, and a concrete result above a void base return keeps its + * `@returns` duty. Heritage members (and classifying an unannotated + * override's inferred return above a void base) are the questions the walk + * asks the TYPE CHECKER; everything else is pure AST. + * Constructors are exempt like the cordis gate's: plugin classes are + * framework-constructed, and the class doc owns the story. + * - Exported interfaces, type aliases, enums: description prose on the + * declaration (member-level docs stay review's job; the highest-value + * member surface — seam service classes — is already under the cordis + * gate). + * - An exported namespace recurses (its exported members are package + * surface; in an ambient `declare` namespace every member exports + * implicitly); the namespace itself needs prose only when it does not + * merge with an already-documented same-name declaration (the + * Config-namespace idiom documents the class/function once, not twice). + * - The cordis plugin-protocol slots are exempt: top-level `name` / `inject` + * / `reusable` / `Config` consts and the `apply` entry, plus the same + * slots as statics on a plugin class. Their shape is fixed by the + * framework, so a doc would restate the protocol — the module doc comment + * and the `interface Config` carry the plugin's real semantics. (These + * names are reserved by cordis convention; documenting one anyway is + * allowed, only absence goes unchecked.) + * - Overload groups: each overload signature carries its own docs; the + * implementation signature is exempt (callers never see it). + * - Skipped: `declare module` / `declare global` augmentation bodies (the + * cordis gate's turf; an augmentation is not an export of the package) and + * re-export statements with a module specifier (`export … from`) — the + * defining module is walked on its own, and external definitions are not + * ours to document. An `export import X = N.member` alias documents + * ITSELF, and only prose-only target kinds are gate-supported: a callable, + * class, or namespace target carries signature/member contracts the alias + * cannot hold and is refused (export the declaration directly). + * - Everything else fails CLOSED: `export =` is refused outright, and an + * exported statement kind the dispatch does not recognize is itself a + * violation, so no export form can pass unchecked by omission. + */ + +import { existsSync, globSync } from 'node:fs' +import { resolve } from 'node:path' +import ts from 'typescript' +import { checkParams, checkReturns, parseJsDoc, parseTags, pointer, rawJsDoc } from './jsdoc.ts' + +const root = resolve(import.meta.dirname, '..') + +/** Plugin-protocol slot names exempt as statics on an exported class. */ +const PROTOCOL_STATICS = new Set(['Config', 'inject', 'name', 'reusable']) + +/** Plugin-protocol slot names exempt as top-level exports (const or function). */ +const PROTOCOL_EXPORTS = new Set(['Config', 'inject', 'name', 'reusable', 'apply']) + +/** Per-file walk state threaded through the scope recursion. */ +interface Walk { + /** Repo-relative path of the file being walked. */ + rel: string + /** The parsed source file. */ + sf: ts.SourceFile + /** Raw file text (rawJsDoc reads comment ranges out of it). */ + text: string + /** The program's checker, consulted only for heritage-member lookups. */ + checker: ts.TypeChecker + /** The aggregate violation list, appended in place. */ + violations: string[] +} + +/** True when a statement carries the `export` modifier. */ +function isExported(stmt: ts.Statement): boolean { + return ts.canHaveModifiers(stmt) && (ts.getModifiers(stmt)?.some(m => m.kind === ts.SyntaxKind.ExportKeyword) ?? false) +} + +/** True for a class member a consumer cannot reach: `private`/`protected`/`#name`. */ +function isNonPublic(member: ts.ClassElement): boolean { + const mods = ts.canHaveModifiers(member) ? ts.getModifiers(member) : undefined + return (mods?.some(m => m.kind === ts.SyntaxKind.PrivateKeyword || m.kind === ts.SyntaxKind.ProtectedKeyword) ?? false) + || ('name' in member && ts.isPrivateIdentifier(member.name)) +} + +/** True when a class member carries the `static` modifier. */ +function isStatic(member: ts.ClassElement): boolean { + const mods = ts.canHaveModifiers(member) ? ts.getModifiers(member) : undefined + return mods?.some(m => m.kind === ts.SyntaxKind.StaticKeyword) ?? false +} + +/** The `this`-receiver exemption every function-like check shares. */ +function thisReceiver(p: ts.ParameterDeclaration): boolean { + return ts.isIdentifier(p.name) && p.name.text === 'this' +} + +/** + * Peel wrapper expressions that carry no surface of their own — parentheses, + * `as` / `satisfies` / angle-bracket casts, non-null assertions — so a + * wrapped function expression is still classified as function-like. + * @param e - the expression to unwrap. + * @returns the innermost non-wrapper expression. + */ +function unwrapExpression(e: ts.Expression): ts.Expression { + let inner = e + while ( + ts.isParenthesizedExpression(inner) || ts.isAsExpression(inner) || ts.isSatisfiesExpression(inner) + || ts.isNonNullExpression(inner) || ts.isTypeAssertionExpression(inner) + ) inner = inner.expression + return inner +} + +/** + * Classify a declarator's type annotation for the function contract: an + * inline function type or a type literal that is EXACTLY one call signature + * is the surface signature itself; a literal mixing call/construct + * signatures with anything else cannot be classified syntactically and is + * refused (fail closed — extract a named type); everything else is a plain + * value shape. + * @param type - the declarator's type annotation. + * @returns the signature to check, 'refuse' for an unclassifiable callable literal, or null for a non-callable shape. + */ +function callableAnnotation(type: ts.TypeNode): ts.SignatureDeclarationBase | 'refuse' | null { + if (ts.isFunctionTypeNode(type)) return type + if (!ts.isTypeLiteralNode(type)) return null + const signatures = type.members.filter(m => ts.isCallSignatureDeclaration(m) || ts.isConstructSignatureDeclaration(m)) + if (signatures.length === 0) return null + if (signatures.length === 1 && type.members.length === 1 && signatures[0] !== undefined && ts.isCallSignatureDeclaration(signatures[0])) { + return signatures[0] + } + return 'refuse' +} + +/** + * The heritage-member exemption for one class member. When the member's name + * is declared by an `extends`/`implements` heritage type, the seam declaration + * is the doc's one home (the IDE inherits it on hover) and the member needs no + * doc of its own — EXCEPT where the override grows public surface the base + * never documented: a base member that is protected on every declaration does + * not exempt a public override (consumers could not call it before); + * parameters the base never names keep their own `@param` duty (the caller + * reads the seam doc, which cannot describe them; an underscore-prefixed + * rename of a base parameter — the deliberately-unused marker — is the same + * parameter, not new surface); and a void base return carried no `@returns` + * duty, so an override returning a concrete result documents it itself. + * Static members are looked up on the base CONSTRUCTOR type (only an + * `extends` expression has one; an unresolvable or interface expression + * yields no property and therefore no exemption). + * @param cls - the class whose heritage to search. + * @param name - the member name to look up. + * @param staticSide - whether to search the constructor side instead of the instance side. + * @param checker - the program's type checker. + * @returns null when no exemption applies; otherwise the parameter names the + * base declarations carry (`baseParams: null` when not syntactically + * recoverable — a complex heritage type — exempting all parameters) plus + * whether every recoverable base return annotation is `void`-like + * (`baseVoidReturn: null` when none is recoverable, exempting the result). + */ +function heritageExemption( + cls: ts.ClassDeclaration, + name: string, + staticSide: boolean, + checker: ts.TypeChecker, +): { baseParams: Set | null; baseVoidReturn: boolean | null } | null { + const isProtected = (d: ts.Declaration): boolean => + (ts.canHaveModifiers(d) ? ts.getModifiers(d) : undefined)?.some(m => m.kind === ts.SyntaxKind.ProtectedKeyword) ?? false + for (const clause of cls.heritageClauses ?? []) { + for (const t of clause.types) { + const type = staticSide ? checker.getTypeAtLocation(t.expression) : checker.getTypeAtLocation(t) + const prop = type.getProperty(name) + if (prop === undefined) continue + const decls = prop.declarations ?? [] + if (decls.length > 0 && decls.every(isProtected)) continue // public override of a protected base: new surface + let baseParams: Set | null = null + let baseVoidReturn: boolean | null = null + for (const d of decls) { + let params: readonly ts.ParameterDeclaration[] | undefined + let returnType: ts.TypeNode | undefined + if (ts.isMethodDeclaration(d) || ts.isMethodSignature(d)) { + params = d.parameters + returnType = d.type + } else if ((ts.isPropertySignature(d) || ts.isPropertyDeclaration(d)) && d.type !== undefined && ts.isFunctionTypeNode(d.type)) { + params = d.type.parameters + returnType = d.type.type + } else continue + baseParams ??= new Set() + // Leading underscores are the deliberately-unused marker (eslint + // argsIgnorePattern), not a rename: `_cwd` overriding `cwd` is the + // same parameter, so compare underscore-stripped on both sides. + for (const p of params) if (ts.isIdentifier(p.name)) baseParams.add(p.name.text.replace(/^_+/, '')) + if (returnType !== undefined) { + const voidish = /^(void|Promise)$/.test(returnType.getText(d.getSourceFile()).replace(/\s+/g, ' ')) + baseVoidReturn = (baseVoidReturn ?? true) && voidish + } + } + return { baseParams, baseVoidReturn } + } + } + return null +} + +/** + * True when a method's INFERRED return type is void-like (void, undefined, + * never, or a promise of one) — the one return the walk asks the checker to + * classify: an unannotated override above a void heritage member, where + * demanding an annotation just to prove faithfulness would be boilerplate. + * @param m - a method declaration with no return type annotation. + * @param checker - the program's type checker. + * @returns true when the inferred result carries nothing to document. + */ +function inferredReturnIsVoidish(m: ts.MethodDeclaration, checker: ts.TypeChecker): boolean { + const sig = checker.getSignatureFromDeclaration(m) + if (sig === undefined) return true // no callable signature: nothing classifiable to document + const returned = checker.getReturnTypeOfSignature(sig) + const awaited = checker.getAwaitedType(returned) ?? returned + return (awaited.flags & (ts.TypeFlags.Void | ts.TypeFlags.Undefined | ts.TypeFlags.Never)) !== 0 +} + +/** + * Check description-prose presence for one labeled declaration: JSDoc must + * exist and carry prose above its block tags. + * @param where - the offender label violations open with. + * @param raw - the declaration's raw JSDoc block ('' if none). + * @param w - the walk state violations append to. + */ +function checkDescribed(where: string, raw: string, w: Walk): void { + if (!raw) w.violations.push(`${where} has no JSDoc.`) + else if (!parseJsDoc(raw).doc) w.violations.push(`${where} has no description prose above its block tags.`) +} + +/** + * Check the full function contract for one labeled function-like declaration: + * description prose, `@param` per parameter, `@returns` on a non-void result. + * @param where - the offender label violations open with. + * @param raw - the declaration's raw JSDoc block ('' if none). + * @param parameters - the declaration's parameter list. + * @param returnType - the return type annotation, or undefined when inferred. + * @param returnsWaived - suppress the `@returns`/annotation requirement (a + * declarator-annotated const defers its return contract to the named type). + * @param w - the walk state violations append to. + */ +function checkFunctionLike( + where: string, + raw: string, + parameters: readonly ts.ParameterDeclaration[], + returnType: ts.TypeNode | undefined, + returnsWaived: boolean, + w: Walk, +): void { + if (!raw) { w.violations.push(`${where} has no JSDoc.`); return } + if (!parseJsDoc(raw).doc) w.violations.push(`${where} has no description prose above its block tags.`) + const { params, returns } = parseTags(raw) + checkParams(where, 'export', parameters, params, w.sf, thisReceiver, w.violations) + if (!returnsWaived) checkReturns(where, returnType, returns, w.sf, w.violations) +} + +/** + * Check one exported class: class-level prose, the function contract on every + * public method (overload implementations exempt), and description prose on + * public properties and accessors (a get/set pair is covered by the getter's + * doc). Heritage-declared members are exempt per heritageExemption (an + * override's extra parameters keep their @param duty); plugin-protocol + * statics are exempt; constructors are not checked (framework-constructed + * plugins, and the class doc owns the story). + * @param cls - the exported class declaration. + * @param name - the class's surface name (namespace-qualified). + * @param w - the walk state violations append to. + */ +function checkClass(cls: ts.ClassDeclaration, name: string, w: Walk): void { + checkDescribed(`exported class '${name}' (${pointer(w.rel, w.sf, cls)})`, rawJsDoc(w.text, cls), w) + const overloadSigs = new Set() + const documentedGetters = new Set() + for (const m of cls.members) { + if ('name' in m && ts.isComputedPropertyName(m.name)) continue + if (ts.isMethodDeclaration(m) && !m.body) overloadSigs.add(m.name.getText(w.sf)) + if (ts.isGetAccessorDeclaration(m)) documentedGetters.add(m.name.getText(w.sf)) + } + for (const m of cls.members) { + if (isNonPublic(m) || ts.isConstructorDeclaration(m)) continue + if (!('name' in m) || ts.isComputedPropertyName(m.name)) continue // computed/symbol members + const mname = m.name.getText(w.sf) + if (isStatic(m) && PROTOCOL_STATICS.has(mname)) continue // cordis plugin-protocol slot + const exemption = heritageExemption(cls, mname, isStatic(m), w.checker) + if (ts.isMethodDeclaration(m)) { + if (m.body && overloadSigs.has(mname)) continue // overload implementation: the signatures carry the docs + const where = `exported class method '${name}.${mname}' (${pointer(w.rel, w.sf, m)})` + if (exemption !== null) { + const raw = rawJsDoc(w.text, m) + // The heritage declaration owns the prose; parameters the base never + // names — including binding patterns, which no base declaration can + // name — are new surface and keep their @param duty. + const base = exemption.baseParams + const inBase = (p: ts.ParameterDeclaration): boolean => + base !== null && ts.isIdentifier(p.name) && base.has(p.name.text.replace(/^_+/, '')) + if (base !== null && m.parameters.some(p => !thisReceiver(p) && !inBase(p))) { + checkParams(where, 'export', m.parameters, parseTags(raw).params, w.sf, + p => thisReceiver(p) || inBase(p), w.violations) + } + // A void base return carried no @returns duty, so an override growing + // a concrete result documents it itself. An annotated override runs + // the standard check; an inferred one is classified by the checker + // (this branch is already the checker's domain), so a faithful void + // override stays exempt without a boilerplate annotation. + if (exemption.baseVoidReturn === true) { + if (m.type !== undefined) { + checkReturns(where, m.type, parseTags(raw).returns, w.sf, w.violations) + } else if (!inferredReturnIsVoidish(m, w.checker)) { + w.violations.push(`${where} returns a non-void result its heritage declaration does not document; annotate the return type and add @returns.`) + } + } + continue + } + checkFunctionLike(where, rawJsDoc(w.text, m), m.parameters, m.type, false, w) + } else if (exemption !== null) { + continue // the heritage declaration owns the doc (properties/accessors carry no own parameters) + } else if (ts.isGetAccessorDeclaration(m) || ts.isPropertyDeclaration(m)) { + const kind = ts.isPropertyDeclaration(m) ? 'property' : 'accessor' + checkDescribed(`exported class ${kind} '${name}.${mname}' (${pointer(w.rel, w.sf, m)})`, rawJsDoc(w.text, m), w) + } else if (ts.isSetAccessorDeclaration(m) && !documentedGetters.has(mname)) { + checkDescribed(`exported class accessor '${name}.${mname}' (${pointer(w.rel, w.sf, m)})`, rawJsDoc(w.text, m), w) + } + // index signatures / static blocks: not named surface + } +} + +/** + * Check one exported declaration statement, dispatching on its kind. Any + * exported statement kind the dispatch does not recognize is a violation + * (fail closed), so no export form can pass unchecked by omission. + * @param stmt - the exported statement (export modifier or export-list target). + * @param prefix - the namespace qualification for surface names ('' at top level). + * @param overloadSigs - names in this scope declared as bodyless function overload signatures. + * @param byName - this scope's named declarations (for namespace/sibling-merge lookups). + * @param ambient - whether the enclosing scope is ambient (`declare`), where members export implicitly. + * @param w - the walk state violations append to. + * @param only - for a multi-declarator variable statement reached through an + * export list (or a default-export identifier), the declarator names that + * are actually exported; `null` means the whole statement is surface + * (direct `export` modifier or ambient scope). Non-variable statements + * declare exactly one name, so the filter never applies to them. + */ +function checkDecl( + stmt: ts.Statement, + prefix: string, + overloadSigs: Set, + byName: Map, + ambient: boolean, + w: Walk, + only: ReadonlySet | null = null, +): void { + const at = (n: ts.Node): string => ` (${pointer(w.rel, w.sf, n)})` + if (ts.isFunctionDeclaration(stmt)) { + const name = stmt.name?.text ?? 'default' + if (prefix === '' && PROTOCOL_EXPORTS.has(name)) return // cordis plugin-protocol slot + if (stmt.body && overloadSigs.has(name)) return // overload implementation: the signatures carry the docs + checkFunctionLike(`exported function '${prefix}${name}'${at(stmt)}`, rawJsDoc(w.text, stmt), + stmt.parameters, stmt.type, false, w) + return + } + if (ts.isClassDeclaration(stmt)) { + checkClass(stmt, `${prefix}${stmt.name?.text ?? 'default'}`, w) + return + } + if (ts.isInterfaceDeclaration(stmt)) { + checkDescribed(`exported interface '${prefix}${stmt.name.text}'${at(stmt)}`, rawJsDoc(w.text, stmt), w) + return + } + if (ts.isTypeAliasDeclaration(stmt)) { + checkDescribed(`exported type '${prefix}${stmt.name.text}'${at(stmt)}`, rawJsDoc(w.text, stmt), w) + return + } + if (ts.isEnumDeclaration(stmt)) { + checkDescribed(`exported enum '${prefix}${stmt.name.text}'${at(stmt)}`, rawJsDoc(w.text, stmt), w) + return + } + if (ts.isVariableStatement(stmt)) { + const raw = rawJsDoc(w.text, stmt) // JSDoc sits on the statement, not the declarator + for (const d of stmt.declarationList.declarations) { + const name = ts.isIdentifier(d.name) ? d.name.text : d.name.getText(w.sf) + if (only !== null && !only.has(name)) continue // sibling declarator the export list never named: not surface + if (prefix === '' && PROTOCOL_EXPORTS.has(name)) continue // cordis plugin-protocol slot + const where = `exported const '${prefix}${name}'${at(d)}` + const annotation = d.type !== undefined ? callableAnnotation(d.type) : null + const init = d.initializer !== undefined ? unwrapExpression(d.initializer) : undefined + if (annotation === 'refuse') { + // A literal mixing call/construct signatures with other members (or + // overloading them) has no single signature the walk can hold the + // tags against — fail closed rather than silently narrow the check. + w.violations.push(`${where}: its callable type literal is not gate-classifiable; extract a named type and document it there.`) + } else if (annotation !== null) { + // An INLINE callable annotation is the surface signature itself: its + // parameters and result need docs right here. (A NAMED reference + // type carries its docs at the type's own declaration instead.) + checkFunctionLike(where, raw, annotation.parameters, annotation.type, false, w) + } else if (init !== undefined && (ts.isArrowFunction(init) || ts.isFunctionExpression(init))) { + // A named declarator type annotation (`const f: Handler = …`) hands + // the return contract to the named type; the arrow's own annotation is + // still checked when it is the only signature the reader has. + checkFunctionLike(where, raw, init.parameters, init.type, init.type === undefined && d.type !== undefined, w) + } else { + checkDescribed(where, raw, w) + } + } + return + } + if (ts.isModuleDeclaration(stmt) && ts.isIdentifier(stmt.name)) { + // A namespace merging with a documented same-name sibling (the + // Config-namespace idiom) needs no second doc block of its own. + const siblings = (byName.get(stmt.name.text) ?? []).filter(s => s !== stmt) + const merged = siblings.some(s => parseJsDoc(rawJsDoc(w.text, s)).doc !== '') + if (!merged) checkDescribed(`exported namespace '${prefix}${stmt.name.text}'${at(stmt)}`, rawJsDoc(w.text, stmt), w) + let body = stmt.body + let nsPrefix = `${prefix}${stmt.name.text}.` + while (body !== undefined && ts.isModuleDeclaration(body)) { // dotted `namespace A.B` + nsPrefix += `${body.name.getText(w.sf)}.` + body = body.body + } + // In an ambient (`declare`) namespace body, members are implicitly + // exported — no `export` modifier required — so the recursion must treat + // every statement as surface. + const declared = ambient + || ((ts.canHaveModifiers(stmt) ? ts.getModifiers(stmt) : undefined)?.some(m => m.kind === ts.SyntaxKind.DeclareKeyword) ?? false) + if (body !== undefined && ts.isModuleBlock(body)) checkScope(body.statements, nsPrefix, w, declared) + return + } + if (ts.isImportEqualsDeclaration(stmt)) { + const where = `exported alias '${prefix}${stmt.name.text}'${at(stmt)}` + // An alias is a distinct exported name whose target may be a non-exported + // namespace member no walk ever visits, so it documents ITSELF — which + // matches the gate's strength only for prose-only target kinds. A + // callable, class, or namespace target carries signature or member + // contracts the alias prose cannot hold: refuse those (fail closed) and + // demand the declaration be exported directly. An unresolvable target is + // refused for the same reason. + const sym = w.checker.getSymbolAtLocation(stmt.name) + const target = sym !== undefined && (sym.flags & ts.SymbolFlags.Alias) !== 0 ? w.checker.getAliasedSymbol(sym) : sym + const RICH_TARGETS = ts.SymbolFlags.Function | ts.SymbolFlags.Class | ts.SymbolFlags.ValueModule | ts.SymbolFlags.NamespaceModule + const rich = target === undefined + || (target.flags & RICH_TARGETS) !== 0 + || w.checker.getTypeOfSymbol(target).getCallSignatures().length > 0 + if (rich) { + w.violations.push(`${where} aliases a callable, class, or namespace target whose signature/member contract the alias cannot carry; export the declaration directly instead.`) + return + } + checkDescribed(where, rawJsDoc(w.text, stmt), w) + return + } + // Fail CLOSED: an exported statement kind this dispatch does not recognize + // must never pass silently — the gate's whole promise is that unchecked + // surface cannot exist. New TypeScript export forms extend the gate here. + w.violations.push(`exported statement${at(stmt)} uses an export form verify-export-jsdoc does not handle; extend the gate.`) +} + +/** + * Walk one lexical scope (file top level or a namespace body): check every + * exported declaration, resolving `export { … }` lists (no module specifier) + * to their local declarations. + * @param statements - the scope's statements. + * @param prefix - the namespace qualification for surface names ('' at top level). + * @param w - the walk state violations append to. + * @param ambient - whether this scope is ambient (`declare` namespace or a declaration file), where members export implicitly. + */ +function checkScope(statements: readonly ts.Statement[], prefix: string, w: Walk, ambient: boolean): void { + const byName = new Map() + const overloadSigs = new Set() + const add = (name: string, stmt: ts.Statement): void => { + byName.set(name, [...(byName.get(name) ?? []), stmt]) + } + for (const stmt of statements) { + if (ts.isFunctionDeclaration(stmt)) { + if (stmt.name) add(stmt.name.text, stmt) + if (!stmt.body && stmt.name) overloadSigs.add(stmt.name.text) + } else if (ts.isClassDeclaration(stmt) || ts.isInterfaceDeclaration(stmt) + || ts.isTypeAliasDeclaration(stmt) || ts.isEnumDeclaration(stmt)) { + if (stmt.name) add(stmt.name.text, stmt) + } else if (ts.isModuleDeclaration(stmt) && ts.isIdentifier(stmt.name)) { + add(stmt.name.text, stmt) + } else if (ts.isVariableStatement(stmt)) { + for (const d of stmt.declarationList.declarations) { + if (ts.isIdentifier(d.name)) add(d.name.text, stmt) + } + } + } + // Two-phase dispatch. Phase one accumulates WHICH statements are surface + // and, for a variable statement reached by name (an export list or a + // default-export identifier), which of its declarators the exports actually + // name — `null` marks the whole statement as surface (a direct `export` + // modifier, or an ambient scope). Requests for the same statement merge: + // `null` absorbs any name set, and name sets union, so + // `export { a }; export { b }` over one `const a = …, b = …` checks both + // declarators while a never-exported sibling stays out of the surface. + // Phase two runs each surfaced statement exactly once. (Checking a + // statement eagerly per request would either re-check on the second list or + // — deduplicated — silently drop the second list's declarators.) + const requested = new Map | null>() + const request = (stmt: ts.Statement, name: string | null): void => { + const prior = requested.get(stmt) + if (name === null || prior === null) { + requested.set(stmt, null) + return + } + requested.set(stmt, prior === undefined ? new Set([name]) : prior.add(name)) + } + for (const stmt of statements) { + if (ts.isModuleDeclaration(stmt) + && (ts.isStringLiteral(stmt.name) || (stmt.flags & ts.NodeFlags.GlobalAugmentation) !== 0)) { + continue // `declare module '…'` / `declare global` augmentation: not an export of this package + } + if (ts.isExportDeclaration(stmt)) { + if (stmt.moduleSpecifier) continue // re-export: the defining module is walked on its own + if (stmt.exportClause && ts.isNamedExports(stmt.exportClause)) { + for (const el of stmt.exportClause.elements) { + const local = (el.propertyName ?? el.name).text + for (const decl of byName.get(local) ?? []) request(decl, local) + // a name with no local declaration is an imported binding re-exported + // without a specifier — its defining module is walked on its own + } + } + continue + } + if (ts.isExportAssignment(stmt)) { + if (stmt.isExportEquals) { + // `export =` has no ESM consumer surface in this repo and the walk + // cannot classify its operand's shape; refuse rather than fail open. + w.violations.push(`export-equals assignment (${pointer(w.rel, w.sf, stmt)}) is not a gate-supported export form; use ESM named exports.`) + continue + } + const where = `default export (${pointer(w.rel, w.sf, stmt)})` + const expr = unwrapExpression(stmt.expression) + if (ts.isIdentifier(expr)) { + for (const decl of byName.get(expr.text) ?? []) request(decl, expr.text) + } else if (ts.isArrowFunction(expr) || ts.isFunctionExpression(expr)) { + checkFunctionLike(where, rawJsDoc(w.text, stmt), expr.parameters, expr.type, false, w) + } else { + checkDescribed(where, rawJsDoc(w.text, stmt), w) + } + continue + } + if (isExported(stmt) || (ambient && !ts.isImportDeclaration(stmt))) request(stmt, null) + } + for (const stmt of statements) { + const only = requested.get(stmt) + if (only !== undefined) checkDecl(stmt, prefix, overloadSigs, byName, ambient, w, only) + } +} + +/** + * Compiler options for the walk's program. The real repo hands over its + * tsconfig.base.json (whose `paths` map resolves cross-package imports to + * source, so heritage-member lookups see seam types); a fixture root without + * one gets `noLib` + no `@types` — fixtures are single-file and + * self-contained, nothing in the walk resolves a lib symbol, and default-lib + * parsing is ~99% of per-program cost (it made the fixture spec time out + * under CI coverage instrumentation). Emit-side options are stripped: the + * walk never emits or asks for diagnostics, it only binds types on demand. + * @param scanRoot - the root being scanned. + * @returns compiler options for ts.createProgram. + */ +function loadCompilerOptions(scanRoot: string): ts.CompilerOptions { + const cfgPath = resolve(scanRoot, 'tsconfig.base.json') + if (!existsSync(cfgPath)) return { skipLibCheck: true, noLib: true, types: [] } + const cfg = ts.readConfigFile(cfgPath, ts.sys.readFile.bind(ts.sys)) as { config?: unknown } + const parsed = ts.parseJsonConfigFileContent(cfg.config ?? {}, ts.sys, scanRoot) + return { + ...parsed.options, + noEmit: true, + composite: false, + declaration: false, + declarationMap: false, + sourceMap: false, + incremental: false, + } +} + +/** + * Walk every non-vendored package source file and collect JSDoc-completeness + * violations for its module-level exports. Returns findings instead of + * throwing so tests assert on the list; the CLI entry turns a non-empty list + * into exit 1. + * @param scanRoot - the repo root to scan; tests pass a fixture dir. + * @returns every violation, in file order, one human-readable line each. + */ +export function collectExportJsdocViolations(scanRoot: string = root): string[] { + const violations: string[] = [] + const rels = globSync('packages/*/*/src/**/*.ts', { cwd: scanRoot }).sort() + const program = ts.createProgram(rels.map(rel => resolve(scanRoot, rel)), loadCompilerOptions(scanRoot)) + const checker = program.getTypeChecker() + for (const rel of rels) { + const sf = program.getSourceFile(resolve(scanRoot, rel)) + if (!sf) continue // program root files always resolve; guard for narrowing + // A script-style declaration file (no imports/exports) is one big ambient + // scope; a module-style .d.ts still honors explicit export modifiers. + checkScope(sf.statements, '', { rel, sf, text: sf.text, checker, violations }, sf.isDeclarationFile && !ts.isExternalModule(sf)) + } + return violations +} + +/** CLI entry: list every violation and exit 1, or confirm a clean surface. */ +function main(): void { + const violations = collectExportJsdocViolations() + if (violations.length === 0) { + console.log('verify-export-jsdoc: every exported name on the package surface is documented.') + return + } + console.error(`verify-export-jsdoc: ${violations.length} JSDoc completeness violation(s) (see AGENTS.md):`) + for (const v of violations) console.error(` ${v}`) + process.exit(1) +} + +// Run only when invoked as a script, not when imported by a test. +if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { + main() +} diff --git a/scripts/verify-md-links.ts b/scripts/verify-md-links.ts index 2a96cfd0af..d3e80e285e 100644 --- a/scripts/verify-md-links.ts +++ b/scripts/verify-md-links.ts @@ -32,9 +32,8 @@ * Run: `tsx scripts/verify-md-links.ts`. */ -import { existsSync, readFileSync, realpathSync } from 'node:fs' +import { existsSync, globSync, readFileSync, realpathSync } from 'node:fs' import { dirname, relative, resolve } from 'node:path' -import { glob } from 'node:fs/promises' import { fromMarkdown } from 'mdast-util-from-markdown' import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfm } from 'micromark-extension-gfm' @@ -134,7 +133,7 @@ const seen = new Set() const all: Violation[] = [] let checked = 0 for (const pattern of PATTERNS) { - for await (const match of glob(pattern, { cwd: root })) { + for (const match of globSync(pattern, { cwd: root })) { const abs = resolve(root, match) // CLAUDE.md symlinks resolve onto AGENTS.md; dedupe by real path so a file // matched twice (or via symlink) is checked once. diff --git a/scripts/verify-md-wrap.ts b/scripts/verify-md-wrap.ts index 3ffad3be43..2d8845e030 100644 --- a/scripts/verify-md-wrap.ts +++ b/scripts/verify-md-wrap.ts @@ -25,9 +25,8 @@ * Run: `tsx scripts/verify-md-wrap.ts`. */ -import { readFileSync, realpathSync } from 'node:fs' +import { globSync, readFileSync, realpathSync } from 'node:fs' import { relative, resolve } from 'node:path' -import { glob } from 'node:fs/promises' import { fromMarkdown } from 'mdast-util-from-markdown' import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfm } from 'micromark-extension-gfm' @@ -76,7 +75,7 @@ const seen = new Set() const all: Violation[] = [] let checked = 0 for (const pattern of PATTERNS) { - for await (const match of glob(pattern, { cwd: root })) { + for (const match of globSync(pattern, { cwd: root })) { const abs = resolve(root, match) // CLAUDE.md symlinks resolve onto AGENTS.md; dedupe by real path so a file // matched twice (or via symlink) is checked once. diff --git a/scripts/verify-mermaid.ts b/scripts/verify-mermaid.ts index 954c246640..3f9af495b3 100644 --- a/scripts/verify-mermaid.ts +++ b/scripts/verify-mermaid.ts @@ -12,9 +12,8 @@ * Run: `tsx scripts/verify-mermaid.ts`. */ -import { readFileSync, realpathSync } from 'node:fs' +import { globSync, readFileSync, realpathSync } from 'node:fs' import { resolve } from 'node:path' -import { glob } from 'node:fs/promises' import { fromMarkdown } from 'mdast-util-from-markdown' import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfm } from 'micromark-extension-gfm' @@ -72,7 +71,7 @@ const blocks: Block[] = [] const seen = new Set() let checkedFiles = 0 for (const pattern of PATTERNS) { - for await (const match of glob(pattern, { cwd: root })) { + for (const match of globSync(pattern, { cwd: root })) { const real = realpathSync(resolve(root, match)) if (seen.has(real)) continue seen.add(real) diff --git a/scripts/verify-package-paths.ts b/scripts/verify-package-paths.ts index 7bec754dba..5d2d91982b 100644 --- a/scripts/verify-package-paths.ts +++ b/scripts/verify-package-paths.ts @@ -39,9 +39,8 @@ * Run: `tsx scripts/verify-package-paths.ts`. */ -import { existsSync, readdirSync, readFileSync, realpathSync } from 'node:fs' +import { existsSync, globSync, readdirSync, readFileSync, realpathSync } from 'node:fs' import { relative, resolve } from 'node:path' -import { glob } from 'node:fs/promises' const root = resolve(import.meta.dirname, '..') @@ -144,7 +143,7 @@ const all: Violation[] = [] let checked = 0 const seen = new Set() for (const pattern of PATTERNS) { - for await (const match of glob(pattern, { cwd: root })) { + for (const match of globSync(pattern, { cwd: root })) { if (isExcluded(match)) continue // Dedup by real path: the root/packages CLAUDE.md are symlinks to AGENTS.md. const real = realpathSync(resolve(root, match)) diff --git a/scripts/verify-translation-pairing.ts b/scripts/verify-translation-pairing.ts index eea30e4b22..3d80572e7c 100644 --- a/scripts/verify-translation-pairing.ts +++ b/scripts/verify-translation-pairing.ts @@ -44,9 +44,8 @@ */ import { createHash } from 'node:crypto' -import { existsSync, readFileSync, writeFileSync } from 'node:fs' +import { existsSync, globSync, readFileSync, writeFileSync } from 'node:fs' import { basename, join, resolve } from 'node:path' -import { glob } from 'node:fs/promises' import { fromMarkdown } from 'mdast-util-from-markdown' import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfm } from 'micromark-extension-gfm' @@ -211,7 +210,7 @@ function parse(content: string): Nodes { // Enumerate the scope once. const files = new Set() for (const pattern of SCOPE_PATTERNS) { - for await (const match of glob(pattern, { cwd: root })) files.add(match) + for (const match of globSync(pattern, { cwd: root })) files.add(match) } const translations = [...files].filter(f => f.endsWith('.zh.md')).sort() const metas = [...files].filter(f => f.endsWith('.i18n.yaml')).sort() diff --git a/scripts/verify-type-equiv.ts b/scripts/verify-type-equiv.ts index c93383e40f..85ccd642d9 100644 --- a/scripts/verify-type-equiv.ts +++ b/scripts/verify-type-equiv.ts @@ -22,9 +22,8 @@ * Run: `tsx scripts/verify-type-equiv.ts`. */ -import { readFileSync, existsSync } from 'node:fs' +import { globSync, readFileSync, existsSync } from 'node:fs' import { resolve } from 'node:path' -import { glob } from 'node:fs/promises' import ts from 'typescript' const root = resolve(import.meta.dirname, '..') @@ -148,7 +147,7 @@ const keyOf = (x: { doc: string; symbol: string }): string => `${x.doc}::${x.sym // as an orphan rather than silently skipped. const docSet = new Set() for (const pattern of MARKDOWN_GLOBS) { - for await (const match of glob(pattern, { cwd: root })) docSet.add(match) + for (const match of globSync(pattern, { cwd: root })) docSet.add(match) } const blocks: EquivBlock[] = [...docSet].sort().flatMap(extractEquivBlocks) diff --git a/tsconfig.base.json b/tsconfig.base.json index 40e4dbe728..e587d44a3d 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -43,10 +43,13 @@ "./packages/core/*/src", "./packages/llm/*/src", "./packages/bash/*/src", + "./packages/code-runtime/*/src", "./packages/fs/*/src", "./packages/compact/*/src", + "./packages/guard/*/src", "./packages/subagent/*/src", "./packages/web/*/src", + "./packages/timeout/*/src", "./packages/todo/*/src", "./packages/hooks/*/src", "./packages/session-persistence/*/src", diff --git a/tsconfig.build.json b/tsconfig.build.json index b6ba7901f2..64fa70e9f9 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -11,6 +11,7 @@ { "path": "./vendor/hmr" }, { "path": "./vendor/logger-console" }, { "path": "./packages/util/brand" }, + { "path": "./packages/util/timeout" }, { "path": "./packages/llm/llm" }, { "path": "./packages/core/session" }, { "path": "./packages/session-persistence/session-persistence" }, @@ -22,6 +23,8 @@ { "path": "./packages/core/agent-loop" }, { "path": "./packages/core/agent-core" }, { "path": "./packages/bash/bash" }, + { "path": "./packages/code-runtime/code-runtime" }, + { "path": "./packages/code-runtime/code-runtime-worker" }, { "path": "./packages/compact/compact" }, { "path": "./packages/compact/compact-basic" }, { "path": "./packages/llm/llm-deepseek" }, @@ -38,12 +41,14 @@ { "path": "./packages/web/web-search-deepseek" }, { "path": "./packages/web/web-fetch-local" }, { "path": "./packages/web/tool-web" }, + { "path": "./packages/timeout/timeout-policy" }, { "path": "./packages/support/invariants" }, { "path": "./packages/ui/acp" }, { "path": "./packages/ui/acp-agent" }, { "path": "./packages/ui/app-boot" }, { "path": "./packages/ui/stdio-agent" }, { "path": "./packages/support/llm-replay" }, + { "path": "./packages/support/acp-snapshot" }, { "path": "./packages/subagent/subagent" }, { "path": "./packages/support/subagent-mock" }, { "path": "./packages/subagent/tool-subagent" }, @@ -52,6 +57,7 @@ { "path": "./packages/subagent/subagent-fork" }, { "path": "./packages/subagent/subagent-acp" }, { "path": "./packages/todo/tool-todo" }, + { "path": "./packages/guard/repeat-tool-guard" }, { "path": "./packages/hooks/hook-protocol" }, { "path": "./packages/hooks/hooks-claude" }, { "path": "./packages/hooks/hooks-codex" } diff --git a/tsconfig.json b/tsconfig.json index 9cd7aa8a6d..8feeea4fce 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -22,6 +22,7 @@ { "path": "./vendor/hmr" }, { "path": "./vendor/logger-console" }, { "path": "./packages/util/brand" }, + { "path": "./packages/util/timeout" }, { "path": "./packages/llm/llm" }, { "path": "./packages/core/session" }, { "path": "./packages/session-persistence/session-persistence" }, @@ -33,6 +34,8 @@ { "path": "./packages/core/agent-loop" }, { "path": "./packages/core/agent-core" }, { "path": "./packages/bash/bash" }, + { "path": "./packages/code-runtime/code-runtime" }, + { "path": "./packages/code-runtime/code-runtime-worker" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, { "path": "./packages/bash/bash-local" }, @@ -49,12 +52,14 @@ { "path": "./packages/web/web-search-deepseek" }, { "path": "./packages/web/web-fetch-local" }, { "path": "./packages/web/tool-web" }, + { "path": "./packages/timeout/timeout-policy" }, { "path": "./packages/support/invariants" }, { "path": "./packages/ui/acp" }, { "path": "./packages/ui/acp-agent" }, { "path": "./packages/ui/app-boot" }, { "path": "./packages/ui/stdio-agent" }, { "path": "./packages/support/llm-replay" }, + { "path": "./packages/support/acp-snapshot" }, { "path": "./packages/subagent/subagent" }, { "path": "./packages/support/subagent-mock" }, { "path": "./packages/subagent/tool-subagent" }, @@ -63,6 +68,7 @@ { "path": "./packages/subagent/subagent-fork" }, { "path": "./packages/subagent/subagent-acp" }, { "path": "./packages/todo/tool-todo" }, + { "path": "./packages/guard/repeat-tool-guard" }, { "path": "./packages/hooks/hook-protocol" }, { "path": "./packages/hooks/hooks-claude" }, { "path": "./packages/hooks/hooks-codex" } diff --git a/vitest.config.ts b/vitest.config.ts index 4200b5c7f4..11d1454b08 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -31,7 +31,12 @@ export default defineConfig({ // can't import one without booting it, so they are driven by the keyless // Loader-path smoke (a real subprocess) instead of the in-process unit // suite — the same reason `examples/start.ts` sat out of coverage scope. - exclude: ['packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts'], + // `worker.ts` files are the same class as bin.ts: self-executing + // worker-thread entrypoints that only ever run inside a spawned isolate + // the v8 provider cannot observe. They stay thin glue over in-process- + // tested logic (bootstrap.ts) and are pinned by real-worker integration + // tests. + exclude: ['packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts', 'packages/*/*/src/worker.ts'], // 100% or it doesn't merge (docs/testing.md: excessive tests are welcome). // Per-file so a well-covered big file can't subsidize a bare one. // Every v8 ignore comment must carry a reason — see the quality-gates RFC