Merge remote-tracking branch 'origin/master' into codex/ask-user-question

# Conflicts:
#	docs/config-catalog.md
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	pnpm-lock.yaml
This commit is contained in:
Yichen Jiang
2026-07-08 11:09:20 +08:00
158 files changed
+5099 -476

No files matched your search

@@ -66,7 +66,7 @@ flowchart LR
`@deepseek-ai/dsh-web` depends only on Cordis and low-level harness support. It declares `ctx.web`, provider interfaces, request/result types, the provider status type, and error codes. It does not import tool, agent, session, LLM, or provider packages.
Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credentials, endpoint config, provider-specific request mapping, provider-specific response parsing, and provider-specific error translation into `WebError`. They issue network requests with the platform-native `fetch` (Node 24), mirroring `@deepseek-ai/dsh-llm-deepseek`'s adapter, NOT a cordis HTTP-client service (`ctx.http`/`@cordisjs/plugin-http`) — even where a Perplexity provider's request is shaped like an OpenAI-compatible chat completion, that wire shape is a provider-private detail and does not make the provider depend on `ctx.llm`. A provider does NOT own the `ctx.web` key (two search providers cannot both own it): like `dsh-llm-deepseek`, each provider package is a function/namespace plugin (`inject: ['web']`) whose `apply` constructs the backend and calls `ctx.web.registerSearchProvider` / `registerFetchProvider`. `@deepseek-ai/dsh-web` is the `export default` service that owns the key.
Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credentials, endpoint config, provider-specific request mapping, provider-specific response parsing, and provider-specific error translation into `WebError`. They issue network requests with platform-native `fetch` at the repo's Node floor, mirroring `@deepseek-ai/dsh-llm-deepseek`'s adapter, NOT a cordis HTTP-client service (`ctx.http`/`@cordisjs/plugin-http`) — even where a Perplexity provider's request is shaped like an OpenAI-compatible chat completion, that wire shape is a provider-private detail and does not make the provider depend on `ctx.llm`. A provider does NOT own the `ctx.web` key (two search providers cannot both own it): like `dsh-llm-deepseek`, each provider package is a function/namespace plugin (`inject: ['web']`) whose `apply` constructs the backend and calls `ctx.web.registerSearchProvider` / `registerFetchProvider`. `@deepseek-ai/dsh-web` is the `export default` service that owns the key.
`@deepseek-ai/dsh-tool-web` depends on `@deepseek-ai/dsh-web`, `@deepseek-ai/dsh-tools`, `@deepseek-ai/dsh-system-prompt`, and Cordis. It never imports concrete provider packages.
@@ -0,0 +1,49 @@
# RFC: Explicit model-facing tool order
Status: implemented
## Problem
The order of the tool list a model call carries — `request/header.tools` on the session log and `GenerateOptions.tools` on the wire — was an emergent artifact: the tool registry returns schemas in registration order, the system-prompt assembly concatenates providers in registration order, and the loop logged and dispatched the result verbatim. Registration order is plugin load order, and plugin load order is a race: the cordis loader imports every `cordis.yml` entry concurrently, so which tool plugin registers first depends on module-import completion timing. The plugin dependency relation cannot rescue this — it is a partial order under which independent tool plugins (e.g. `tool-subagent` vs `tool-todo`) are incomparable, so both interleavings are legal linearizations. This stopped being theoretical when a CI runner resolved the race differently from every recording machine: snapshot goldens pinned one permutation of `request/header.tools`, the `node 22.18` CI leg produced the other, and 5/5 snapshot tests failed on a diff that was pure array reordering. Tool order is part of the request bytes (prompt-cache stability, potentially model behavior) and, since the reconstructability contract, part of the durable session log — it must be a decision, not a residue.
## Decision
The system-prompt assembly owns the canonical model-facing tool order, exactly where it already owns section order. `toolOrder?: string[]` on `dsh-system-prompt` is the optional explicit policy:
- A listed tool that is registered takes its listed position.
- A listed name with no registered tool is a configuration error. Shape errors (rest entry missing or duplicate names) fail from the service constructor; an unregistered name rejects every `assemble()` — the earliest moment the registered tool set exists to check against (tool plugins register after the service constructs), and the only universal one (registrations can change at any time; cordis has no "all plugins loaded" event). Under the shipped loop the first turn fails before any model request — see the consequences below for the exact blast radius.
- A registered tool absent from the list is inserted at the `'<unlisted-tools>'` rest entry (`TOOL_ORDER_REST`), in lexicographic name order among the other unlisted tools.
- No collected tool may use `TOOL_ORDER_REST` as its `ToolSchema.name`; the assembly rejects that reserved name before ordering.
- The list must contain the rest entry exactly once and no duplicate names.
- When `toolOrder` is unset, the canonical order is plain lexicographic name order (code-unit comparison, locale-independent), so determinism requires no configuration.
The policy is applied where the list is born: `assemble()`, before the `system-prompt/assemble` waterfall. The assembly canonicalizes the tools it collects from providers the same way it sorts sections by their `order` field — on the initial assembly, killing the registration-order entropy at its source. Everything downstream inherits the order untouched: the waterfall, the loop's `EpochHeader`, the `request/header` event, the deep-frozen request, and the dev invariant's cross-check all see one deterministic list, with no new loop change.
Scope is deliberately narrow: this fixes the REGISTRATION-ORDER race, not plugin behavior. A `system-prompt/assemble` listener may still add, remove, or rearrange tools — same as it may edit sections after their sort — and owns the determinism of what it emits; the waterfall contract already demands deterministic listeners (the reconstructability invariant would catch a listener that diverges between build and replay).
Config plumbing follows the `persona` precedent, and `toolOrder` sits beside it: the app configs (`dsh-stdio-agent`, `dsh-acp-agent`) accept the key and forward it through `dsh-agent-core` (whose schema is the intersection of the owners' schemas) to the `SystemPrompt` child. One schemastery footnote is load-bearing: a schemastery array defaults to `[]`, but an omitted `toolOrder` must stay ABSENT (= lexicographic) rather than become an explicitly-configured empty list (invalid — it lacks the rest entry), so every schema on the chain forces the default to `undefined`.
## Alternatives considered
- **Registration order (the status quo)** — a concurrent-import race, host-dependent (the CI flake above), invisible in review.
- **A linearization of the plugin dependency graph** — the relation is partial and independent tool plugins are incomparable; the flake happened with the partial order fully satisfied.
- **Per-plugin `weight` on each tool contribution** — scatters the order across plugins yet still needs a global numbering convention nobody owns (the section `order` bands show that coordination cost being paid by hand).
- **Sorting in `ToolRegistry.schemas()` (the registry layer)** — equally deterministic, but the registry is a membership store consumed by more than the assembly; ordering is a prompt-composition concern, and the assembly already owns the composition policy for sections.
- **A `LlmService` config + `orderTools()` method the loop calls before logging the header** — works, but adds a public service method and a loop edit solely to apply a policy at a distance; every future request composer must remember the call. Canonicalizing where the list is born makes an unordered list unrepresentable, with zero new surface.
- **Normalizing inside `llm.stream()`** — runs after the header event is logged (the flake survives) and rebuilds the deep-frozen envelope, silently disarming the reconstruction invariant.
- **An exhaustive list (no rest entry)** — every newly loaded tool plugin would break boot; the mandatory rest entry keeps unlisted tools deterministic and their position explicit.
- **A boot-time validation pass (a `SystemPrompt.assertToolOrderSatisfied()` called by `dsh-app-boot` after `loader.await()`)** — would turn the misconfiguration into a startup death instead of a first-turn failure, but costs a public service method plus a structural coupling from the generic boot glue to one service, and cannot replace the assembly-time check anyway (embedded callers never run app boot; registrations change after boot). No existing event can host the check either: cordis v4 has no ready-like event, `loader/entry-init`/`internal/status` fire mid-load (racy against tool registration, the very entropy this RFC kills), and the agent lifecycle events are no earlier than the assembly. One enforcement point at `assemble()` was judged worth the later failure moment.
## Consequences
- Every assembly — and therefore every `request/header` event and model request — has a deterministic tool order on every host; the CI-vs-local golden flip is structurally gone. The default order is lexicographic, no longer registration order.
- `PromptAssembly.tools` itself is canonical, so every assembly consumer (the loop, waterfall listeners, any future prompt inspector) sees the model-facing order; provider registration order is observable nowhere downstream of the registry.
- The snapshot suite's single pinned request-header fixture (`text-turn`) carries the new canonical tool order; every other ACP snapshot keeps the header bulk scrubbed as `{{system}}`/`{{tools}}`, per the pinned-header design.
- A pure tool reordering between steps is representable only as a `request/header` `'fallback'` snapshot (the name-keyed `ToolsDelta` cannot express it); with a stable canonical order such reorders no longer occur in practice, so the fallback path stays a safety valve.
- The `toolOrder` key rides the app → `agent-core``SystemPrompt` forwarding chain, so deployments set it next to `persona` in the app config; `dsh-llm` and the agent loop are untouched.
- A misspelled or unloaded tool name in `toolOrder` fails the turn at prompt assembly, not the boot: the loop assembles inside the turn (after `turn/start`, before `step/start`), so the rejection reaches the turn's outer catch — the turn closes balanced with an `error` reason carrying the message, `agent/error` mirrors it, no step opens, no `request/header` is logged, no request reaches the adapter, and the agent returns to idle. Every turn fails identically until the config is fixed; the process itself stays up (matching the repo rule that explicit config references must not be silently ignored — the enforcement point is the assembly because no earlier universal moment exists).
- A tool provider that returns the reserved rest-entry name has the same prompt-assembly failure shape as an unknown listed name. This keeps the sentinel from becoming an ambiguous real tool and preserves the "never drops a tool" ordering contract.
## Testing
Unit tests on `dsh-system-prompt` pin the ordering semantics (lexicographic default, listed/rest placement, unknown-name rejection at assembly, reserved tool-name rejection, stable handling of shared names, provider-order independence), the pre-waterfall contract (listeners observe the canonical list; a listener-appended tool is not re-sorted), and each invalid-list rejection at load. Loop-level tests assert the `request/header` fold carries the canonical order for scrambled registration orders (identical across permutations), that a configured `toolOrder` reaches both the logged header and the dispatched deep-frozen request, that the frozen loop-built envelope survives to the adapter, and that an unregistered `toolOrder` name fails the turn with a balanced `error` `turn/end`, an `agent/error`, no step, no logged header, and no dispatched request. Forwarding is asserted at every level that exposes the key (`dsh-agent-core`, `dsh-stdio-agent`, `dsh-acp-agent`). The snapshot tier replays all scenarios while only the pinned `text-turn` header carries the full canonical tool list; non-pinning fixtures continue to compare through `{{tools}}`.
@@ -14,7 +14,7 @@ Every AGENTS.md promise gets a command that exits non-zero, wired into git hooks
- ESLint strict-type-checked + @stylistic (the house style, enforced); vendored code excluded.
- Per-file 100% coverage on `packages/*/src` (v8); unreachable defensive guards carry `/* v8 ignore */ ` with stated reasons instead of deletion.
- knip (dead code/deps), publint (package correctness), workspace constraints (workspace rules: private, cordis peer+dev, uniform version, ESM), and a NodeNext consumer typecheck for built package declarations.
- lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and pre-push (tests, hygiene); CI runs the full matrix on node 24/26 plus a demo smoke test driving the echo-agent end to end.
- lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and pre-push (tests, hygiene); CI runs the full matrix on node 22.19/24/26 plus a demo smoke test driving the echo-agent end to end.
## Consequences
@@ -0,0 +1,43 @@
# RFC: Export-surface JSDoc gate
Status: implemented
## Problem
The [cordis JSDoc completeness gate](2026-07-04-cordis-jsdoc-completeness-gate.md) made undocumented parameters and results impossible on the cordis surface — `interface Events` members and `ctx.<key>` service classes — but that surface is a fraction of what a plugin author imports. The AGENTS.md rule "every export (and non-obvious method) has a JSDoc explaining semantics" stayed prose-checkable only by review everywhere else, and nothing at all asked for `@param`/`@returns` on ordinary exported functions. A survey at adoption found 203 under-documented module-level exports across 34 packages: seam-adjacent helpers (`runBash`, `readForEdit`, `htmlToMarkdown`), format codecs, whole undocumented interfaces and type aliases — exactly the names an IDE consumer hovers.
## Decision
A new gate, `scripts/verify-export-jsdoc.ts` (`pnpm run verify-export-jsdoc`, wired into `doc-sync` beside `verify-cordis-catalog`), walks every module-level exported name under each `packages/<group>/<pkg>/src/` tree. The parsing and check helpers moved from `gen-cordis-catalog.ts` into a shared `scripts/jsdoc.ts`, so "documented" means the same thing on both surfaces: description prose ends at the first block tag, every checkable parameter needs a non-empty `@param`, a non-void ANNOTATED return needs a non-empty `@returns`, a stale `@param` errors, and violations aggregate into one report.
The contract by declaration kind:
- Every exported name needs JSDoc with non-empty description prose.
- Function-like exports (function declarations; consts with function initializers or an INLINE callable annotation; non-identifier function default exports) follow the full function contract, with wrapper expressions (parentheses, `as`/`satisfies` casts, non-null assertions) peeled before classifying. A const whose declarator is annotated with a NAMED type (`export const f: Handler = …`) defers the signature contract to that type's own declaration and `@returns` stays optional; an inline `(x: T) => U` annotation or single-call-signature literal is the surface signature itself and gets the full contract, and a literal mixing call/construct signatures with anything else is refused outright (no single signature to hold the tags against — extract a named type).
- Exported classes need class-level prose; public methods (statics included — reachable on the exported name) follow the function contract; public properties and accessors need prose (a get/set pair is covered by the getter). Overload implementations are exempt — the signatures carry the docs.
- Exported interfaces, type aliases, and enums need prose on the declaration; member-level enforcement is deliberately deferred (the highest-value member surface — seam service classes — is already under the cordis gate).
- Exported namespaces recurse (inside an ambient `declare` namespace every member exports implicitly); the namespace itself needs prose only when it does not merge with a documented same-name declaration (the Config-namespace idiom documents the plugin once).
- `declare module` / `declare global` bodies and `export … from` re-export statements are skipped: an augmentation is not an export of the package, and a re-exported definition is checked where it is defined. An `export import X = N.member` alias documents ITSELF — its target may be a non-exported namespace member no walk visits — and only prose-only target kinds are gate-supported: a callable, class, or namespace target carries signature/member contracts the alias prose cannot hold, so the gate refuses it and demands the declaration be exported directly.
- Everything else fails CLOSED: `export =` is refused outright, parameters the base never names keep their `@param` duty even as binding patterns, and an exported statement kind the dispatch does not recognize is itself a violation — no export form can pass unchecked by omission.
Three exemption families keep the gate from demanding boilerplate, in the spirit of the cordis gate's `this`/`next` exemptions (documenting an exempt name anyway is allowed; only absence goes unchecked):
- **Heritage members.** A class member whose name exists on an `extends`/`implements` heritage type is exempt: the seam declaration is the doc's one home, and the IDE inherits it on hover — re-documenting every `LocalBashExecutor.run` invites drift. The exemption stops where the override grows surface the base never documented: a protected-only base member does not exempt a public override, parameters the base never names keep their `@param` duty (an underscore-prefixed rename of a base parameter — the deliberately-unused marker — is the same parameter), and a concrete result above a void base return keeps its `@returns` duty (an unannotated override's inferred return is classified by the checker, so a faithful void override needs no boilerplate annotation). Heritage lookups and that one return classification are the walk's only TYPE CHECKER questions (heritage types live across package boundaries, resolved through the repo `paths` map); everything else stays pure AST, and the annotated-return requirement is kept for symmetry with the cordis gate (it bound nothing at adoption — every exported function was already annotated).
- **Plugin-protocol slots.** Top-level `name` / `inject` / `reusable` / `Config` consts and the `apply` entry, plus the same slots as statics on a plugin class, are framework protocol: their shape is fixed by cordis, and the module doc comment plus the `interface Config` carry the plugin's real semantics.
- **Constructors**, mirroring the cordis gate: plugin classes are framework-constructed, and the class doc owns the story.
`collectExportJsdocViolations()` returns the violation list (the CLI exits 1 on non-empty) so the negative-path tests in `packages/core/agent/tests/verify-export-jsdoc.spec.ts` assert on findings directly, driving fixture packages through every rejection and every exemption.
## Alternatives considered
- **eslint-plugin-jsdoc** (`require-jsdoc`/`require-param`/`require-returns`) — covers the mechanical core but cannot express the repo's contract: the heritage-member exemption needs cross-package type resolution, the protocol-slot and namespace-merge idioms are cordis-specific, and the completeness semantics (prose-above-tags, stale-tag errors, aggregate reporting) already have one home in `scripts/jsdoc.ts` shared with the catalog generator. Two subtly different definitions of "documented" is the failure mode this repo's one-home rule exists to prevent.
- **Extending `gen-cordis-catalog.ts`** — the catalog generator renders a curated surface and gates its freshness; a repo-wide walk has no catalog to render. Sharing the helpers while keeping the walks separate keeps each gate's scope legible.
- **Enforcing interface/type-alias member docs** — deferred: it would multiply the checked surface for members that are largely self-describing fields, while the seam classes carrying the load-bearing member contracts are already gated. Revisit if member-doc drift shows up in review.
## Consequences
- A new export cannot land undocumented: `verify-export-jsdoc` fails `doc-sync`, which pre-push and CI already run. The 203 gaps found at adoption were filled in the same change, so the gate landed green.
- Exported functions must annotate return types (universal at adoption, now load-bearing) and use identifier parameters where `@param` must name them.
- Seam docs are canonical: an implementation inherits its heritage docs, and behavior notes worth keeping on the implementation are additions, not requirements.
- The gate builds a `ts.Program` (~6s) — the one doc gate that pays for type resolution; acceptable inside `doc-sync`, which already compiles doc snippets.
- The protocol-slot names are reserved by convention at module top level; a non-protocol export coincidentally named `apply` or `Config` would go unchecked — accepted, documented here.
@@ -0,0 +1,37 @@
# RFC: Raise the Node LTS engine floor to 22.19
Status: implemented
## Problem
The Node 22 branch of the root `engines.node` range is a contract for the installed workspace, not only for the runtime APIs the harness source calls directly. It must be no lower than package `engines.node` declarations for dependencies the workspace installs on that branch; otherwise `pnpm install --engine-strict` fails at an advertised LTS version, and non-strict installs run outside a dependency's supported runtime.
## Decision
Set `engines.node` to `^22.19.0 || >=24.0.0` and test the keyless CI compatibility matrix on `['22.19', 24, 26]`. The real-API e2e workflow stays on Node 24 because it exercises API integration rather than the runtime floor.
Two Node features gate the source runtime:
- **`node:sqlite`** — `packages/session-persistence/session-persistence-sqlite` does a top-level `import { DatabaseSync } from 'node:sqlite'`. The module dropped its `--experimental-sqlite` flag requirement at **22.13** (LTS) and **23.4** (Current); before those, importing it throws at load.
- **Native TypeScript type-stripping** — the `packages/ui/stdio-agent/tests/built-bin.e2e.ts` smoke boots the published `lib/bin.js` under plain `node` (no tsx) and loads the example's `.ts` plugins (`mock-llm.ts`, `echo-tool.ts`). Type-stripping is the default from **22.18** (LTS) and **23.6** (Current); before those it needs `--experimental-strip-types`.
Those source features clear on the 22.x line at **22.18**, but the installed Pi adapter dependency raises the advertised LTS floor. `@deepseek-ai/dsh-llm-pi-ai` depends on `@earendil-works/pi-ai@0.79.3`, whose package declares `engines.node >=22.19.0`, so the LTS floor is **22.19**. The 24.x branch remains `>=24.0.0`. The disjoint range excludes Node 23 entirely: Node 23.023.5 still has at least one flagged source feature, and the 23 line is non-LTS/EOL, so advertising `>=23.6` would add a dead release line and a CI leg no deployment should use.
`@types/node` remains pinned to the 22.x line (`^22.20.0`) to match the LTS support line: reaching for a Node 23+/24+/25+ API fails `tsc` on every machine and in the typecheck gate, rather than compiling clean and surviving to a runtime failure only a floor matrix leg could catch. The whole tree typechecks clean against the Node 22 type surface today, so the pin costs nothing.
## Consequences
- The advertised LTS branch no longer undercuts the Pi adapter dependency floor.
- CI proves the Node 22 LTS floor directly with Node 22.19, keeps the Node 24 branch on `node: 24`, and keeps Node 26 for the next even line.
- The built-bin smoke needs no version-conditional flag: at 22.19 type-stripping is already the default, so the test stays the plain `node lib/bin.js` path it documents.
- A future dependency or source API that raises the runtime floor must move `engines.node`, the compatibility matrix, and this RFC in the same change.
## Alternatives considered
- **Keep `^22.18.0 || >=24.0.0`.** Rejected: it advertises an LTS version lower than the Pi adapter dependency floor. `@earendil-works/pi-ai@0.79.3` requires `>=22.19.0`.
- **Downgrade or pin `@earendil-works/pi-ai` to preserve the 22.18 advertised range.** Rejected: the current Pi adapter dependency is part of the intended workspace, and 22.19 is still inside the Node 22 LTS line.
- **Floor `>=22.13` (the `node:sqlite` boundary) plus `--experimental-strip-types` in the built-bin smoke on 22.1322.17.** Rejected: it adds a version-conditional test flag for one narrow range and dresses up an experimental-flag dependency as first-class support. The Pi adapter dependency already requires a higher LTS floor.
- **Open-ended `>=22.19`.** Rejected: it advertises support for Node 23.023.5, where `node:sqlite` (until 23.4) or type-stripping (until 23.6) is still flagged.
- **Include Node 23.6+ (`^22.19.0 || >=23.6.0`).** Rejected: 23.6+ does run both source features unflagged, but Node 23 is end-of-life; advertising a dead release line adds a range term and a CI leg for a runtime no deployment should use.
- **Matrix `[22, 24, 26]` instead of pinning `22.19`.** Rejected: floating major-version entries drift upward over time and silently stop exercising the declared LTS floor.
- **Keep `@types/node` ahead of the floor (`^25`).** Rejected: types ahead of the runtime floor let a Node 24/25-only API compile clean and fail only at runtime on 22.x. Pinning `@types/node` to the 22.x line turns that into a compile error everywhere.
@@ -54,7 +54,7 @@ The repo secret is named `DEEPSEEK_API_KEY_EXTERNAL`; it is mapped to the `DEEPS
### Scope, runtime shape
Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the `engines` floor): these tests exercise *API integration*, not node-version compat, which ci.yml's Node 24/26 jobs already own; a second Node version would double real-API calls for no added signal. `vitest.e2e.config.ts` runs files through a bounded worker pool (`DSH_E2E_MAX_WORKERS`, default `4`, CI value `14`) so CI and local with-key runs parallelize independent files while retaining a one-line serial escape hatch for quota investigations. `timeout-minutes: 45` bounds a wedged run given 120s/test and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled.
Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the primary line): these tests exercise API integration, not node-version compatibility, which ci.yml's Node 22.19/24/26 matrix owns. `vitest.e2e.config.ts` runs files through a bounded worker pool (`DSH_E2E_MAX_WORKERS`, default `4`, CI value `14`) so CI and local with-key runs parallelize independent files while retaining a one-line serial escape hatch for quota investigations. `timeout-minutes: 45` bounds a wedged run given 120s/test and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled.
## Security