From 066f94c7e08e3d34b5ea3e605d082089dea00608 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 13 Jun 2026 18:39:20 +0800 Subject: [PATCH] docs: unwrap hard-wrapped Markdown to one line per paragraph MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. Reflow all tracked non-vendor Markdown (plus vendor/AGENTS.md) so each prose paragraph is a single line; soft-wrapping is the editor's job. Fenced code, tables, and list structure are preserved (wrapped list items fold to one line per bullet). Documents the convention in AGENTS.md. --- AGENTS.md | 205 +++--------------- docs/adr/0001-vendor-cordis-as-source.md | 28 +-- docs/adr/0002-microkernel-event-taxonomy.md | 30 +-- docs/adr/0003-event-sourced-sessions.md | 28 +-- docs/adr/0004-own-content-block-vocabulary.md | 30 +-- ...0005-custom-schema-dsl-over-schemastery.md | 28 +-- .../0006-tool-schemas-in-prompt-assembly.md | 24 +- docs/adr/0007-quality-gates.md | 31 +-- docs/adr/0008-tsdown-over-dumble.md | 44 +--- docs/adr/README.md | 8 +- docs/architecture.md | 182 ++++------------ docs/cookbook/adding-a-package.md | 20 +- docs/cookbook/adding-a-tool.md | 39 +--- docs/cookbook/adding-an-llm-adapter.md | 58 ++--- docs/rfc/001-property-based-testing.md | 37 +--- docs/rfc/002-mutation-testing.md | 26 +-- .../003-deterministic-and-stress-testing.md | 29 +-- docs/rfc/004-architectural-conformance.md | 30 +-- ...5-runtime-validation-and-error-taxonomy.md | 40 +--- docs/rfc/006-doc-sync-and-api-reports.md | 30 +-- docs/rfc/007-supply-chain-and-vendor-drift.md | 33 +-- docs/rfc/008-immutable-public-surfaces.md | 34 +-- docs/rfc/README.md | 5 +- examples/coding-agent/README.md | 19 +- examples/echo-agent/README.md | 19 +- packages/AGENTS.md | 28 +-- packages/README.md | 36 +-- packages/agent-loop/README.md | 27 +-- packages/agent/README.md | 35 +-- packages/bash-local/README.md | 38 +--- packages/bash/README.md | 22 +- packages/llm-deepseek/README.md | 63 ++---- packages/llm-pi-ai/README.md | 42 +--- packages/llm/README.md | 46 ++-- packages/session/README.md | 39 +--- packages/system-prompt/README.md | 30 +-- packages/tool-bash/README.md | 41 +--- packages/tools/README.md | 40 +--- vendor/AGENTS.md | 10 +- 39 files changed, 348 insertions(+), 1206 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index dd4b75562f..54aed8ecac 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,200 +63,59 @@ yarn demo:coding # run examples/coding-agent — the real agent (needs ## Secrets / .env -Real-API e2e tests (`yarn test:e2e`) read `DEEPSEEK_API_KEY` (and optionally -`DEEPSEEK_BASE_URL`) from the environment, or from a gitignored `.env` at the -repo root loaded via Node's native `process.loadEnvFile()`: +Real-API e2e tests (`yarn test:e2e`) read `DEEPSEEK_API_KEY` (and optionally `DEEPSEEK_BASE_URL`) from the environment, or from a gitignored `.env` at the repo root loaded via Node's native `process.loadEnvFile()`: ``` DEEPSEEK_API_KEY=sk-… DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API ``` -cordis.yml configs reference env vars with the `!!js` tag: -`apiKey: !!js process.env.DEEPSEEK_API_KEY`. Never commit real credentials; -CI has no secrets and e2e suites must self-skip without them. +cordis.yml configs reference env vars with the `!!js` tag: `apiKey: !!js process.env.DEEPSEEK_API_KEY`. Never commit real credentials; CI has no secrets and e2e suites must self-skip without them. -Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root -`tsconfig.json` (`vitest` resolves through `tsconfig.test.json`). Building is -only needed for publishing/consumption outside the repo — with one exception: -`yarn lint`'s type-aware rules resolve vendor packages through their built -declarations (`tsconfig.typecheck.json` → `vendor/*/lib`), so run -`yarn typecheck` once after a fresh clone (CI does the same) or lint reports -unresolved-type `no-unsafe-*` errors. +Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root `tsconfig.json` (`vitest` resolves through `tsconfig.test.json`). Building is only needed for publishing/consumption outside the repo — with one exception: `yarn lint`'s type-aware rules resolve vendor packages through their built declarations (`tsconfig.typecheck.json` → `vendor/*/lib`), so run `yarn typecheck` once after a fresh clone (CI does the same) or lint reports unresolved-type `no-unsafe-*` errors. ## Conventions -- **Package naming**: every npm package in this repo is `@deepseek-ai/dsh-` - (vendored packages keep their upstream names and are `private: true`). -- **ESM everywhere** (`"type": "module"`); imports between workspace packages - use package names, never relative paths across package boundaries. - In-package imports use explicit `.ts` extensions (allowImportingTsExtensions). -- **`cordis` is a peerDependency** (+ devDependency) of every harness package, - mirroring upstream convention. -- **Registrations are effects**: anything a plugin contributes (adapter, tool, - section, agent, event listener) goes through `ctx.effect()` / `ctx.on()` so - disposal and HMR work. If you write a registry, `register()` must return the - disposer. -- **Typed events via declaration merging**: services declare their events in - `declare module 'cordis' { interface Events { … } }`, and their ctx key in - `interface Context`. Extensible unions use the merge-extensible-map pattern - (see `ContentBlockMap`, `MessageSourceMap`). -- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)` - and MUST call `next()` to delegate; returning without it short-circuits. - This is the veto mechanism — use deliberately. -- **Discriminated unions: match, don't chain**: branch on a tagged union - (`StreamChunk`, `FinishReason`, `SessionEvent`, …) with a `switch` on the - tag, not a chain of `if (x.kind === '…')`. The switch narrows each arm so - member-only fields (`finish.message`, `finish.code`) are reachable in the - right case and a typo'd tag fails to compile. Prefer extracting a small - typed helper (`finishError(finish: FinishReason)`) over inlining the - branches at the call site. -- **Switch exhaustiveness**: switches over CLOSED unions (e.g. `StreamChunk`) - end with `default: assertNever(value, 'context')` (from dsh-llm) so adding a - variant breaks compilation at every switch that must handle it. Switches - over MERGE-EXTENSIBLE unions (`SessionEventMap`, `ContentBlockMap`, - `FinishReason`, …) must NOT use assertNever — plugin-added variants are - valid unknown values; handle known cases and fall through `default` with a - comment (the lint rule `switch-exhaustiveness-check` makes the choice - explicit either way; a redundant disable directive is itself a lint error). -- **Plugins, not loop changes**: new behavior goes into a plugin on the - documented extension seams (see the plugin sanity checklist in - docs/architecture.md). Changing `agent-loop` requires updating that doc. -- **Capability seams are three packages**: when adding a swappable capability - (an execution backend, a provider integration, …), split it into - *interface* (abstract service + vocabulary types, e.g. `bash/`), - *implementation* (a concrete subclass, e.g. `bash-local/`), and - *consumer* (what the model/plugins see, e.g. `tool-bash/`). Implementations - and consumers then evolve independently — a sandboxed executor replaces - `bash-local` without touching tool schemas. The LLM seam follows the same - shape (`llm/` is interface + consumer surface; adapters are implementations). - See docs/architecture.md § "Capability seams" for when NOT to split. -- **Explicit > implicit at package seams**: interface/vocabulary types spell - out every field a consumer must supply — no optional field that the - implementation silently fills with a hidden `?? default`. Put defaulting in - the owning implementation as an explicit step (a `resolve(request): Spec` - method that turns the optional-field request into the required-field spec), - not smuggled inside `run()`/`start()`. Example: `dsh-bash` splits - `BashExecRequest` (optional `workdir`/`timeoutMs`, model-facing) from - `BashExecSpec` (required, what `run`/`start` act on); the tool layer calls - `ctx.bash.resolve()` between them. The reader of a `BashExecSpec` never has - to wonder where the working directory came from. -- **An empty `catch` must name what it swallows and why nothing else can hit - it**: a bare `catch {}` hides bugs. When you deliberately ignore a throw, the - comment must (a) name the single expected failure, (b) say why ignoring it is - correct — usually because the useful state was already captured *before* the - `try` — and (c) make clear nothing else of consequence can reach the catch - (ideally the `try` wraps a single statement). Example: the error-body - `response.json()` parse in `dsh-llm-deepseek`'s adapter sets `code` + HTTP - `status` from the status line before the `try`, so a malformed provider body - can only cost a richer message, never the real error. -- **Symmetry is usually more correct**: when two related values play parallel - roles (a test fixture and its expected output, a request shape and its - response shape, a buggy input and the test that checks the fix), give them - parallel form — both named consts, or both inline, not one each way. Asymmetry - is a smell that usually points at a missed extraction. -- **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every - registry needs an HMR-safety test (dispose the contributing fiber, assert - cleanup). **Excessive tests are welcome** — when in doubt, write the test; - err on the side of covering edge cases, error paths, event ordering, and - concurrency races even if they seem unlikely. Review findings get regression - tests (see `packages/agent-loop/tests/review-fixes.spec.ts`). +- **Package naming**: every npm package in this repo is `@deepseek-ai/dsh-` (vendored packages keep their upstream names and are `private: true`). +- **ESM everywhere** (`"type": "module"`); imports between workspace packages use package names, never relative paths across package boundaries. In-package imports use explicit `.ts` extensions (allowImportingTsExtensions). +- **`cordis` is a peerDependency** (+ devDependency) of every harness package, mirroring upstream convention. +- **Registrations are effects**: anything a plugin contributes (adapter, tool, section, agent, event listener) goes through `ctx.effect()` / `ctx.on()` so disposal and HMR work. If you write a registry, `register()` must return the disposer. +- **Typed events via declaration merging**: services declare their events in `declare module 'cordis' { interface Events { … } }`, and their ctx key in `interface Context`. Extensible unions use the merge-extensible-map pattern (see `ContentBlockMap`, `MessageSourceMap`). +- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)` and MUST call `next()` to delegate; returning without it short-circuits. This is the veto mechanism — use deliberately. +- **Discriminated unions: match, don't chain**: branch on a tagged union (`StreamChunk`, `FinishReason`, `SessionEvent`, …) with a `switch` on the tag, not a chain of `if (x.kind === '…')`. The switch narrows each arm so member-only fields (`finish.message`, `finish.code`) are reachable in the right case and a typo'd tag fails to compile. Prefer extracting a small typed helper (`finishError(finish: FinishReason)`) over inlining the branches at the call site. +- **Switch exhaustiveness**: switches over CLOSED unions (e.g. `StreamChunk`) end with `default: assertNever(value, 'context')` (from dsh-llm) so adding a variant breaks compilation at every switch that must handle it. Switches over MERGE-EXTENSIBLE unions (`SessionEventMap`, `ContentBlockMap`, `FinishReason`, …) must NOT use assertNever — plugin-added variants are valid unknown values; handle known cases and fall through `default` with a comment (the lint rule `switch-exhaustiveness-check` makes the choice explicit either way; a redundant disable directive is itself a lint error). +- **Plugins, not loop changes**: new behavior goes into a plugin on the documented extension seams (see the plugin sanity checklist in docs/architecture.md). Changing `agent-loop` requires updating that doc. +- **Capability seams are three packages**: when adding a swappable capability (an execution backend, a provider integration, …), split it into *interface* (abstract service + vocabulary types, e.g. `bash/`), *implementation* (a concrete subclass, e.g. `bash-local/`), and *consumer* (what the model/plugins see, e.g. `tool-bash/`). Implementations and consumers then evolve independently — a sandboxed executor replaces `bash-local` without touching tool schemas. The LLM seam follows the same shape (`llm/` is interface + consumer surface; adapters are implementations). See docs/architecture.md § "Capability seams" for when NOT to split. +- **Explicit > implicit at package seams**: interface/vocabulary types spell out every field a consumer must supply — no optional field that the implementation silently fills with a hidden `?? default`. Put defaulting in the owning implementation as an explicit step (a `resolve(request): Spec` method that turns the optional-field request into the required-field spec), not smuggled inside `run()`/`start()`. Example: `dsh-bash` splits `BashExecRequest` (optional `workdir`/`timeoutMs`, model-facing) from `BashExecSpec` (required, what `run`/`start` act on); the tool layer calls `ctx.bash.resolve()` between them. The reader of a `BashExecSpec` never has to wonder where the working directory came from. +- **An empty `catch` must name what it swallows and why nothing else can hit it**: a bare `catch {}` hides bugs. When you deliberately ignore a throw, the comment must (a) name the single expected failure, (b) say why ignoring it is correct — usually because the useful state was already captured *before* the `try` — and (c) make clear nothing else of consequence can reach the catch (ideally the `try` wraps a single statement). Example: the error-body `response.json()` parse in `dsh-llm-deepseek`'s adapter sets `code` + HTTP `status` from the status line before the `try`, so a malformed provider body can only cost a richer message, never the real error. +- **Symmetry is usually more correct**: when two related values play parallel roles (a test fixture and its expected output, a request shape and its response shape, a buggy input and the test that checks the fix), give them parallel form — both named consts, or both inline, not one each way. Asymmetry is a smell that usually points at a missed extraction. +- **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test (dispose the contributing fiber, assert cleanup). **Excessive tests are welcome** — when in doubt, write the test; err on the side of covering edge cases, error paths, event ordering, and concurrency races even if they seem unlikely. Review findings get regression tests (see `packages/agent-loop/tests/review-fixes.spec.ts`). ## Defensive patterns (hard-won) Each bullet is a bug class that bit us; the rule prevents the reoccurrence. -- **Report orthogonal outcomes independently.** A result can be several - things at once (a process can both time out AND exit 0 because it trapped - the signal). Don't nest the report of one flag inside the branch of - another. Surface each independent fact (`timedOut`, `signal`, `exitCode`) - on its own so a caller never reads a cut-short run as a clean success. -- **Honor cross-seam contracts on BOTH sides.** When an interface documents - two valid ways to signal something (e.g. an adapter may report a model - failure by THROWING from `stream()` *or* by ending the stream with a - `finish {kind:'error'|'aborted'}` chunk), the consumer must handle both — - not just the one the first implementation happened to use. A library-backed - adapter that can't throw mid-stream relies on the finish-chunk path; if the - loop only catches throws, a provider 401 becomes a normal completed turn. - Document the contract where the type is defined and exercise every branch - through the real consumer in tests. -- **Async state is not synchronous state.** `agent.send()` does not flip - status to `running` before it returns; a background task's completion races - turn boundaries; `reader.close()` fires for both EOF and disposal. Never - gate control flow on a status you only *just* requested. Drive lifecycle off - the events/promises that actually fire (`agent/status`, `task.done`), and - when "done" needs a settle signal, observe the transition (saw `running` - THEN `idle`) rather than counting actions you assume map 1:1 to turns — - the loop batches queued messages into one turn. But a settle-signal guard - cuts both ways: if the awaited transition can *never* occur (EOF with no - work submitted → no turn ever starts → never `running`), it hangs forever. - Always handle the "nothing to wait for" branch explicitly alongside the - "wait for the work" branch. -- **Dispose must reach quiescence, not just request it.** A teardown that - issues kills/aborts but returns before the work stops leaves orphans. Make - cleanup `async` and `await` the children's exit (kill → await `done`), and - close listener/notification registries *before* killing so late completions - stay silent. Tests must prove disposal *waited* (pid already gone right - after `await fiber.dispose()`), not merely that the process eventually dies. -- **Contain callback exceptions at the boundary.** A user-supplied listener - (`onTaskDone`, event handlers) that throws must not reject the promise it - runs inside or starve the listeners after it. Wrap the dispatch loop in - try/catch and log; never let one bad subscriber break core lifecycle. -- **Never hand untrusted/model output the ambient environment or predictable - paths.** Spawned commands get a scrubbed env (drop `*KEY*`/`*SECRET*`/ - `*TOKEN*`) so the harness's own credentials can't leak into output, `env`, - or spill files. Temp/spill files use a private (0700) dir, random names, - and exclusive owner-only (`'wx'`, `0o600`) opens — predictable - world-readable paths invite symlink races and disclosure. -- **e2e tests own their resources.** Real-API/integration tests must create - the harness in the test and dispose it in `afterEach` (even on - failure/retry/timeout), so a flaky run doesn't leak processes or contexts. - Shared fixtures live in a plain `tests/harness.ts` module, NOT another - `*.e2e.ts` file — importing a spec file re-registers its `describe` and - duplicates real API calls. Verify the WORLD, not the agent's self-report: - re-run the command/check externally and assert files are byte-identical - where they should be unchanged (a keyword probe lets a cheating agent pass). -- **Tag spelling and EOF hygiene.** cordis.yml interpolates env via the - `!!js` tag (js-yaml resolves custom tags under `tag:yaml.org,2002:js`), not - `!js` — keep code, comments, and docs consistent. Files end with exactly - one trailing newline; `git diff --check` (a pre-push gate) rejects new - blank lines at EOF. +- **Report orthogonal outcomes independently.** A result can be several things at once (a process can both time out AND exit 0 because it trapped the signal). Don't nest the report of one flag inside the branch of another. Surface each independent fact (`timedOut`, `signal`, `exitCode`) on its own so a caller never reads a cut-short run as a clean success. +- **Honor cross-seam contracts on BOTH sides.** When an interface documents two valid ways to signal something (e.g. an adapter may report a model failure by THROWING from `stream()` *or* by ending the stream with a `finish {kind:'error'|'aborted'}` chunk), the consumer must handle both — not just the one the first implementation happened to use. A library-backed adapter that can't throw mid-stream relies on the finish-chunk path; if the loop only catches throws, a provider 401 becomes a normal completed turn. Document the contract where the type is defined and exercise every branch through the real consumer in tests. +- **Async state is not synchronous state.** `agent.send()` does not flip status to `running` before it returns; a background task's completion races turn boundaries; `reader.close()` fires for both EOF and disposal. Never gate control flow on a status you only *just* requested. Drive lifecycle off the events/promises that actually fire (`agent/status`, `task.done`), and when "done" needs a settle signal, observe the transition (saw `running` THEN `idle`) rather than counting actions you assume map 1:1 to turns — the loop batches queued messages into one turn. But a settle-signal guard cuts both ways: if the awaited transition can *never* occur (EOF with no work submitted → no turn ever starts → never `running`), it hangs forever. Always handle the "nothing to wait for" branch explicitly alongside the "wait for the work" branch. +- **Dispose must reach quiescence, not just request it.** A teardown that issues kills/aborts but returns before the work stops leaves orphans. Make cleanup `async` and `await` the children's exit (kill → await `done`), and close listener/notification registries *before* killing so late completions stay silent. Tests must prove disposal *waited* (pid already gone right after `await fiber.dispose()`), not merely that the process eventually dies. +- **Contain callback exceptions at the boundary.** A user-supplied listener (`onTaskDone`, event handlers) that throws must not reject the promise it runs inside or starve the listeners after it. Wrap the dispatch loop in try/catch and log; never let one bad subscriber break core lifecycle. +- **Never hand untrusted/model output the ambient environment or predictable paths.** Spawned commands get a scrubbed env (drop `*KEY*`/`*SECRET*`/ `*TOKEN*`) so the harness's own credentials can't leak into output, `env`, or spill files. Temp/spill files use a private (0700) dir, random names, and exclusive owner-only (`'wx'`, `0o600`) opens — predictable world-readable paths invite symlink races and disclosure. +- **e2e tests own their resources.** Real-API/integration tests must create the harness in the test and dispose it in `afterEach` (even on failure/retry/timeout), so a flaky run doesn't leak processes or contexts. Shared fixtures live in a plain `tests/harness.ts` module, NOT another `*.e2e.ts` file — importing a spec file re-registers its `describe` and duplicates real API calls. Verify the WORLD, not the agent's self-report: re-run the command/check externally and assert files are byte-identical where they should be unchanged (a keyword probe lets a cheating agent pass). +- **Tag spelling and EOF hygiene.** cordis.yml interpolates env via the `!!js` tag (js-yaml resolves custom tags under `tag:yaml.org,2002:js`), not `!js` — keep code, comments, and docs consistent. Files end with exactly one trailing newline; `git diff --check` (a pre-push gate) rejects new blank lines at EOF. ## Type Safety and Documentation -This codebase aims to be **very type-safe and well documented** for -maintainability. Code that fails to compile under `strict: true` (with -`noImplicitAny` enabled for all `packages/*` source) is not acceptable. Every -`any` that remains must have a specific justification (a comment explaining why -a narrower type is infeasible). +This codebase aims to be **very type-safe and well documented** for maintainability. Code that fails to compile under `strict: true` (with `noImplicitAny` enabled for all `packages/*` source) is not acceptable. Every `any` that remains must have a specific justification (a comment explaining why a narrower type is infeasible). -In the **core** packages (`packages/llm`, `packages/tools`, `packages/agent`, -`packages/agent-loop`, `packages/session`, `packages/system-prompt`), **type -gymnastics are acceptable when they improve the DX of plugin authors** for -common plugin types. The `defineTool` typed schema DSL in `dsh-tools` is the -canonical example: the `SchemaSpec` to `InferArgs` type-level mapping gives -tool authors zero-cast typed `execute` args, and the cost of the conditional -types stays inside the core package. +In the **core** packages (`packages/llm`, `packages/tools`, `packages/agent`, `packages/agent-loop`, `packages/session`, `packages/system-prompt`), **type gymnastics are acceptable when they improve the DX of plugin authors** for common plugin types. The `defineTool` typed schema DSL in `dsh-tools` is the canonical example: the `SchemaSpec` to `InferArgs` type-level mapping gives tool authors zero-cast typed `execute` args, and the cost of the conditional types stays inside the core package. -Verbose documentation is fine **as long as docs and code stay strictly in -sync**. Out-of-sync docs are worse than no docs. **When you change code, update -its docs in the SAME change** — grep the package README and the module/JSDoc -comments for the old behavior (config keys, defaults, error codes, wire field -names, event names) and fix every hit. CI has no doc-sync gate, so this is on -the author. Every module has a module-level doc comment explaining its role. -Every exported class, interface, type, function, and non-obvious method has a -JSDoc that explains semantics (not just the name) — contracts (what events fire -when), disposal behavior, error behavior, and extension intent. Internal -helpers get docs only where non-obvious. Prefer one-liners when one line -suffices. +Verbose documentation is fine **as long as docs and code stay strictly in sync**. Out-of-sync docs are worse than no docs. **When you change code, update its docs in the SAME change** — grep the package README and the module/JSDoc comments for the old behavior (config keys, defaults, error codes, wire field names, event names) and fix every hit. CI has no doc-sync gate, so this is on the author. Every module has a module-level doc comment explaining its role. Every exported class, interface, type, function, and non-obvious method has a JSDoc that explains semantics (not just the name) — contracts (what events fire when), disposal behavior, error behavior, and extension intent. Internal helpers get docs only where non-obvious. Prefer one-liners when one line suffices. -**Editing these instructions**: `AGENTS.md` is the real file; `CLAUDE.md` is a -symlink to it (at the repo root and in `packages/`). Always edit `AGENTS.md` — -never write through the `CLAUDE.md` symlink or replace it with a regular file. +**Markdown is not hard-wrapped**: write one line per paragraph and let the editor soft-wrap. Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. This applies to prose only: leave fenced code blocks, tables, and list structure intact (a wrapped list item folds to one line per bullet). Code comments / JSDoc are exempt — they stay under the linter's column limit. + +**Editing these instructions**: `AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it (at the repo root and in `packages/`). Always edit `AGENTS.md` — never write through the `CLAUDE.md` symlink or replace it with a regular file. ## Vendoring Policy -`vendor/` packages are pinned source copies (manifest with upstream commit -SHAs in [vendor/README.md](vendor/README.md)). To update one, follow the sync -procedure there; re-apply (or retire) the logged local modifications and rerun -`yarn test && yarn build`. +`vendor/` packages are pinned source copies (manifest with upstream commit SHAs in [vendor/README.md](vendor/README.md)). To update one, follow the sync procedure there; re-apply (or retire) the logged local modifications and rerun `yarn test && yarn build`. diff --git a/docs/adr/0001-vendor-cordis-as-source.md b/docs/adr/0001-vendor-cordis-as-source.md index 28d6cdf83d..460f4a8651 100644 --- a/docs/adr/0001-vendor-cordis-as-source.md +++ b/docs/adr/0001-vendor-cordis-as-source.md @@ -4,31 +4,17 @@ Status: accepted (2026-06-11) ## Context -DeepSeek Code is built on the Cordis framework. Cordis core was at 4.0.0-rc.6 -(a release candidate) when this repo started; the harness depends on framework -internals (fiber lifecycle, effect disposal, waterfall dispatch) whose exact -behavior matters to the agent loop's correctness guarantees. +DeepSeek Code is built on the Cordis framework. Cordis core was at 4.0.0-rc.6 (a release candidate) when this repo started; the harness depends on framework internals (fiber lifecycle, effect disposal, waterfall dispatch) whose exact behavior matters to the agent loop's correctness guarantees. ## Decision -Copy the needed Cordis packages (core, loader, include, group, timer, hmr, -logger-console) and the cordiverse foundation libraries (cosmokit, -schemastery) into `vendor/` as source, flattened, keeping their original npm -names so workspace resolution is transparent. Truly third-party dependencies -(js-yaml, chokidar, @standard-schema/spec, …) stay on npm. +Copy the needed Cordis packages (core, loader, include, group, timer, hmr, logger-console) and the cordiverse foundation libraries (cosmokit, schemastery) into `vendor/` as source, flattened, keeping their original npm names so workspace resolution is transparent. Truly third-party dependencies (js-yaml, chokidar, @standard-schema/spec, …) stay on npm. -`vendor/README.md` is the manifest: upstream repo + commit SHA per package and -an exhaustive local-modification log. A pre-commit guard -(`scripts/check-vendor-manifest.sh`) rejects vendored-source changes that -don't update the manifest in the same commit. +`vendor/README.md` is the manifest: upstream repo + commit SHA per package and an exhaustive local-modification log. A pre-commit guard (`scripts/check-vendor-manifest.sh`) rejects vendored-source changes that don't update the manifest in the same commit. ## Consequences -- The harness fully owns its framework layer: auditable, patchable, pinned — - an RC upstream can't break us, and we can fix framework bugs in-tree. -- Upstream sync is manual (documented procedure in the manifest). The - modification log keeps the diff surface known. -- Vendored packages keep upstream code style; lint/strictness gates exclude - them (their tsconfigs relax our newer compiler flags locally). -- One local patch exists from day one: hmr's locale-YAML imports removed (the - runtime YAML import hook isn't vendored). +- The harness fully owns its framework layer: auditable, patchable, pinned — an RC upstream can't break us, and we can fix framework bugs in-tree. +- Upstream sync is manual (documented procedure in the manifest). The modification log keeps the diff surface known. +- Vendored packages keep upstream code style; lint/strictness gates exclude them (their tsconfigs relax our newer compiler flags locally). +- One local patch exists from day one: hmr's locale-YAML imports removed (the runtime YAML import hook isn't vendored). diff --git a/docs/adr/0002-microkernel-event-taxonomy.md b/docs/adr/0002-microkernel-event-taxonomy.md index 0ddf1f4b6c..900fed23d1 100644 --- a/docs/adr/0002-microkernel-event-taxonomy.md +++ b/docs/adr/0002-microkernel-event-taxonomy.md @@ -4,35 +4,21 @@ Status: accepted (2026-06-11) ## Context -The product principle (see the 微内核Harness实现思路 design doc) is -"everything is a plugin": hooks, /goal, /loop, dynamic workflows, compaction, -sandboxing, permissions, UI, persistence, MCP, skills must all be writable as -plugins without modifying the core. Candidate mechanisms considered: a -purpose-built middleware stack (koa-compose style), an explicit phase state -machine plugins can insert into, or Cordis's native event system. +The product principle (see the 微内核Harness实现思路 design doc) is "everything is a plugin": hooks, /goal, /loop, dynamic workflows, compaction, sandboxing, permissions, UI, persistence, MCP, skills must all be writable as plugins without modifying the core. Candidate mechanisms considered: a purpose-built middleware stack (koa-compose style), an explicit phase state machine plugins can insert into, or Cordis's native event system. ## Decision -Pure Cordis event taxonomy. The loop's extension seams are typed events with -deliberate dispatch modes: +Pure Cordis event taxonomy. The loop's extension seams are typed events with deliberate dispatch modes: -- **waterfall** (around-middleware) where plugins mutate or veto: - `agent/request`, `agent/step-result`, `agent/turn-continuation`, - `tools/execute`, `llm/stream`, `llm/generate`, `system-prompt/assemble`. -- **emit** (sync fire-and-forget) for notifications: turn/step boundaries, - stream chunks, lifecycle, errors. +- **waterfall** (around-middleware) where plugins mutate or veto: `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/execute`, `llm/stream`, `llm/generate`, `system-prompt/assemble`. +- **emit** (sync fire-and-forget) for notifications: turn/step boundaries, stream chunks, lifecycle, errors. - **parallel** (awaited) for the one durability checkpoint: `session/flush`. -The event vocabulary lives in interface packages (dsh-agent declares the -agent/* events); `@deepseek-ai/dsh-agent-loop` is the only concrete plugin and -is itself swappable — nothing outside it may depend on it. +The event vocabulary lives in interface packages (dsh-agent declares the agent/* events); `@deepseek-ai/dsh-agent-loop` is the only concrete plugin and is itself swappable — nothing outside it may depend on it. ## Consequences -- Every MVP feature maps to a listener (the "plugin sanity checklist" in - docs/architecture.md is the proof obligation, kept current). +- Every MVP feature maps to a listener (the "plugin sanity checklist" in docs/architecture.md is the proof obligation, kept current). - HMR and disposal come free: listeners and registrations are Cordis effects. -- Waterfall semantics (call `next()` or short-circuit) are non-obvious and - must be taught — documented in AGENTS.md and covered by composition tests. -- The loop must be defensive: plugin exceptions are contained at turn level, - steering from any seam is never stranded (regression-tested). +- Waterfall semantics (call `next()` or short-circuit) are non-obvious and must be taught — documented in AGENTS.md and covered by composition tests. +- The loop must be defensive: plugin exceptions are contained at turn level, steering from any seam is never stranded (regression-tested). diff --git a/docs/adr/0003-event-sourced-sessions.md b/docs/adr/0003-event-sourced-sessions.md index e8046dc034..33de7b27bc 100644 --- a/docs/adr/0003-event-sourced-sessions.md +++ b/docs/adr/0003-event-sourced-sessions.md @@ -4,35 +4,19 @@ Status: accepted (2026-06-11) ## Context -The MVP requires strict event-based tracing with fully replayable sessions -(严格的基于事件的trace、logging系统，session完全可回放). Two models were -considered: a mutable message array with events fired as notifications -(simpler, but state and log can diverge), or event-sourcing where the log IS -the state. +The MVP requires strict event-based tracing with fully replayable sessions (严格的基于事件的trace、logging系统，session完全可回放). Two models were considered: a mutable message array with events fired as notifications (simpler, but state and log can diverge), or event-sourcing where the log IS the state. ## Decision -A `Session` is an append-only log of typed `SessionEvent`s — the single -source of truth. The LLM message history is *derived* from the log -(`deriveMessages()`); raw stream chunks are logged for token-level replay -fidelity while the assembled `assistant/message` event is authoritative for -derivation. Replay/fork = seed a new session with an existing log. +A `Session` is an append-only log of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`); raw stream chunks are logged for token-level replay fidelity while the assembled `assistant/message` event is authoritative for derivation. Replay/fork = seed a new session with an existing log. -Appends are synchronous (the hot path never blocks on I/O); `session/event` -is a sync notification; persistence plugins buffer write-behind and drain at -the awaited `session/flush` checkpoint fired at every turn end. +Appends are synchronous (the hot path never blocks on I/O); `session/event` is a sync notification; persistence plugins buffer write-behind and drain at the awaited `session/flush` checkpoint fired at every turn end. -Ordering contract: the loop appends to the session *before* emitting the -corresponding Cordis event, and the `agent/step-result` waterfall runs before -the `assistant/message` append so the log records what tool dispatch actually -used (post-review fix; regression-tested). +Ordering contract: the loop appends to the session *before* emitting the corresponding Cordis event, and the `agent/step-result` waterfall runs before the `assistant/message` append so the log records what tool dispatch actually used (post-review fix; regression-tested). ## Consequences - Replay, trace, and telemetry are structurally guaranteed, not bolted on. - Persistence stays a plugin concern; the in-memory store ships in dsh-session. -- The event vocabulary is merge-extensible (plugins add e.g. compaction - events); it carries a TODO(review) marker until the first persistence - plugin and real adapter exercise it. -- Derivation cost grows with log length — compaction (future plugin) is the - intended mitigation, not log mutation. +- The event vocabulary is merge-extensible (plugins add e.g. compaction events); it carries a TODO(review) marker until the first persistence plugin and real adapter exercise it. +- Derivation cost grows with log length — compaction (future plugin) is the intended mitigation, not log mutation. diff --git a/docs/adr/0004-own-content-block-vocabulary.md b/docs/adr/0004-own-content-block-vocabulary.md index f566081e7e..7ffe7ac14e 100644 --- a/docs/adr/0004-own-content-block-vocabulary.md +++ b/docs/adr/0004-own-content-block-vocabulary.md @@ -4,34 +4,16 @@ Status: accepted (2026-06-11) ## Context -The harness needs one internal language for messages that the loop, session -log, and all plugins speak. Options: mirror the DeepSeek/OpenAI -chat-completions shape (zero mapping for the first provider, awkward for rich -content), adopt Anthropic's Messages block structure verbatim (battle-tested, -but our canonical types would mirror a third-party API we don't target -first), or own a vocabulary. +The harness needs one internal language for messages that the loop, session log, and all plugins speak. Options: mirror the DeepSeek/OpenAI chat-completions shape (zero mapping for the first provider, awkward for rich content), adopt Anthropic's Messages block structure verbatim (battle-tested, but our canonical types would mirror a third-party API we don't target first), or own a vocabulary. ## Decision -Own it: messages are arrays of typed content blocks (`text`, `reasoning`, -`tool-call`, `tool-result`, `image`), with the union derived from the -merge-extensible `ContentBlockMap` so plugins add block types via declaration -merging. The same merge-extensible-map pattern types every "stringly" field -(`MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`). Streaming -is a raw chunk protocol; `BlockAssembler` is the single shared assembly -implementation. Adapters translate to provider wire formats — mapping cost -lives in adapters, where it belongs. +Own it: messages are arrays of typed content blocks (`text`, `reasoning`, `tool-call`, `tool-result`, `image`), with the union derived from the merge-extensible `ContentBlockMap` so plugins add block types via declaration merging. The same merge-extensible-map pattern types every "stringly" field (`MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`). Streaming is a raw chunk protocol; `BlockAssembler` is the single shared assembly implementation. Adapters translate to provider wire formats — mapping cost lives in adapters, where it belongs. -In-session context injection (`context/message`, `steering/message`) renders -as tagged user-role envelopes (the system-reminder pattern) rather than a new -role, so adapters carry zero burden. TODO(review): revisit once the DeepSeek -V4 adapter exists. +In-session context injection (`context/message`, `steering/message`) renders as tagged user-role envelopes (the system-reminder pattern) rather than a new role, so adapters carry zero burden. TODO(review): revisit once the DeepSeek V4 adapter exists. ## Consequences -- Reasoning, prefill, cache hints, and multimodal content all have a home - without provider contortions. -- Every adapter pays a translation cost; the streaming protocol carries a - TODO(review) marker until the first real adapter validates it. -- IDs that cross package boundaries are branded (`CallId`, `SessionId`, - `AgentId`) — nominal typing at zero runtime cost. +- Reasoning, prefill, cache hints, and multimodal content all have a home without provider contortions. +- Every adapter pays a translation cost; the streaming protocol carries a TODO(review) marker until the first real adapter validates it. +- IDs that cross package boundaries are branded (`CallId`, `SessionId`, `AgentId`) — nominal typing at zero runtime cost. diff --git a/docs/adr/0005-custom-schema-dsl-over-schemastery.md b/docs/adr/0005-custom-schema-dsl-over-schemastery.md index 9d7011a6db..e9380099aa 100644 --- a/docs/adr/0005-custom-schema-dsl-over-schemastery.md +++ b/docs/adr/0005-custom-schema-dsl-over-schemastery.md @@ -4,32 +4,16 @@ Status: accepted (2026-06-11) ## Context -Tool parameters must reach the model as standard JSON Schema (the wire -format), and tool authors deserve typed `execute(args)` without casts. The -repo already vendors schemastery (used for plugin Config), so reusing it was -the obvious candidate. The user also explicitly preferred per-property -`required: true` booleans over JSON Schema's separate `required` array. +Tool parameters must reach the model as standard JSON Schema (the wire format), and tool authors deserve typed `execute(args)` without casts. The repo already vendors schemastery (used for plugin Config), so reusing it was the obvious candidate. The user also explicitly preferred per-property `required: true` booleans over JSON Schema's separate `required` array. ## Decision -A small custom DSL in dsh-tools: `SchemaSpec` (per-property specs with -`required: true` booleans), type-level `InferArgs` mapping a spec to the -argument type (required keys non-optional, others genuinely optional via `?`), -a runtime `schemaSpecToJsonSchema()` converter, and `defineTool()` tying them -together. Raw JSON-Schema `ToolDefinition`s remain accepted by -`ToolRegistry.register()` — that's how MCP-sourced tools arrive. +A small custom DSL in dsh-tools: `SchemaSpec` (per-property specs with `required: true` booleans), type-level `InferArgs` mapping a spec to the argument type (required keys non-optional, others genuinely optional via `?`), a runtime `schemaSpecToJsonSchema()` converter, and `defineTool()` tying them together. Raw JSON-Schema `ToolDefinition`s remain accepted by `ToolRegistry.register()` — that's how MCP-sourced tools arrive. -Schemastery was evaluated and rejected for this use: it targets validation / -transformation against StandardSchema, not JSON Schema *generation*, so it -would add indirection without producing the wire format cleanly. +Schemastery was evaluated and rejected for this use: it targets validation / transformation against StandardSchema, not JSON Schema *generation*, so it would add indirection without producing the wire format cleanly. ## Consequences -- First-party tool authors get zero-cast typed args; the type gymnastics cost - stays inside the core package (sanctioned by the AGENTS.md type-safety - policy). -- The DSL is deliberately small (string/number/boolean/object/array, enum, - default, nested properties/items). Gaps vs full JSON Schema (unions, - formats, constraints) are accepted until real tools demand them. -- The InferArgs mapping is regression-tested at the type level (expectTypeOf) - after an early optionality bug shipped and was caught by review. +- First-party tool authors get zero-cast typed args; the type gymnastics cost stays inside the core package (sanctioned by the AGENTS.md type-safety policy). +- The DSL is deliberately small (string/number/boolean/object/array, enum, default, nested properties/items). Gaps vs full JSON Schema (unions, formats, constraints) are accepted until real tools demand them. +- The InferArgs mapping is regression-tested at the type level (expectTypeOf) after an early optionality bug shipped and was caught by review. diff --git a/docs/adr/0006-tool-schemas-in-prompt-assembly.md b/docs/adr/0006-tool-schemas-in-prompt-assembly.md index b6158a66ec..72be5bf63d 100644 --- a/docs/adr/0006-tool-schemas-in-prompt-assembly.md +++ b/docs/adr/0006-tool-schemas-in-prompt-assembly.md @@ -4,28 +4,14 @@ Status: accepted (2026-06-11) ## Context -On the wire, tool schemas travel in a dedicated `tools` field of the model -request, not in prompt text. Architecturally, though, "what the model is told -it can do" is one coherent concern: prompt sections and the tool list are -assembled from the same plugin contributions and consumed at the same moment. -The alternative — the loop querying the tool registry separately from the -prompt service — splits one concern across two seams. +On the wire, tool schemas travel in a dedicated `tools` field of the model request, not in prompt text. Architecturally, though, "what the model is told it can do" is one coherent concern: prompt sections and the tool list are assembled from the same plugin contributions and consumed at the same moment. The alternative — the loop querying the tool registry separately from the prompt service — splits one concern across two seams. ## Decision -`PromptAssembly { sections, tools }`: the system-prompt service collects -ordered text sections AND tool schemas (the tool registry auto-contributes a -provider). The loop consumes one assembly per step; adapters map `sections` -to the provider's system slot and `tools` to the wire `tools` field. The -`system-prompt/assemble` waterfall is therefore a single interception point -for everything the model is told up front — tool filtering (ToolSearch / -progressive disclosure) is an assembly rewrite, same as prompt edits. +`PromptAssembly { sections, tools }`: the system-prompt service collects ordered text sections AND tool schemas (the tool registry auto-contributes a provider). The loop consumes one assembly per step; adapters map `sections` to the provider's system slot and `tools` to the wire `tools` field. The `system-prompt/assemble` waterfall is therefore a single interception point for everything the model is told up front — tool filtering (ToolSearch / progressive disclosure) is an assembly rewrite, same as prompt edits. ## Consequences -- One waterfall governs the model's standing context; plugins like plan mode - can swap prompt text and visible tools in one listener. -- The assembly interface is merge-extensible for future slots (no untyped - `extras` bag — extension is declaration merging). -- Slight conceptual surprise (schemas in a "prompt" service) is documented - here and in the package README. +- One waterfall governs the model's standing context; plugins like plan mode can swap prompt text and visible tools in one listener. +- The assembly interface is merge-extensible for future slots (no untyped `extras` bag — extension is declaration merging). +- Slight conceptual surprise (schemas in a "prompt" service) is documented here and in the package README. diff --git a/docs/adr/0007-quality-gates.md b/docs/adr/0007-quality-gates.md index 357b34177c..7bcba2a912 100644 --- a/docs/adr/0007-quality-gates.md +++ b/docs/adr/0007-quality-gates.md @@ -4,33 +4,20 @@ Status: accepted (2026-06-11) ## Context -This codebase is developed primarily by coding agents. Agents follow enforced -gates far more reliably than prose conventions, and "a lot of work" is not a -cost argument when agents do the labor. Early evidence: tests that didn't -typecheck shipped (vitest doesn't typecheck) and were only caught by a review. +This codebase is developed primarily by coding agents. Agents follow enforced gates far more reliably than prose conventions, and "a lot of work" is not a cost argument when agents do the labor. Early evidence: tests that didn't typecheck shipped (vitest doesn't typecheck) and were only caught by a review. ## Decision -Every AGENTS.md promise gets a command that exits non-zero, wired into git -hooks and CI both calling the same package.json scripts: +Every AGENTS.md promise gets a command that exits non-zero, wired into git hooks and CI both calling the same package.json scripts: -- Max-strict TypeScript (`noUncheckedIndexedAccess`, - `exactOptionalPropertyTypes`, …); tests and examples typecheck in CI via - `tsconfig.typecheck.json` (vendored packages resolve as built declarations). -- ESLint strict-type-checked + @stylistic (the house style, enforced); - vendored code excluded. -- Per-file 100% coverage on `packages/*/src` (v8); unreachable defensive - guards carry `/* v8 ignore */ ` with stated reasons instead of deletion. -- knip (dead code/deps), publint (package correctness), yarn constraints - (workspace rules: private, cordis peer+dev, uniform version, ESM). -- lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and - pre-push (tests, hygiene); CI runs the full matrix on node 24/26 plus a - demo smoke test driving the echo-agent end to end. +- Max-strict TypeScript (`noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, …); tests and examples typecheck in CI via `tsconfig.typecheck.json` (vendored packages resolve as built declarations). +- ESLint strict-type-checked + @stylistic (the house style, enforced); vendored code excluded. +- Per-file 100% coverage on `packages/*/src` (v8); unreachable defensive guards carry `/* v8 ignore */ ` with stated reasons instead of deletion. +- knip (dead code/deps), publint (package correctness), yarn constraints (workspace rules: private, cordis peer+dev, uniform version, ESM). +- lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and pre-push (tests, hygiene); CI runs the full matrix on node 24/26 plus a demo smoke test driving the echo-agent end to end. ## Consequences - Conventions survive agent turnover; violations fail fast and locally. -- The gates themselves are code to maintain; config changes are reviewed like - any change. -- 100%-coverage pressure can produce assertion-free tests — mutation testing - is the planned counterweight (see RFC 002). +- The gates themselves are code to maintain; config changes are reviewed like any change. +- 100%-coverage pressure can produce assertion-free tests — mutation testing is the planned counterweight (see RFC 002). diff --git a/docs/adr/0008-tsdown-over-dumble.md b/docs/adr/0008-tsdown-over-dumble.md index b982d35fa8..f6026311ac 100644 --- a/docs/adr/0008-tsdown-over-dumble.md +++ b/docs/adr/0008-tsdown-over-dumble.md @@ -4,49 +4,21 @@ Status: accepted (2026-06-11) ## Context -The initial build used **dumble**, the cordiverse zero-config esbuild wrapper -that upstream Cordis itself builds with — maximum alignment with the vendored -packages' conventions (it reads each package.json and infers entries/formats -from the `exports` field). But dumble is a liability as a load-bearing tool in -this repo: v0.2.x, ~530 npm downloads/week, effectively one maintainer, and we -were invoking it through a custom orchestration script (`scripts/build.ts`) -because it has no workspace mode. +The initial build used **dumble**, the cordiverse zero-config esbuild wrapper that upstream Cordis itself builds with — maximum alignment with the vendored packages' conventions (it reads each package.json and infers entries/formats from the `exports` field). But dumble is a liability as a load-bearing tool in this repo: v0.2.x, ~530 npm downloads/week, effectively one maintainer, and we were invoking it through a custom orchestration script (`scripts/build.ts`) because it has no workspace mode. -Build output currently matters only for `yarn build` + publint (nothing -publishes yet; dev/test/demo run unbuilt via tsx), so the switching cost is at -its lowest now and only grows once packages publish. +Build output currently matters only for `yarn build` + publint (nothing publishes yet; dev/test/demo run unbuilt via tsx), so the switching cost is at its lowest now and only grows once packages publish. ## Decision -Replace dumble with **tsdown** (rolldown-based, ~2.5M downloads/week, -VoidZero-backed, actively released): +Replace dumble with **tsdown** (rolldown-based, ~2.5M downloads/week, VoidZero-backed, actively released): -- Root `tsdown.config.ts` with `workspace: ['vendor/*', 'packages/*']` - (explicit globs, not `workspace: true`, which would also pick up - `examples/*` — they have package.json files but are not yarn workspaces). -- Shared shape: entry `src/index.ts`, `outDir: 'lib'`, ESM, `platform: node`, - `target: es2024`, `fixedExtension: false` (keeps `.js` for - `"type": "module"` packages), `dts: false` (tsc -b owns declarations), - `clean: false` (lib/ holds tsc's .d.ts output). -- Two per-package overrides in vendor/ (ours, like the regenerated tsconfigs; - logged in vendor/README.md): schemastery (dual `.mjs`/`.cjs` via - `outExtensions`), logger-console (two single-entry passes so the shared - base class is inlined into each entry instead of a hash-named chunk, - matching upstream's published shape). +- Root `tsdown.config.ts` with `workspace: ['vendor/*', 'packages/*']` (explicit globs, not `workspace: true`, which would also pick up `examples/*` — they have package.json files but are not yarn workspaces). +- Shared shape: entry `src/index.ts`, `outDir: 'lib'`, ESM, `platform: node`, `target: es2024`, `fixedExtension: false` (keeps `.js` for `"type": "module"` packages), `dts: false` (tsc -b owns declarations), `clean: false` (lib/ holds tsc's .d.ts output). +- Two per-package overrides in vendor/ (ours, like the regenerated tsconfigs; logged in vendor/README.md): schemastery (dual `.mjs`/`.cjs` via `outExtensions`), logger-console (two single-entry passes so the shared base class is inlined into each entry instead of a hash-named chunk, matching upstream's published shape). - `scripts/build.ts` deleted; `yarn build` = `tsc -b && tsdown`. -Alternatives considered: **direct esbuild script** (most established engine, -zero wrapper risk, but hand-maintains the per-package spec table tsdown's -workspace mode gives us); **pkgroll** (closest drop-in philosophically, but -78k dl/wk and Rollup-based — strictly weaker maintenance story than tsdown); -**keep dumble** (perfect upstream alignment, unacceptable bus factor). +Alternatives considered: **direct esbuild script** (most established engine, zero wrapper risk, but hand-maintains the per-package spec table tsdown's workspace mode gives us); **pkgroll** (closest drop-in philosophically, but 78k dl/wk and Rollup-based — strictly weaker maintenance story than tsdown); **keep dumble** (perfect upstream alignment, unacceptable bus factor). ## Consequences -Output file lists are byte-for-byte-list identical to dumble's (verified by -snapshot diff at migration time); externals still come from each package's -dependencies/peerDependencies. We give up dumble's exports-field inference — -new packages with non-default shapes need a per-package `tsdown.config.ts` -instead of just package.json fields. Future option: tsdown could also absorb -declaration bundling (isolatedDeclarations) if `tsc -b` ever becomes the -bottleneck; that would be a new ADR. +Output file lists are byte-for-byte-list identical to dumble's (verified by snapshot diff at migration time); externals still come from each package's dependencies/peerDependencies. We give up dumble's exports-field inference — new packages with non-default shapes need a per-package `tsdown.config.ts` instead of just package.json fields. Future option: tsdown could also absorb declaration bundling (isolatedDeclarations) if `tsc -b` ever becomes the bottleneck; that would be a new ADR. diff --git a/docs/adr/README.md b/docs/adr/README.md index 5c257b9fac..033b973689 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -1,12 +1,8 @@ # Architecture Decision Records -Short, immutable records of the *why* behind decisions that shape this -codebase. Code and docs say what the system does; ADRs say why it does it -that way and what we gave up. +Short, immutable records of the *why* behind decisions that shape this codebase. Code and docs say what the system does; ADRs say why it does it that way and what we gave up. -Format: one file per decision, numbered, with Status / Context / Decision / -Consequences. An ADR is never edited into a different decision — supersede it -with a new one and cross-link. +Format: one file per decision, numbered, with Status / Context / Decision / Consequences. An ADR is never edited into a different decision — supersede it with a new one and cross-link. | # | Title | Status | |---|---|---| diff --git a/docs/architecture.md b/docs/architecture.md index 52820e4b87..8c7f136923 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,16 +1,10 @@ # DeepSeek Harness Architecture -This document describes the phase-1 architecture of the DeepSeek Harness — the -foundation of **DeepSeek Code**. The governing principle, from the -[microkernel design discussion][microkernel-doc], is: +This document describes the phase-1 architecture of the DeepSeek Harness — the foundation of **DeepSeek Code**. The governing principle, from the [microkernel design discussion][microkernel-doc], is: > **Microkernel approach. Everything is a plugin.** -The harness core is deliberately tiny: a handful of abstract services plus one -concrete plugin (the agent loop). Every product feature — tools, hooks, -compaction, sandboxing, UI, persistence, sub-agents, MCP, skills — is meant to -be written as a plugin against the extension surface described here, without -modifying the loop. +The harness core is deliberately tiny: a handful of abstract services plus one concrete plugin (the agent loop). Every product feature — tools, hooks, compaction, sandboxing, UI, persistence, sub-agents, MCP, skills — is meant to be written as a plugin against the extension surface described here, without modifying the loop. Requirement context: [Coding Harness MVP 需求分析][mvp-doc]. @@ -39,9 +33,7 @@ Requirement context: [Coding Harness MVP 需求分析][mvp-doc]. └─────────────────────────────────────────────────────────────┘ ``` -Dependency rule: plugins depend on interface packages, never on -`dsh-agent-loop`. The loop itself is swappable — UI/hook/tool plugins keep -working against the `dsh-agent` vocabulary if the loop is replaced. +Dependency rule: plugins depend on interface packages, never on `dsh-agent-loop`. The loop itself is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. ## Service map @@ -55,34 +47,17 @@ working against the `dsh-agent` vocabulary if the loop is replaced. | `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `LoopAgent`s and drives their loops | | `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks | -All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go -through `ctx.effect()` and return disposers, so plugin hot-reload (vendored -HMR) and fiber disposal clean up automatically. +All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically. ## Capability seams: interface / implementation / consumer -Swappable capabilities are split into **three packages** so each part evolves -independently. The bash capability is the template: +Swappable capabilities are split into **three packages** so each part evolves independently. The bash capability is the template: -1. **Interface** (`dsh-bash`) — an abstract service plus the vocabulary types - (`BashExecutor`, `BashRunResult`, `BashTask`, …). Defines the contract, - owns the `ctx.bash` key, depends only on cordis. -2. **Implementation** (`dsh-bash-local`) — a concrete subclass loaded as a - plugin (local subprocesses, process-group kills, spill-file truncation). - Sandboxed, containerized, or remote backends are sibling packages - implementing the same interface. -3. **Consumer** (`dsh-tool-bash`) — what the model and other plugins program - against (the `bash`/`bash_output`/`bash_kill` tool schemas). Consumers - `inject` the interface's ctx key and never import implementation types. +1. **Interface** (`dsh-bash`) — an abstract service plus the vocabulary types (`BashExecutor`, `BashRunResult`, `BashTask`, …). Defines the contract, owns the `ctx.bash` key, depends only on cordis. +2. **Implementation** (`dsh-bash-local`) — a concrete subclass loaded as a plugin (local subprocesses, process-group kills, spill-file truncation). Sandboxed, containerized, or remote backends are sibling packages implementing the same interface. +3. **Consumer** (`dsh-tool-bash`) — what the model and other plugins program against (the `bash`/`bash_output`/`bash_kill` tool schemas). Consumers `inject` the interface's ctx key and never import implementation types. -The LLM seam has the same topology folded differently: `dsh-llm` carries the -interface (`LlmAdapter`) AND the consumer surface (`ctx.llm.stream()`), with -adapters as implementation packages — there the consumer is the loop itself, -not a swappable schema surface. Use the full three-package split when the -consumer is independently replaceable; keep interface + consumer together -when they are one concern. Don't split preemptively: a capability with one -conceivable implementation and one consumer stays one package until proven -otherwise. +The LLM seam has the same topology folded differently: `dsh-llm` carries the interface (`LlmAdapter`) AND the consumer surface (`ctx.llm.stream()`), with adapters as implementation packages — there the consumer is the loop itself, not a swappable schema surface. Use the full three-package split when the consumer is independently replaceable; keep interface + consumer together when they are one concern. Don't split preemptively: a capability with one conceivable implementation and one consumer stays one package until proven otherwise. > **"Capability" — two unrelated meanings.** (1) The *seam pattern* above > ("one plugin provides a capability, another needs it") is realized by @@ -98,103 +73,55 @@ otherwise. ## The vocabulary (dsh-llm) -Messages are arrays of typed **content blocks** (`text`, `reasoning`, -`tool-call`, `tool-result`, `image`); the union is derived from the -merge-extensible `ContentBlockMap`, so plugins can add block types via -declaration merging. The same merge-extensible-map pattern is used for -`MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason` — typed -sum types instead of strings. +Messages are arrays of typed **content blocks** (`text`, `reasoning`, `tool-call`, `tool-result`, `image`); the union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. The same merge-extensible-map pattern is used for `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason` — typed sum types instead of strings. -Streaming is a raw chunk protocol (`block-start`, `text-delta`, -`reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). -`BlockAssembler` is the single shared implementation that assembles chunks -into blocks/messages; the loop logs raw chunks (replay fidelity) while feeding -the same chunks through an assembler. +Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages; the loop logs raw chunks (replay fidelity) while feeding the same chunks through an assembler. -`LlmAdapter` is the provider seam: subclass, implement `stream()`, call -`ctx.llm.registerAdapter(models, adapter)`. Two real adapters implement it — -`dsh-llm-deepseek` (hand-rolled fetch/SSE against the DeepSeek API) and -`dsh-llm-pi-ai` (the same endpoint through the `@earendil-works/pi-ai` -library). They exist as a pair deliberately: two independent internals over -one contract verified the StreamChunk protocol, which is now documented (in -`dsh-llm/src/types.ts`) with the conventions that review pinned down — usage -before finish, nothing after finish, raw-string tool arguments, and the two -sanctioned error paths (thrown vs `finish {kind:'error'}`). +`LlmAdapter` is the provider seam: subclass, implement `stream()`, call `ctx.llm.registerAdapter(models, adapter)`. Two real adapters implement it — `dsh-llm-deepseek` (hand-rolled fetch/SSE against the DeepSeek API) and `dsh-llm-pi-ai` (the same endpoint through the `@earendil-works/pi-ai` library). They exist as a pair deliberately: two independent internals over one contract verified the StreamChunk protocol, which is now documented (in `dsh-llm/src/types.ts`) with the conventions that review pinned down — usage before finish, nothing after finish, raw-string tool arguments, and the two sanctioned error paths (thrown vs `finish {kind:'error'}`). ## Event-sourced sessions (dsh-session) -A `Session` is an append-only log of typed `SessionEvent`s — the single source -of truth. The LLM message history is *derived* from the log -(`deriveMessages()`): +A `Session` is an append-only log of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`): - `user/message` → user message -- `assistant/message` → assistant message (raw `assistant/chunk` events are - replay/UI data and are skipped in derivation) +- `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation) - `tool/result` → user message carrying a `tool-result` block -- `context/message`, `steering/message` → user-role messages wrapped in a - tagged envelope (`…`) at their chronological - position — the "system-reminder" pattern; models distinguish them from real - user prompts by the envelope. **TODO(review)**: revisit the envelope once a - real adapter exists. +- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: revisit the envelope once a real adapter exists. -Replay/fork = `ctx.sessions.create(id, seedEvents)`. Trace/telemetry = listen -to `session/event`. +Replay/fork = `ctx.sessions.create(id, seedEvents)`. Trace/telemetry = listen to `session/event`. -**Durability seam**: `session/event` is a synchronous notification; -persistence plugins buffer (write-behind) and drain at the awaited -`session/flush` checkpoint the loop fires at every turn end (see -`examples/echo-agent/src/session-jsonl.ts` for the pattern). -**TODO**: real persistence backends (JSONL per session dir, sqlite) are a -future phase. +**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end (see `examples/echo-agent/src/session-jsonl.ts` for the pattern). **TODO**: real persistence backends (JSONL per session dir, sqlite) are a future phase. ## Prompt assembly (dsh-system-prompt) -Plugins contribute `PromptSection`s (named, ordered, static or computed) and -tool-schema providers. `assemble()` returns a `PromptAssembly { sections, -tools }` through the `system-prompt/assemble` waterfall. +Plugins contribute `PromptSection`s (named, ordered, static or computed) and tool-schema providers. `assemble()` returns a `PromptAssembly { sections, tools }` through the `system-prompt/assemble` waterfall. -Tool schemas are deliberately **part of the assembly**: "what the model is -told it can do" is one coherent thing managed here, even though adapters -transmit schemas as the wire-level `tools` field rather than prompt text. +Tool schemas are deliberately **part of the assembly**: "what the model is told it can do" is one coherent thing managed here, even though adapters transmit schemas as the wire-level `tools` field rather than prompt text. ## Tool pipeline (dsh-tools) -`ToolRegistry.register()` takes schema + `execute()`. The registry feeds its -schemas into the system-prompt assembly automatically. +`ToolRegistry.register()` takes schema + `execute()`. The registry feeds its schemas into the system-prompt assembly automatically. -`execute()` runs through the **`tools/execute` waterfall** — the single seam -where sandbox, permission, hooks, and plan-mode plugins wrap or veto a call. -This collapses Claude Code's validate → PreToolUse → permission → execute → -PostToolUse pipeline into ordered waterfall listeners. +`execute()` runs through the **`tools/execute` waterfall** — the single seam where sandbox, permission, hooks, and plan-mode plugins wrap or veto a call. This collapses Claude Code's validate → PreToolUse → permission → execute → PostToolUse pipeline into ordered waterfall listeners. -**TODO**: tool shapes get revisited when real tools land (e.g. a -concurrency-safety hint for parallel execution; phase 1 executes tool calls -sequentially). +**TODO**: tool shapes get revisited when real tools land (e.g. a concurrency-safety hint for parallel execution; phase 1 executes tool calls sequentially). ## Agents (dsh-agent) and the loop (dsh-agent-loop) `Agent` is the handle every plugin programs against: - `send(content)` — queued message; starts a turn when idle, else next turn -- `steer(content)` — mid-turn injection, drained **between steps**; behaves - like `send` when idle -- `inject(content)` — in-session context (`context/message` event) without - triggering a turn; the next request sees it (Claude Code attachment / - system-reminder analog) +- `steer(content)` — mid-turn injection, drained **between steps**; behaves like `send` when idle +- `inject(content)` — in-session context (`context/message` event) without triggering a turn; the next request sees it (Claude Code attachment / system-reminder analog) - `abort(reason)` — aborts the in-flight step via `AbortSignal` - `session`, `status`, `options` -**TODO(sub-agents)**: `spawn`/`fork` land on `AgentLoop.create()` — fork seeds -the child Session with the parent's event log, spawn starts fresh; children -are ordinary `Agent` handles so `steer()` and event subscription work -uniformly. Inter-agent channels beyond these primitives are deliberately -deferred. +**TODO(sub-agents)**: `spawn`/`fork` land on `AgentLoop.create()` — fork seeds the child Session with the parent's event log, spawn starts fresh; children are ordinary `Agent` handles so `steer()` and event subscription work uniformly. Inter-agent channels beyond these primitives are deliberately deferred. ### Loop lifecycle (session / turn / step) - **Session**: the whole event log of one agent. -- **Turn**: triggered by ≥1 queued message; runs steps until the model stops - requesting tools and no plugin requests continuation. +- **Turn**: triggered by ≥1 queued message; runs steps until the model stops requesting tools and no plugin requests continuation. - **Step**: one model request + its tool executions. ``` @@ -231,14 +158,7 @@ forever: emit agent/status(idle) unless more queued ``` -Error containment: a throwing `agent/turn-continuation` listener or a -rejecting `session/flush` ends the **turn** with an `error` event — never the -driver loop. An adapter that ends its stream with a `finish {kind:'error'}` -or `{kind:'aborted'}` chunk (the in-band error path, for adapters that can't -throw mid-stream) is likewise translated into a step error, so the turn ends -`error`/`aborted` instead of logging a normal `completed` assistant message. -`abort()` is honored mid-stream **and** between tool calls; disposal mid-turn -ends the turn with reason `disposed` and emits `agent/status('disposed')`. +Error containment: a throwing `agent/turn-continuation` listener or a rejecting `session/flush` ends the **turn** with an `error` event — never the driver loop. An adapter that ends its stream with a `finish {kind:'error'}` or `{kind:'aborted'}` chunk (the in-band error path, for adapters that can't throw mid-stream) is likewise translated into a step error, so the turn ends `error`/`aborted` instead of logging a normal `completed` assistant message. `abort()` is honored mid-stream **and** between tool calls; disposal mid-turn ends the turn with reason `disposed` and emits `agent/status('disposed')`. ### Event taxonomy @@ -262,24 +182,17 @@ Declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package). ### Cordis waterfall semantics (important) -`ctx.waterfall` is **around-middleware**, not a value reducer. Each listener -receives `(...args, next)`: +`ctx.waterfall` is **around-middleware**, not a value reducer. Each listener receives `(...args, next)`: -- call `next()` to delegate to later listeners (and ultimately the core - behavior), possibly wrapping it; +- call `next()` to delegate to later listeners (and ultimately the core behavior), possibly wrapping it; - return a value **without** calling `next()` to short-circuit (veto); - listeners run in registration order; `prepend: true` jumps the queue. -Composition caveat: values propagate through `next()`'s **return value**. -Mutating the passed-in object works when later listeners receive the same -reference, but a listener that returns a *new* object makes earlier mutations -invisible downstream. Prefer mutate-then-`next()` for cooperative middleware; -return a replacement only when you mean to take over the result. +Composition caveat: values propagate through `next()`'s **return value**. Mutating the passed-in object works when later listeners receive the same reference, but a listener that returns a *new* object makes earlier mutations invisible downstream. Prefer mutate-then-`next()` for cooperative middleware; return a replacement only when you mean to take over the result. ## Plugin sanity checklist -Every MVP feature (including the TODO-marked ones), with the mechanism that -implements it **without modifying the loop**: +Every MVP feature (including the TODO-marked ones), with the mechanism that implements it **without modifying the loop**: | MVP feature | Plugin mechanism | |---|---| @@ -334,9 +247,7 @@ export function apply(ctx: Context) { } ``` -(Raw JSON-Schema `ToolDefinition`s are still accepted by -`ctx.tools.register()` directly — that's how MCP-sourced tools arrive. -`defineTool` is the typed sugar for first-party tools.) +(Raw JSON-Schema `ToolDefinition`s are still accepted by `ctx.tools.register()` directly — that's how MCP-sourced tools arrive. `defineTool` is the typed sugar for first-party tools.) ### A hook plugin (permission gate) @@ -371,31 +282,18 @@ export function apply(ctx: Context) { } ``` -Two complete runnable wirings exist: [`examples/echo-agent`](../examples/echo-agent) -(mock model + echo tool — the all-mock skeleton check) and -[`examples/coding-agent`](../examples/coding-agent) (DeepSeek V4 + the bash -tool suite — the real thing; `yarn demo:coding`). Both load from `cordis.yml` -with HMR. +Two complete runnable wirings exist: [`examples/echo-agent`](../examples/echo-agent) (mock model + echo tool — the all-mock skeleton check) and [`examples/coding-agent`](../examples/coding-agent) (DeepSeek V4 + the bash tool suite — the real thing; `yarn demo:coding`). Both load from `cordis.yml` with HMR. -Step-by-step guides live in [`docs/cookbook`](./cookbook): adding a package, -adding a tool, adding an LLM adapter. +Step-by-step guides live in [`docs/cookbook`](./cookbook): adding a package, adding a tool, adding an LLM adapter. ## Deferred work (TODO) Tracked here deliberately — each is designed-for but not implemented: -- **Restructure this document** — it has grown long; split it into focused - sections (or per-area files) so readers can navigate it without scrolling - the whole thing. -- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent - channels beyond `send`/`steer`/events. -- **Persistence backends** (JSONL session dirs, sqlite) on the - `session/event` + `session/flush` seam. -- **Compaction implementation** (auto thresholds, summarization prompts) on - the `agent/request` seam, with its session-event types added by declaration - merging. +- **Restructure this document** — it has grown long; split it into focused sections (or per-area files) so readers can navigate it without scrolling the whole thing. +- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent channels beyond `send`/`steer`/events. +- **Persistence backends** (JSONL session dirs, sqlite) on the `session/event` + `session/flush` seam. +- **Compaction implementation** (auto thresholds, summarization prompts) on the `agent/request` seam, with its session-event types added by declaration merging. - **Parallel tool execution** (concurrency-safety hints on ToolDefinition). -- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based - forking. -- **Session event vocabulary review** once the loop and a persistence plugin - coexist (`TODO(review)` in dsh-session). +- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based forking. +- **Session event vocabulary review** once the loop and a persistence plugin coexist (`TODO(review)` in dsh-session). diff --git a/docs/cookbook/adding-a-package.md b/docs/cookbook/adding-a-package.md index 99d5d330e4..4780ccca7e 100644 --- a/docs/cookbook/adding-a-package.md +++ b/docs/cookbook/adding-a-package.md @@ -1,7 +1,6 @@ # Cookbook: adding a workspace package -The file-by-file checklist for a new `@deepseek-ai/dsh-` package. -(Verified by the bash and adapter packages; if it drifts, fix it here.) +The file-by-file checklist for a new `@deepseek-ai/dsh-` package. (Verified by the bash and adapter packages; if it drifts, fix it here.) ## 1. Create the package @@ -16,11 +15,7 @@ packages// README.md # service API, events, extension points, design notes ``` -package.json invariants (enforced by `yarn constraints` / yarn.config.cjs): -`private: true`, `version: 0.0.1`, `type: module`, `cordis` in BOTH -peerDependencies and devDependencies (same range). Mirror every dsh peer -dependency in devDependencies. `schemastery` goes in `dependencies` (it is a -runtime validator), matching agent-loop. +package.json invariants (enforced by `yarn constraints` / yarn.config.cjs): `private: true`, `version: 0.0.1`, `type: module`, `cordis` in BOTH peerDependencies and devDependencies (same range). Mirror every dsh peer dependency in devDependencies. `schemastery` goes in `dependencies` (it is a runtime validator), matching agent-loop. ## 2. Register it in the root configs @@ -32,14 +27,11 @@ runtime validator), matching agent-loop. | `scripts/publint-all.ts` | add `'packages/'` to the array | | `knip.json` | only if the package has non-`*.spec.ts` entries (e.g. `*.e2e.ts` → add a per-workspace override like `packages/llm-deepseek`) | -Covered automatically by globs — no edits needed: root `package.json` -workspaces, `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`. +Covered automatically by globs — no edits needed: root `package.json` workspaces, `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`. ## 3. Decide the package topology -For a swappable capability, split interface / implementation / consumer into -separate packages (see docs/architecture.md § "Capability seams" — the bash -trio is the template). A single-purpose plugin stays one package. +For a swappable capability, split interface / implementation / consumer into separate packages (see docs/architecture.md § "Capability seams" — the bash trio is the template). A single-purpose plugin stays one package. ## 4. Verify @@ -50,6 +42,4 @@ yarn test:coverage # 100% per-file over src (types.ts exempt) yarn build && yarn knip && yarn publint ``` -Test expectations: every registry/registration needs an HMR-safety test -(register from a child fiber, dispose it, assert cleanup). Excessive tests -are welcome — see AGENTS.md. +Test expectations: every registry/registration needs an HMR-safety test (register from a child fiber, dispose it, assert cleanup). Excessive tests are welcome — see AGENTS.md. diff --git a/docs/cookbook/adding-a-tool.md b/docs/cookbook/adding-a-tool.md index d00556c26b..88bbde7a84 100644 --- a/docs/cookbook/adding-a-tool.md +++ b/docs/cookbook/adding-a-tool.md @@ -1,8 +1,6 @@ # Cookbook: adding a tool -How to give the model a new capability. Reference implementations: -`examples/echo-agent/src/echo-tool.ts` (minimal) and -`packages/tool-bash` (production-grade, three-package seam). +How to give the model a new capability. Reference implementations: `examples/echo-agent/src/echo-tool.ts` (minimal) and `packages/tool-bash` (production-grade, three-package seam). ## The minimal shape @@ -30,33 +28,18 @@ export function apply(ctx: Context) { } ``` -Registration is effect-based: disposing the plugin fiber unregisters the -tool (write the HMR test). Schemas flow into the system-prompt assembly -automatically. +Registration is effect-based: disposing the plugin fiber unregisters the tool (write the HMR test). Schemas flow into the system-prompt assembly automatically. ## Rules of the execute() contract -- **Validate args at runtime.** `defineTool`'s `InferArgs` typing is - compile-time only; at runtime `arguments` is whatever JSON the model - emitted. Check every field; throw a descriptive Error for bad input. -- **Throwing means isError.** The registry catches anything `execute()` - throws and returns `{isError: true}` to the model. Use that for - infrastructure failures (bad input, spawn errors, aborts) — but REPORT - domain failures in the result text instead (e.g. tool-bash returns - `[exit code: 9]` with `isError: false`: the model decides what a failing - command means). +- **Validate args at runtime.** `defineTool`'s `InferArgs` typing is compile-time only; at runtime `arguments` is whatever JSON the model emitted. Check every field; throw a descriptive Error for bad input. +- **Throwing means isError.** The registry catches anything `execute()` throws and returns `{isError: true}` to the model. Use that for infrastructure failures (bad input, spawn errors, aborts) — but REPORT domain failures in the result text instead (e.g. tool-bash returns `[exit code: 9]` with `isError: false`: the model decides what a failing command means). - **Honor `exec.signal`.** Cancel in-flight work when it fires. -- **Use `exec.agent` for async notifications.** `agent.inject(content, - {source: {kind: 'plugin', plugin: ''}})` appends durable context the - NEXT model request sees — it is not a wake-up (an idle agent stays idle). - Guard against disposed agents (try/catch). +- **Use `exec.agent` for async notifications.** `agent.inject(content, {source: {kind: 'plugin', plugin: ''}})` appends durable context the NEXT model request sees — it is not a wake-up (an idle agent stays idle). Guard against disposed agents (try/catch). ## Long-running work -Follow tool-bash's background pattern: a `run_in_background` flag returns a -task id immediately; companion tools poll incrementally and kill; completion -notices arrive via `agent.inject()`. Bound buffers and spill full output to -disk so nothing is silently lost. +Follow tool-bash's background pattern: a `run_in_background` flag returns a task id immediately; companion tools poll incrementally and kill; completion notices arrive via `agent.inject()`. Bound buffers and spill full output to disk so nothing is silently lost. > TODO: each tool reimplements this background pattern by hand today. At some > point we need a generic long-running-tool layer that handles task ids, @@ -64,14 +47,8 @@ disk so nothing is silently lost. ## Permissions / sandboxing -Prefer not to build policy into the tool. The seam is the `tools/execute` waterfall -(veto or wrap — see the permission-gate example in docs/architecture.md), or -a sandboxing implementation behind the tool's executor seam. +Prefer not to build policy into the tool. The seam is the `tools/execute` waterfall (veto or wrap — see the permission-gate example in docs/architecture.md), or a sandboxing implementation behind the tool's executor seam. ## Tests every tool needs -Arg-validation rejections, result shaping for every outcome, the HMR -disposal test, and — for tools with side effects — an integration spec that -drives the tool through the agent loop with a scripted `MockAdapter` -(`packages/agent-loop/tests/mock-adapter.ts`), asserting the `tool/call` / -`tool/result` session events. +Arg-validation rejections, result shaping for every outcome, the HMR disposal test, and — for tools with side effects — an integration spec that drives the tool through the agent loop with a scripted `MockAdapter` (`packages/agent-loop/tests/mock-adapter.ts`), asserting the `tool/call` / `tool/result` session events. diff --git a/docs/cookbook/adding-an-llm-adapter.md b/docs/cookbook/adding-an-llm-adapter.md index 361bca6f28..a110e751b4 100644 --- a/docs/cookbook/adding-an-llm-adapter.md +++ b/docs/cookbook/adding-an-llm-adapter.md @@ -1,10 +1,6 @@ # Cookbook: adding an LLM adapter -How to connect a new model provider. Reference implementations: -`packages/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm-pi-ai` -(wrapping an LLM library). Read the `StreamChunk` doc in -`packages/llm/src/types.ts` first — it records the protocol conventions both -adapters were verified against. +How to connect a new model provider. Reference implementations: `packages/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm-pi-ai` (wrapping an LLM library). Read the `StreamChunk` doc in `packages/llm/src/types.ts` first — it records the protocol conventions both adapters were verified against. ## The shape @@ -22,54 +18,26 @@ export function apply(ctx: Context, config: Config) { } ``` -Registration is effect-based (HMR-safe); one adapter per model name — -duplicates throw. Secrets are cordis-native: schemastery Config with env -fallbacks, fed from cordis.yml via `!!js process.env.MY_KEY`. Never read -ad-hoc key files in code. +Registration is effect-based (HMR-safe); one adapter per model name — duplicates throw. Secrets are cordis-native: schemastery Config with env fallbacks, fed from cordis.yml via `!!js process.env.MY_KEY`. Never read ad-hoc key files in code. ## Protocol obligations (the contract two implementations verified) -- Emit `usage` BEFORE `finish`; emit NOTHING after `finish`. The robust way: - buffer finish/usage until the provider's end-of-stream marker, then flush - (handles providers that send trailing usage-only chunks). -- Tool-call `arguments` are RAW JSON strings end-to-end; stream fragments as - `argumentsDelta`. If your provider hands back parsed objects, re-stringify - at `block-end`. -- Allocate block `index`es in first-seen stream order; reuse the index for - every delta of the same block. -- Errors have exactly two sanctioned paths: THROW from `stream()` (transport - and protocol failures — use `LlmError` with a stable code), or end the - stream with `finish {kind: 'error' | 'aborted'}` (provider in-band - failures). Consumers handle both; pick per failure class and document it. +- Emit `usage` BEFORE `finish`; emit NOTHING after `finish`. The robust way: buffer finish/usage until the provider's end-of-stream marker, then flush (handles providers that send trailing usage-only chunks). +- Tool-call `arguments` are RAW JSON strings end-to-end; stream fragments as `argumentsDelta`. If your provider hands back parsed objects, re-stringify at `block-end`. +- Allocate block `index`es in first-seen stream order; reuse the index for every delta of the same block. +- Errors have exactly two sanctioned paths: THROW from `stream()` (transport and protocol failures — use `LlmError` with a stable code), or end the stream with `finish {kind: 'error' | 'aborted'}` (provider in-band failures). Consumers handle both; pick per failure class and document it. - Honor `options.signal` (pass it to fetch / your SDK). -- `prefill` and other unsupported `GenerateOptions` fields: throw - `LlmError(..., 'UNSUPPORTED')` rather than silently dropping. +- `prefill` and other unsupported `GenerateOptions` fields: throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping. -Provider-specific request knobs (thinking modes, effort levels) belong in -the ADAPTER's Config, not in `GenerateOptions` — the core vocabulary stays -provider-neutral. +Provider-specific request knobs (thinking modes, effort levels) belong in the ADAPTER's Config, not in `GenerateOptions` — the core vocabulary stays provider-neutral. ## Structure that worked -Split the adapter into testable stages (llm-deepseek's layout): wire types -(`types.ts`, coverage-exempt) → request serializer → SSE/transport parser → -chunk-translation state machine → a thin adapter class wiring them. Each -stage gets its own unit suite. +Split the adapter into testable stages (llm-deepseek's layout): wire types (`types.ts`, coverage-exempt) → request serializer → SSE/transport parser → chunk-translation state machine → a thin adapter class wiring them. Each stage gets its own unit suite. ## Testing -- **Unit: mock the provider, not the harness.** A scripted `node:http` - server speaking the provider's wire format covers happy paths, every error - status, malformed payloads, premature closes, and aborts — no network, and - it drives the 100% per-file coverage gate. Works for SDK-backed adapters - too (point the SDK's baseURL at the mock). -- **Hostile framing tests.** Split stream payloads at arbitrary byte - positions (including mid-UTF-8) — real networks do. -- **E2E: `tests/*.e2e.ts`** under `yarn test:e2e`, gated with - `describe.skipIf(!process.env.MY_KEY)` so CI (no secrets) stays green. - Cover each model × each provider mode you map (thinking on/off, effort - levels), a tool-call round trip INCLUDING the follow-up turn with results - in history, and loose assertions only (substring/structure, bounded - maxTokens — real models are nondeterministic). -- Register the e2e file pattern in `knip.json` (per-workspace `entry` - override) or knip flags it unused. +- **Unit: mock the provider, not the harness.** A scripted `node:http` server speaking the provider's wire format covers happy paths, every error status, malformed payloads, premature closes, and aborts — no network, and it drives the 100% per-file coverage gate. Works for SDK-backed adapters too (point the SDK's baseURL at the mock). +- **Hostile framing tests.** Split stream payloads at arbitrary byte positions (including mid-UTF-8) — real networks do. +- **E2E: `tests/*.e2e.ts`** under `yarn test:e2e`, gated with `describe.skipIf(!process.env.MY_KEY)` so CI (no secrets) stays green. Cover each model × each provider mode you map (thinking on/off, effort levels), a tool-call round trip INCLUDING the follow-up turn with results in history, and loose assertions only (substring/structure, bounded maxTokens — real models are nondeterministic). +- Register the e2e file pattern in `knip.json` (per-workspace `entry` override) or knip flags it unused. diff --git a/docs/rfc/001-property-based-testing.md b/docs/rfc/001-property-based-testing.md index 78193496de..cb519129de 100644 --- a/docs/rfc/001-property-based-testing.md +++ b/docs/rfc/001-property-based-testing.md @@ -4,44 +4,21 @@ Status: proposed ## Problem -Example-based tests pin the cases we thought of. The harness's core is -protocol-shaped — chunk streams, event logs, schema conversion — where the -input space is combinatorial and the interesting bugs live in interleavings -nobody wrote an example for (the `streamBlocks` ordering bug survived 100% -line coverage of the happy paths). +Example-based tests pin the cases we thought of. The harness's core is protocol-shaped — chunk streams, event logs, schema conversion — where the input space is combinatorial and the interesting bugs live in interleavings nobody wrote an example for (the `streamBlocks` ordering bug survived 100% line coverage of the happy paths). ## Proposal Adopt fast-check (vitest integration) with generators for our vocabulary: -- **BlockAssembler**: arbitrary chunk sequences (valid and malformed — - duplicate indices, stragglers after block-end, missing block-start). - Invariants: `flushReady() + flushRemaining() ≡ blocks()` in order; - `streamBlocks ≡ generate().message.content`; memory bounded (partials map - size ≤ distinct indices); idempotent re-assembly. -- **Session**: arbitrary event logs (seeded generators over SessionEventMap). - Invariants: `deriveMessages` deterministic; replay-from-seed produces - identical derivation; seq strictly monotonic; derived history unaffected by - non-message events. -- **Schema DSL**: arbitrary SchemaSpecs. Invariants: generated JSON Schema's - `required` array equals the `required: true` keys at every nesting level; - conversion is total (never throws); generated args satisfying `InferArgs` - validate against the generated schema (once RFC 005's validator exists — - the two RFCs compose). -- **Inbox/loop**: arbitrary send/steer/abort schedules against a scripted - adapter. Invariants: no message lost (every send/steer appears in the log - exactly once), turn numbers strictly increase, status transitions follow - idle→running→idle/disposed. +- **BlockAssembler**: arbitrary chunk sequences (valid and malformed — duplicate indices, stragglers after block-end, missing block-start). Invariants: `flushReady() + flushRemaining() ≡ blocks()` in order; `streamBlocks ≡ generate().message.content`; memory bounded (partials map size ≤ distinct indices); idempotent re-assembly. +- **Session**: arbitrary event logs (seeded generators over SessionEventMap). Invariants: `deriveMessages` deterministic; replay-from-seed produces identical derivation; seq strictly monotonic; derived history unaffected by non-message events. +- **Schema DSL**: arbitrary SchemaSpecs. Invariants: generated JSON Schema's `required` array equals the `required: true` keys at every nesting level; conversion is total (never throws); generated args satisfying `InferArgs` validate against the generated schema (once RFC 005's validator exists — the two RFCs compose). +- **Inbox/loop**: arbitrary send/steer/abort schedules against a scripted adapter. Invariants: no message lost (every send/steer appears in the log exactly once), turn numbers strictly increase, status transitions follow idle→running→idle/disposed. ## Plan -One `tests/properties.spec.ts` per package; fast-check as devDependency; -numRuns tuned so the suite stays under ~10s locally, with a nightly CI job -running 100× the iterations. Failures persist their seed in the report so -agents can reproduce deterministically. +One `tests/properties.spec.ts` per package; fast-check as devDependency; numRuns tuned so the suite stays under ~10s locally, with a nightly CI job running 100× the iterations. Failures persist their seed in the report so agents can reproduce deterministically. ## Risks -Generator quality determines value — invest in generators that produce -*realistic-but-adversarial* streams, not uniform noise. Property flake from -timeouts must be treated as a finding, not retried away. +Generator quality determines value — invest in generators that produce *realistic-but-adversarial* streams, not uniform noise. Property flake from timeouts must be treated as a finding, not retried away. diff --git a/docs/rfc/002-mutation-testing.md b/docs/rfc/002-mutation-testing.md index 0ac4f95e08..ffdf43495e 100644 --- a/docs/rfc/002-mutation-testing.md +++ b/docs/rfc/002-mutation-testing.md @@ -4,35 +4,23 @@ Status: proposed ## Problem -The per-file 100% coverage gate (ADR 0007) proves every line *executes* under -test — not that any assertion would notice if the line were wrong. Under -agent-written tests, coverage pressure can produce execution-without-assertion. -Mutation testing measures what coverage cannot: whether the suite *kills* -deliberately injected bugs. +The per-file 100% coverage gate (ADR 0007) proves every line *executes* under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite *kills* deliberately injected bugs. ## Proposal Stryker (`@stryker-mutator/vitest-runner`) over `packages/*/src`: -- **PR-scoped incremental runs** (changed files only) as a CI job — fast - enough to gate merges once tuned. -- **Nightly full runs** with a tracked mutation score; start by recording, - then set the threshold at the observed baseline and ratchet upward (same - policy as coverage: thresholds only ever tighten). -- Surviving mutants are work items: an agent picks a survivor, writes the - killing test, repeats — a well-shaped autonomous loop. -- Equivalent mutants (provably behavior-preserving) get annotated exclusions - with reasons, mirroring the `/* v8 ignore */` policy. +- **PR-scoped incremental runs** (changed files only) as a CI job — fast enough to gate merges once tuned. +- **Nightly full runs** with a tracked mutation score; start by recording, then set the threshold at the observed baseline and ratchet upward (same policy as coverage: thresholds only ever tighten). +- Surviving mutants are work items: an agent picks a survivor, writes the killing test, repeats — a well-shaped autonomous loop. +- Equivalent mutants (provably behavior-preserving) get annotated exclusions with reasons, mirroring the `/* v8 ignore */` policy. ## Plan -1. Add Stryker config scoped to one package (llm — smallest, most algorithmic) - and measure runtime. +1. Add Stryker config scoped to one package (llm — smallest, most algorithmic) and measure runtime. 2. Expand to all packages; record baseline scores in the config. 3. Wire the nightly job; add the incremental PR job once runtime is acceptable. ## Risks -Runtime: mutation testing is expensive; per-file 100% coverage helps (every -mutant is at least reached). If PR-scoped runs stay too slow, keep them -nightly-only and rely on the score ratchet. +Runtime: mutation testing is expensive; per-file 100% coverage helps (every mutant is at least reached). If PR-scoped runs stay too slow, keep them nightly-only and rely on the score ratchet. diff --git a/docs/rfc/003-deterministic-and-stress-testing.md b/docs/rfc/003-deterministic-and-stress-testing.md index 2b1bcb5e2e..b1329a3123 100644 --- a/docs/rfc/003-deterministic-and-stress-testing.md +++ b/docs/rfc/003-deterministic-and-stress-testing.md @@ -4,37 +4,20 @@ Status: proposed ## Problem -Several loop tests synchronize with `setTimeout(30)` sleeps — flakiness debt -that wastes agent cycles on retries and can mask ordering bugs. Separately, -our core architectural promise (any session log replays to identical derived -history) is asserted in two tests but is cheap to assert *everywhere*. And -the inbox wakeup race was verified by hand exactly once; nothing re-verifies -it continuously. +Several loop tests synchronize with `setTimeout(30)` sleeps — flakiness debt that wastes agent cycles on retries and can mask ordering bugs. Separately, our core architectural promise (any session log replays to identical derived history) is asserted in two tests but is cheap to assert *everywhere*. And the inbox wakeup race was verified by hand exactly once; nothing re-verifies it continuously. ## Proposal Three measures: -1. **No wall-clock sleeps in tests.** Replace `setTimeout(N)` waits with - event-driven waits (the existing `waitForIdle` pattern, extended to - `waitForStatus`, `waitForEvent(n)`) or vitest fake timers where time - itself is under test. Enforce with a lint rule banning `setTimeout` in - `packages/*/tests` outside an allowlisted helper module. -2. **Universal replay fixture.** A shared test helper wraps the loop harness - so that after every test, the agent's session log is replayed into a fresh - Session and `deriveMessages()` equality is asserted automatically. The - invariant then gets checked hundreds of times per CI run across every - scenario the suite produces, not twice. -3. **Nightly race stress.** A CI job running the agent-loop and inbox suites - with `vitest --repeat=200` (and `--shuffle`) to flush scheduling-dependent - failures; any flake found is a bug to fix, never a retry. +1. **No wall-clock sleeps in tests.** Replace `setTimeout(N)` waits with event-driven waits (the existing `waitForIdle` pattern, extended to `waitForStatus`, `waitForEvent(n)`) or vitest fake timers where time itself is under test. Enforce with a lint rule banning `setTimeout` in `packages/*/tests` outside an allowlisted helper module. +2. **Universal replay fixture.** A shared test helper wraps the loop harness so that after every test, the agent's session log is replayed into a fresh Session and `deriveMessages()` equality is asserted automatically. The invariant then gets checked hundreds of times per CI run across every scenario the suite produces, not twice. +3. **Nightly race stress.** A CI job running the agent-loop and inbox suites with `vitest --repeat=200` (and `--shuffle`) to flush scheduling-dependent failures; any flake found is a bug to fix, never a retry. ## Plan -Land 1 and 2 together (they touch the same helpers); add the nightly job -after the suite is sleep-free so repeats are fast. +Land 1 and 2 together (they touch the same helpers); add the nightly job after the suite is sleep-free so repeats are fast. ## Risks -Fake timers interact subtly with Promise scheduling in the loop — prefer -event-driven waits; reserve fake timers for timer-service behavior itself. +Fake timers interact subtly with Promise scheduling in the loop — prefer event-driven waits; reserve fake timers for timer-service behavior itself. diff --git a/docs/rfc/004-architectural-conformance.md b/docs/rfc/004-architectural-conformance.md index 338659557a..aa5b785be0 100644 --- a/docs/rfc/004-architectural-conformance.md +++ b/docs/rfc/004-architectural-conformance.md @@ -4,40 +4,24 @@ Status: proposed ## Problem -Two architectural guarantees currently live only in prose: (1) nothing -depends on the concrete loop package (the microkernel promise, ADR 0002), and -(2) every LlmAdapter speaks the chunk protocol correctly. Both should be -mechanical (ADR 0007). +Two architectural guarantees currently live only in prose: (1) nothing depends on the concrete loop package (the microkernel promise, ADR 0002), and (2) every LlmAdapter speaks the chunk protocol correctly. Both should be mechanical (ADR 0007). ## Proposal **dependency-cruiser** with rules: -- `packages/*` (except agent-loop's own tests and examples/) must not import - `@deepseek-ai/dsh-agent-loop`. -- No cross-package deep imports (`@deepseek-ai/dsh-*/src/...` paths) — public - entry points only. +- `packages/*` (except agent-loop's own tests and examples/) must not import `@deepseek-ai/dsh-agent-loop`. +- No cross-package deep imports (`@deepseek-ai/dsh-*/src/...` paths) — public entry points only. - No import cycles anywhere in packages/. - `vendor/*` must not import from `packages/*`. -- Layering: dsh-llm imports nothing from other dsh packages; dsh-session only - dsh-llm; etc. (the dependency table in packages/README.md, enforced). +- Layering: dsh-llm imports nothing from other dsh packages; dsh-session only dsh-llm; etc. (the dependency table in packages/README.md, enforced). -**Adapter conformance kit** in dsh-llm (`@deepseek-ai/dsh-llm/conformance`): -a reusable vitest suite parameterized by an adapter factory, asserting the -chunk-protocol contract — index monotonicity per block, no deltas after -`block-end` for an index, exactly one `finish`, usage at most once, every -`tool-call-delta` carries the call id, abort honored promptly. Run it against -the mocks now; the DeepSeek V4 adapter inherits it on day one. Optionally a -dev-mode `strictAdapter()` wrapper enforcing the same at runtime behind a -debug flag (pairs with RFC 005's invariants). +**Adapter conformance kit** in dsh-llm (`@deepseek-ai/dsh-llm/conformance`): a reusable vitest suite parameterized by an adapter factory, asserting the chunk-protocol contract — index monotonicity per block, no deltas after `block-end` for an index, exactly one `finish`, usage at most once, every `tool-call-delta` carries the call id, abort honored promptly. Run it against the mocks now; the DeepSeek V4 adapter inherits it on day one. Optionally a dev-mode `strictAdapter()` wrapper enforcing the same at runtime behind a debug flag (pairs with RFC 005's invariants). ## Plan -dependency-cruiser config + CI step first (an hour of work, permanent -guarantee); the conformance kit lands with its first consumer test against -MockAdapter, and is a prerequisite for the V4 adapter phase. +dependency-cruiser config + CI step first (an hour of work, permanent guarantee); the conformance kit lands with its first consumer test against MockAdapter, and is a prerequisite for the V4 adapter phase. ## Risks -Dep-cruiser rule maintenance as packages are added — keep rules pattern-based -(`dsh-*`) rather than enumerated. +Dep-cruiser rule maintenance as packages are added — keep rules pattern-based (`dsh-*`) rather than enumerated. diff --git a/docs/rfc/005-runtime-validation-and-error-taxonomy.md b/docs/rfc/005-runtime-validation-and-error-taxonomy.md index 742152e0cd..4b5486604a 100644 --- a/docs/rfc/005-runtime-validation-and-error-taxonomy.md +++ b/docs/rfc/005-runtime-validation-and-error-taxonomy.md @@ -6,44 +6,20 @@ Status: proposed Three gaps where compile-time guarantees stop: -1. Tool args are model-generated JSON — `defineTool`'s `InferArgs` claim - is only as true as the model's output. Today a malformed call reaches - `execute` untyped-in-practice. -2. Tool errors flatten to a text block; name/code/stack are lost, so future - sandbox/retry plugins can't distinguish ENOENT from EACCES, and the model - gets less actionable feedback than it could. -3. Loop ordering invariants (seq monotonicity, step/turn event nesting, - turn-number continuity) are asserted only where tests look. +1. Tool args are model-generated JSON — `defineTool`'s `InferArgs` claim is only as true as the model's output. Today a malformed call reaches `execute` untyped-in-practice. +2. Tool errors flatten to a text block; name/code/stack are lost, so future sandbox/retry plugins can't distinguish ENOENT from EACCES, and the model gets less actionable feedback than it could. +3. Loop ordering invariants (seq monotonicity, step/turn event nesting, turn-number continuity) are asserted only where tests look. ## Proposal -1. **Schema validation in defineTool**: before `execute`, validate parsed - args against the SchemaSpec (the converter already encodes the structure — - a small interpreter walks it: presence of required keys, primitive type - checks, enum membership, recursion into objects/arrays). On mismatch, - return an `isError` ToolExecutionResult describing the violation — the - model can self-correct. Raw-registered tools (MCP) keep validating their - own input. -2. **Structured error taxonomy**: per-package error classes extending a - common `HarnessError` (name, `code`, `cause` chaining). - `ToolExecutionResult` gains optional `error: { name, code }` alongside the - model-facing text. The loop's `errorData` consumes it; session `error` - events carry the code. This also properly fixes the non-Error-throw - message degradation found in review. -3. **Dev-mode invariants**: a `dsh-invariants` debug plugin (everything is a - plugin — it's just listeners) asserting, when enabled: session seq strictly - increases; `step/start` precedes its chunks; `turn/start`/`turn/end` pair - and nest; tool/call has a matching tool/result; status transitions are - legal. Enabled in tests and the demo; off in production. Doubles as - executable documentation of the event contract. +1. **Schema validation in defineTool**: before `execute`, validate parsed args against the SchemaSpec (the converter already encodes the structure — a small interpreter walks it: presence of required keys, primitive type checks, enum membership, recursion into objects/arrays). On mismatch, return an `isError` ToolExecutionResult describing the violation — the model can self-correct. Raw-registered tools (MCP) keep validating their own input. +2. **Structured error taxonomy**: per-package error classes extending a common `HarnessError` (name, `code`, `cause` chaining). `ToolExecutionResult` gains optional `error: { name, code }` alongside the model-facing text. The loop's `errorData` consumes it; session `error` events carry the code. This also properly fixes the non-Error-throw message degradation found in review. +3. **Dev-mode invariants**: a `dsh-invariants` debug plugin (everything is a plugin — it's just listeners) asserting, when enabled: session seq strictly increases; `step/start` precedes its chunks; `turn/start`/`turn/end` pair and nest; tool/call has a matching tool/result; status transitions are legal. Enabled in tests and the demo; off in production. Doubles as executable documentation of the event contract. ## Plan -2 first (taxonomy is a dependency of 1's error shape), then 1, then 3. -Property tests (RFC 001) then close the loop: generated args ↔ validator ↔ -InferArgs agreement. +2 first (taxonomy is a dependency of 1's error shape), then 1, then 3. Property tests (RFC 001) then close the loop: generated args ↔ validator ↔ InferArgs agreement. ## Risks -Validator/InferArgs drift — covered by the RFC 001 composition property. -Validation cost per call is negligible next to a model call. +Validator/InferArgs drift — covered by the RFC 001 composition property. Validation cost per call is negligible next to a model call. diff --git a/docs/rfc/006-doc-sync-and-api-reports.md b/docs/rfc/006-doc-sync-and-api-reports.md index cc3a546173..3b59a1f07b 100644 --- a/docs/rfc/006-doc-sync-and-api-reports.md +++ b/docs/rfc/006-doc-sync-and-api-reports.md @@ -4,36 +4,18 @@ Status: proposed ## Problem -AGENTS.md policy says docs and code must stay strictly in sync, but sync is -verified by eyeball. Review has already caught drift twice (a cookbook -example contradicting the type policy; a README citing the wrong -registerAdapter call). Public API changes are similarly invisible — nothing -makes "this commit changed the public surface" an explicit, reviewable fact. +AGENTS.md policy says docs and code must stay strictly in sync, but sync is verified by eyeball. Review has already caught drift twice (a cookbook example contradicting the type policy; a README citing the wrong registerAdapter call). Public API changes are similarly invisible — nothing makes "this commit changed the public surface" an explicit, reviewable fact. ## Proposal -1. **Typecheck documentation code blocks.** A script extracts fenced ```ts - blocks from README.md / docs/architecture.md / packages/*/README.md into a - temp project resolving workspace packages, and runs tsc. Blocks that are - intentionally elided get an explicit `ts ignore-check` info string — - opt-out is visible in the source. (twoslash is the fancier alternative; - start with plain extraction.) -2. **Generate or verify the event-taxonomy table.** The table in - docs/architecture.md duplicates the `Events` declarations. Either generate - it from source (ts-morph walk over the `declare module 'cordis'` blocks) - or CI-assert that every declared event name appears in the table and vice - versa. -3. **API reports.** api-extractor (or `tsc --emitDeclarationOnly` + a - normalized public-surface dump) producing a checked-in `etc/.api.md` - per package; CI fails if regeneration differs. Every public-API change - becomes a diff line a reviewer (or review agent) must see. +1. **Typecheck documentation code blocks.** A script extracts fenced ```ts blocks from README.md / docs/architecture.md / packages/*/README.md into a temp project resolving workspace packages, and runs tsc. Blocks that are intentionally elided get an explicit `ts ignore-check` info string — opt-out is visible in the source. (twoslash is the fancier alternative; start with plain extraction.) +2. **Generate or verify the event-taxonomy table.** The table in docs/architecture.md duplicates the `Events` declarations. Either generate it from source (ts-morph walk over the `declare module 'cordis'` blocks) or CI-assert that every declared event name appears in the table and vice versa. +3. **API reports.** api-extractor (or `tsc --emitDeclarationOnly` + a normalized public-surface dump) producing a checked-in `etc/.api.md` per package; CI fails if regeneration differs. Every public-API change becomes a diff line a reviewer (or review agent) must see. ## Plan -1 is a standalone script + CI step. 3 next (it also documents the surface for -plugin authors). 2 last — verify-don't-generate is likely sufficient. +1 is a standalone script + CI step. 3 next (it also documents the surface for plugin authors). 2 last — verify-don't-generate is likely sufficient. ## Risks -Doc blocks often show fragments; the ignore-check escape hatch must stay rare -or the gate is theater — lint the ratio if needed. +Doc blocks often show fragments; the ignore-check escape hatch must stay rare or the gate is theater — lint the ratio if needed. diff --git a/docs/rfc/007-supply-chain-and-vendor-drift.md b/docs/rfc/007-supply-chain-and-vendor-drift.md index 8f639b84fc..efe8529f29 100644 --- a/docs/rfc/007-supply-chain-and-vendor-drift.md +++ b/docs/rfc/007-supply-chain-and-vendor-drift.md @@ -4,38 +4,19 @@ Status: proposed ## Problem -The vendor manifest (ADR 0001) is enforced at commit time in the *forward* -direction (vendored change ⇒ manifest update) but nothing verifies the -manifest's *claims*: that vendor/ actually equals upstream-at-SHA plus -exactly the logged modifications. And the handful of true npm dependencies -have no advisory monitoring or update cadence. +The vendor manifest (ADR 0001) is enforced at commit time in the *forward* direction (vendored change ⇒ manifest update) but nothing verifies the manifest's *claims*: that vendor/ actually equals upstream-at-SHA plus exactly the logged modifications. And the handful of true npm dependencies have no advisory monitoring or update cadence. ## Proposal -1. **Vendor drift check** (nightly CI): clone the upstream repos at the - manifest SHAs (shallow), copy the corresponding package sources, and diff - against `vendor/*/src`. The job fails unless the diff matches the logged - local modifications (kept as a checked-in patch file per modification — - the log entries become verifiable artifacts rather than prose). -2. **Dependency advisories**: osv-scanner (or `yarn npm audit`) job on the - lockfile, scheduled + on lockfile-touching PRs. -3. **License inventory**: a script asserting every vendored package carries - its LICENSE and that package.json `license` fields match the inventory in - vendor/README.md (we mix vendored MIT with our BSD-3) — CI step. -4. **Renovate** (or a scheduled agent task) proposing npm dependency updates - in small PRs that ride the full gate suite; vendored packages are excluded - (their updates follow the manifest sync procedure, ideally as a - semi-automated agent workflow: fetch upstream, re-apply patches, run - gates, open PR with the manifest table updated). +1. **Vendor drift check** (nightly CI): clone the upstream repos at the manifest SHAs (shallow), copy the corresponding package sources, and diff against `vendor/*/src`. The job fails unless the diff matches the logged local modifications (kept as a checked-in patch file per modification — the log entries become verifiable artifacts rather than prose). +2. **Dependency advisories**: osv-scanner (or `yarn npm audit`) job on the lockfile, scheduled + on lockfile-touching PRs. +3. **License inventory**: a script asserting every vendored package carries its LICENSE and that package.json `license` fields match the inventory in vendor/README.md (we mix vendored MIT with our BSD-3) — CI step. +4. **Renovate** (or a scheduled agent task) proposing npm dependency updates in small PRs that ride the full gate suite; vendored packages are excluded (their updates follow the manifest sync procedure, ideally as a semi-automated agent workflow: fetch upstream, re-apply patches, run gates, open PR with the manifest table updated). ## Plan -3 is trivial — do first. 1 requires network access from CI to the upstream -repos (private — needs a token) and converting the two existing logged -modifications into patch files. 2 and 4 are config. +3 is trivial — do first. 1 requires network access from CI to the upstream repos (private — needs a token) and converting the two existing logged modifications into patch files. 2 and 4 are config. ## Risks -Upstream repos are private mirrors; CI credentials and availability are the -main friction for the drift check. If blocked, run it as a local scheduled -agent task instead of CI. +Upstream repos are private mirrors; CI credentials and availability are the main friction for the drift check. If blocked, run it as a local scheduled agent task instead of CI. diff --git a/docs/rfc/008-immutable-public-surfaces.md b/docs/rfc/008-immutable-public-surfaces.md index ca0b6b6eff..e4faf9a0a4 100644 --- a/docs/rfc/008-immutable-public-surfaces.md +++ b/docs/rfc/008-immutable-public-surfaces.md @@ -4,41 +4,21 @@ Status: proposed ## Problem -The session log is append-only by contract, but `session.events` returns -`readonly SessionEvent[]` whose *elements* are mutable: a plugin can reach in -and rewrite history (`events[0].data.content.push(...)`), silently breaking -replay equivalence and the derived-history guarantee. The same applies to -derived messages and prompt assemblies passed through waterfalls — mutation -is sometimes the intended idiom (waterfall middleware mutates the request) -and sometimes corruption (mutating a *logged* event), and the types don't -distinguish. +The session log is append-only by contract, but `session.events` returns `readonly SessionEvent[]` whose *elements* are mutable: a plugin can reach in and rewrite history (`events[0].data.content.push(...)`), silently breaking replay equivalence and the derived-history guarantee. The same applies to derived messages and prompt assemblies passed through waterfalls — mutation is sometimes the intended idiom (waterfall middleware mutates the request) and sometimes corruption (mutating a *logged* event), and the types don't distinguish. ## Proposal Make immutability part of the type where mutation is corruption: -- `SessionEvent` data becomes `DeepReadonly` on the way OUT of a session - (`events`, `session/event` listeners); `append()` keeps taking plain - mutable input. A `DeepReadonly` utility type lands in dsh-llm next to - the brand/never helpers. -- `deriveMessages()` returns deep-readonly messages; the loop clones before - handing a mutable request to the `agent/request` waterfall (mutation there - is sanctioned — the clone makes the boundary explicit and cheap, once per - step). -- `PromptAssembly` stays mutable through its waterfall (sanctioned) but the - registry's internal section list is cloned per assembly (already true). -- Optionally, dev-mode `Object.freeze` of event data behind the RFC 005 - invariants flag, so sanctioned-mutation violations throw in tests rather - than corrupting silently. +- `SessionEvent` data becomes `DeepReadonly` on the way OUT of a session (`events`, `session/event` listeners); `append()` keeps taking plain mutable input. A `DeepReadonly` utility type lands in dsh-llm next to the brand/never helpers. +- `deriveMessages()` returns deep-readonly messages; the loop clones before handing a mutable request to the `agent/request` waterfall (mutation there is sanctioned — the clone makes the boundary explicit and cheap, once per step). +- `PromptAssembly` stays mutable through its waterfall (sanctioned) but the registry's internal section list is cloned per assembly (already true). +- Optionally, dev-mode `Object.freeze` of event data behind the RFC 005 invariants flag, so sanctioned-mutation violations throw in tests rather than corrupting silently. ## Plan -Introduce `DeepReadonly`, flip the session read paths, fix resulting -compile errors in consumers (expected: a handful in tests), add the -freeze-in-dev option alongside RFC 005's invariants plugin. +Introduce `DeepReadonly`, flip the session read paths, fix resulting compile errors in consumers (expected: a handful in tests), add the freeze-in-dev option alongside RFC 005's invariants plugin. ## Risks -`DeepReadonly` types can produce noisy errors at waterfall boundaries where -mutation IS the API — keep the mutable/readonly boundary exactly at "logged -vs in-flight" and document it in the session README. +`DeepReadonly` types can produce noisy errors at waterfall boundaries where mutation IS the API — keep the mutable/readonly boundary exactly at "logged vs in-flight" and document it in the session README. diff --git a/docs/rfc/README.md b/docs/rfc/README.md index b7e50632bf..5c527b6b4c 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -1,9 +1,6 @@ # RFCs -Proposals for substantial future work — reviewed before implementation, -unlike ADRs (which record decisions already made). Each RFC groups a related -set of ideas from the quality/robustness proposal (2026-06-11); statuses -move proposed → accepted → implemented (then usually graduate to an ADR). +Proposals for substantial future work — reviewed before implementation, unlike ADRs (which record decisions already made). Each RFC groups a related set of ideas from the quality/robustness proposal (2026-06-11); statuses move proposed → accepted → implemented (then usually graduate to an ADR). | # | Title | Status | |---|---|---| diff --git a/examples/coding-agent/README.md b/examples/coding-agent/README.md index b9ce3ee7af..8775bc8891 100644 --- a/examples/coding-agent/README.md +++ b/examples/coding-agent/README.md @@ -1,8 +1,7 @@ # coding-agent The first REAL agent wiring: DeepSeek V4 + the bash tool suite + stdio chat -+ JSONL persistence, loaded from `cordis.yml`. Where echo-agent proves the -skeleton with mocks, this example is a usable coding assistant. ++ JSONL persistence, loaded from `cordis.yml`. Where echo-agent proves the skeleton with mocks, this example is a usable coding assistant. ## Run it @@ -13,11 +12,7 @@ skeleton with mocks, this example is a usable coding assistant. yarn demo:coding ``` -Type a coding task. The agent's only tools are `bash` (+ `bash_output` / -`bash_kill` for background tasks): file reads, writes, searches, and test -runs all happen through shell commands, each in a fresh `bash -c` (the -system prompt tells the model to pass `workdir` instead of `cd`). Reasoning -streams dimmed; tool calls/results render inline. +Type a coding task. The agent's only tools are `bash` (+ `bash_output` / `bash_kill` for background tasks): file reads, writes, searches, and test runs all happen through shell commands, each in a fresh `bash -c` (the system prompt tells the model to pass `workdir` instead of `cd`). Reasoning streams dimmed; tool calls/results render inline. ``` > fix the failing test in /path/to/project @@ -39,13 +34,7 @@ streams dimmed; tool calls/results render inline. ## End-to-end tests (`yarn test:e2e`, key-gated) -- `tests/full-loop.e2e.ts` — the canary: real model runs `echo e2e-ok` - through the real bash tool; asserts `tool/call`/`tool/result` session - events and the final answer. -- `tests/coding-task.e2e.ts` — the swebench-style smoke: a temp dir holds - `add.js` (with `a - b` where `a + b` belongs) and a failing - `add.test.js`; the agent must fix the bug and verify. The test re-runs - `node add.test.js` ITSELF and inspects the files — agent claims are not - trusted. +- `tests/full-loop.e2e.ts` — the canary: real model runs `echo e2e-ok` through the real bash tool; asserts `tool/call`/`tool/result` session events and the final answer. +- `tests/coding-task.e2e.ts` — the swebench-style smoke: a temp dir holds `add.js` (with `a - b` where `a + b` belongs) and a failing `add.test.js`; the agent must fix the bug and verify. The test re-runs `node add.test.js` ITSELF and inspects the files — agent claims are not trusted. Both self-skip without `DEEPSEEK_API_KEY`. diff --git a/examples/echo-agent/README.md b/examples/echo-agent/README.md index 5823d28b1d..9bde765ea3 100644 --- a/examples/echo-agent/README.md +++ b/examples/echo-agent/README.md @@ -5,14 +5,10 @@ Runnable demo: stdin chat with a scripted mock model and an echo tool. ## What it shows - A complete Cordis app loaded from `cordis.yml` — the standard "stack of plugins" pattern -- `mock-llm.ts` — a mock `LlmAdapter` that streams scripted responses and calls the - `echo` tool when the user types "echo " -- `echo-tool.ts` — a tool registered via `ctx.tools.register()` that echoes text - back uppercased -- `session-jsonl.ts` — a minimal persistence plugin: write-behind buffering of - `session/event` notifications, drained to a JSONL file at `session/flush` -- `stdio-chat.ts` — a minimal UI plugin: reads stdin lines and `send`/`steer`s - the agent, renders stream deltas, tool calls, and tool results +- `mock-llm.ts` — a mock `LlmAdapter` that streams scripted responses and calls the `echo` tool when the user types "echo " +- `echo-tool.ts` — a tool registered via `ctx.tools.register()` that echoes text back uppercased +- `session-jsonl.ts` — a minimal persistence plugin: write-behind buffering of `session/event` notifications, drained to a JSONL file at `session/flush` +- `stdio-chat.ts` — a minimal UI plugin: reads stdin lines and `send`/`steer`s the agent, renders stream deltas, tool calls, and tool results ## Plugin files @@ -32,9 +28,6 @@ yarn demo node --expose-internals --import tsx examples/echo-agent/start.ts ``` -Type a message and press Enter. "echo " triggers a tool call round-trip -(the mock model requests the `echo` tool, which echoes the text uppercased, -and the next model step acknowledges it). +Type a message and press Enter. "echo " triggers a tool call round-trip (the mock model requests the `echo` tool, which echoes the text uppercased, and the next model step acknowledges it). -The session is persisted to `.jsonl` in the `examples/echo-agent/` -directory. Clean up with: `rm -f examples/echo-agent/*.jsonl` +The session is persisted to `.jsonl` in the `examples/echo-agent/` directory. Clean up with: `rm -f examples/echo-agent/*.jsonl` diff --git a/packages/AGENTS.md b/packages/AGENTS.md index 1b4ee426d7..a246559c2c 100644 --- a/packages/AGENTS.md +++ b/packages/AGENTS.md @@ -1,30 +1,16 @@ # AGENTS.md — Harness Packages -This directory contains all `@deepseek-ai/dsh-*` harness packages. When editing -code here, follow these conventions: +This directory contains all `@deepseek-ai/dsh-*` harness packages. When editing code here, follow these conventions: -- **Effect-based registrations**: every contribution (tool, section, adapter, - agent, event listener) goes through `ctx.effect()` / `ctx.on()`, and - `register()` methods return disposers. Never use bare arrays or manual cleanup. -- **Declaration merging**: services declare their ctx key in - `declare module 'cordis' { interface Context { } }` and their events in - `interface Events`. Merge-extensible maps (`ContentBlockMap`, - `MessageSourceMap`, `FinishReasonMap`, `TurnTriggerMap`, `TurnEndReasonMap`, - `SessionEventMap`) are how plugins add new variants. -- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)`; - call `next()` to delegate, or return without it to short-circuit (veto). Never - call `next()` after returning. -- **Tests**: vitest in `packages//tests/*.spec.ts`. Every registry needs an - HMR-safety test (register a plugin, dispose its fiber, assert cleanup). Err on - the side of more tests — edge cases, error paths, event ordering, races. +- **Effect-based registrations**: every contribution (tool, section, adapter, agent, event listener) goes through `ctx.effect()` / `ctx.on()`, and `register()` methods return disposers. Never use bare arrays or manual cleanup. +- **Declaration merging**: services declare their ctx key in `declare module 'cordis' { interface Context { } }` and their events in `interface Events`. Merge-extensible maps (`ContentBlockMap`, `MessageSourceMap`, `FinishReasonMap`, `TurnTriggerMap`, `TurnEndReasonMap`, `SessionEventMap`) are how plugins add new variants. +- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)`; call `next()` to delegate, or return without it to short-circuit (veto). Never call `next()` after returning. +- **Tests**: vitest in `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test (register a plugin, dispose its fiber, assert cleanup). Err on the side of more tests — edge cases, error paths, event ordering, races. Naming notes: - Files `src/index.ts` export the service default + all public types - `src/types.ts` contain only types — no runtime code - Tests live at package level under `tests/`, not `src/__tests__/` -- A package's README and module/JSDoc comments are part of the change: when you - alter behavior (config keys, defaults, error codes, wire fields), update them - in the same commit. CI has no doc-sync gate, so stale docs are on the author. +- A package's README and module/JSDoc comments are part of the change: when you alter behavior (config keys, defaults, error codes, wire fields), update them in the same commit. CI has no doc-sync gate, so stale docs are on the author. -Read the per-package README.md for package-specific details: service API, -events, extension points, TODOs. +Read the per-package README.md for package-specific details: service API, events, extension points, TODOs. diff --git a/packages/README.md b/packages/README.md index dd38752a6f..1a85d7c6f1 100644 --- a/packages/README.md +++ b/packages/README.md @@ -1,10 +1,6 @@ # Packages -Harness packages, all under the `@deepseek-ai/dsh-*` scope. Each package is a -Cordis service (microkernel plugin-style): it exports a default `Service` class -that gets registered via `ctx.plugin()`, declares its ctx key and events through -declaration merging, and exposes extension points through `ctx.effect()`, -`ctx.on()`, and `ctx.waterfall()`. +Harness packages, all under the `@deepseek-ai/dsh-*` scope. Each package is a Cordis service (microkernel plugin-style): it exports a default `Service` class that gets registered via `ctx.plugin()`, declares its ctx key and events through declaration merging, and exposes extension points through `ctx.effect()`, `ctx.on()`, and `ctx.waterfall()`. ## Dependency graph @@ -17,9 +13,7 @@ dsh-tools ← dsh-llm, dsh-system-prompt, dsh-agent dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent ``` -The rule: plugins depend on interfaces, never on the concrete loop. -`dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the -`dsh-agent` vocabulary if the loop is replaced. +The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. ## What goes where @@ -32,25 +26,13 @@ The rule: plugins depend on interfaces, never on the concrete loop. | `agent/` | Agent interface, registry, `agent/*` event vocabulary | `ctx.agents` | | `agent-loop/` | THE concrete plugin: `LoopAgent` + the loop driver | `ctx.agentLoop` | -Each package has its own `README.md` with purpose, service API, events, -extension points, and deliberate non-goals (TODOs). +Each package has its own `README.md` with purpose, service API, events, extension points, and deliberate non-goals (TODOs). ## Conventions (applied across all harness packages) -- **Registrations are effects**: every contribution (adapter, tool, section, - agent, event listener) goes through `ctx.effect()` / `ctx.on()`, so disposal - and HMR clean up automatically. Every `register()` returns the disposer. -- **Declaration merging for events and ctx**: services declare their events in - `declare module 'cordis' { interface Events { ... } }` and their ctx key in - `interface Context`. -- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)` - and MUST call `next()` to delegate; returning without it short-circuits (the - veto mechanism). -- **Extensible unions**: `ContentBlockMap`, `MessageSourceMap`, - `FinishReasonMap`, `TurnTriggerMap`, `TurnEndReasonMap`, and `SessionEventMap` - use the merge-extensible-map pattern so plugins can add variants via - declaration merging. -- **ESM everywhere**; imports use package names across package boundaries, - `.ts` extensions within a package. -- **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every - registry needs an HMR-safety test. Err on the side of more tests. +- **Registrations are effects**: every contribution (adapter, tool, section, agent, event listener) goes through `ctx.effect()` / `ctx.on()`, so disposal and HMR clean up automatically. Every `register()` returns the disposer. +- **Declaration merging for events and ctx**: services declare their events in `declare module 'cordis' { interface Events { ... } }` and their ctx key in `interface Context`. +- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)` and MUST call `next()` to delegate; returning without it short-circuits (the veto mechanism). +- **Extensible unions**: `ContentBlockMap`, `MessageSourceMap`, `FinishReasonMap`, `TurnTriggerMap`, `TurnEndReasonMap`, and `SessionEventMap` use the merge-extensible-map pattern so plugins can add variants via declaration merging. +- **ESM everywhere**; imports use package names across package boundaries, `.ts` extensions within a package. +- **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test. Err on the side of more tests. diff --git a/packages/agent-loop/README.md b/packages/agent-loop/README.md index 42eda325c1..e68698612f 100644 --- a/packages/agent-loop/README.md +++ b/packages/agent-loop/README.md @@ -1,24 +1,18 @@ # dsh-agent-loop -THE concrete agent plugin: `LoopAgent` and the loop driver. Implements the -`Agent` interface and drives the session/turn/step lifecycle. +THE concrete agent plugin: `LoopAgent` and the loop driver. Implements the `Agent` interface and drives the session/turn/step lifecycle. -This is the only package in the harness that contains concrete loop logic. -Everything else is an abstract service or a plugin against extension seams — -new behavior goes into plugins, not here. +This is the only package in the harness that contains concrete loop logic. Everything else is an abstract service or a plugin against extension seams — new behavior goes into plugins, not here. ## Service: `AgentLoop` (ctx key: `agentLoop`) ### Public API -- `ctx.agentLoop.create(id: string, options?: AgentOptions): LoopAgent` - Create an agent, start its loop, and register it in `ctx.agents`. Disposed - with the calling fiber. +- `ctx.agentLoop.create(id: string, options?: AgentOptions): LoopAgent` Create an agent, start its loop, and register it in `ctx.agents`. Disposed with the calling fiber. ### Injected services -`agents`, `sessions`, `llm`, `tools`, `systemPrompt` — all five interface -services. +`agents`, `sessions`, `llm`, `tools`, `systemPrompt` — all five interface services. ### Configuration (schemastery) @@ -36,11 +30,8 @@ Agents listed in config are auto-created at startup. ### Classes -- `LoopAgent` — the concrete `Agent` implementation. Owns the inbox (`Inbox`), - the per-step `AbortController`, and the loop driver. Everything observable - happens through session events and the `agent/*` event taxonomy. -- `Inbox` — per-agent queued + steering FIFOs (`enqueue`, `steer`, `drainQueued`, - `drainSteering`, `waitForQueued`). +- `LoopAgent` — the concrete `Agent` implementation. Owns the inbox (`Inbox`), the per-step `AbortController`, and the loop driver. Everything observable happens through session events and the `agent/*` event taxonomy. +- `Inbox` — per-agent queued + steering FIFOs (`enqueue`, `steer`, `drainQueued`, `drainSteering`, `waitForQueued`). ### Loop lifecycle (`loop.ts`) @@ -68,13 +59,11 @@ forever: idle unless more queued ``` -Error containment: a throwing plugin ends the **turn**, never the loop. Dispose -mid-turn emits `agent/status('disposed')` and ends with reason `disposed`. +Error containment: a throwing plugin ends the **turn**, never the loop. Dispose mid-turn emits `agent/status('disposed')` and ends with reason `disposed`. ### What is NOT here -Everything that goes beyond "call the model, run the tools, repeat" belongs to -plugins listening on the event taxonomy: +Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy: - Hooks: `agent/request`, `agent/step-result`, `tools/execute`, `agent/turn-continuation` - Compaction: `agent/request` - Sandbox, permission, plan mode: `tools/execute` diff --git a/packages/agent/README.md b/packages/agent/README.md index ff21c41708..5f6fb85893 100644 --- a/packages/agent/README.md +++ b/packages/agent/README.md @@ -1,25 +1,20 @@ # dsh-agent -Agent interface, registry, and `agent/*` event vocabulary. Every plugin (UI, -hooks, orchestrators) programs against the `Agent` handle defined here — it has -zero loop dependency, so the loop is swappable. +Agent interface, registry, and `agent/*` event vocabulary. Every plugin (UI, hooks, orchestrators) programs against the `Agent` handle defined here — it has zero loop dependency, so the loop is swappable. ## Service: `AgentRegistry` (ctx key: `agents`) -Tracks live agents so UI, hook, and orchestrator plugins can find them without -importing the concrete loop package. +Tracks live agents so UI, hook, and orchestrator plugins can find them without importing the concrete loop package. ### Public API -- `ctx.agents.register(agent: Agent): () => void` - Register a live agent. Disposed with the calling fiber. +- `ctx.agents.register(agent: Agent): () => void` Register a live agent. Disposed with the calling fiber. - `ctx.agents.get(id: string): Agent | undefined` - `ctx.agents.list(): Agent[]` ### Events -The full `agent/*` event taxonomy is declared via declaration merging in -`dsh-agent` (not `dsh-agent-loop`), so plugins depend only on this package. +The full `agent/*` event taxonomy is declared via declaration merging in `dsh-agent` (not `dsh-agent-loop`), so plugins depend only on this package. #### Lifecycle (emit) @@ -34,12 +29,9 @@ The full `agent/*` event taxonomy is declared via declaration merging in #### Interception seams (waterfall) -- `agent/request` — mutate `GenerateOptions` before the model call (hooks, - compaction, model switching, tool filtering) -- `agent/step-result` — post-process the assembled assistant message before tool - dispatch (validates what the log records) -- `agent/turn-continuation` — override the continue/stop decision - (force-continue /loop, force-stop budget guard) +- `agent/request` — mutate `GenerateOptions` before the model call (hooks, compaction, model switching, tool filtering) +- `agent/step-result` — post-process the assembled assistant message before tool dispatch (validates what the log records) +- `agent/turn-continuation` — override the continue/stop decision (force-continue /loop, force-stop budget guard) #### Streaming + tool (emit) @@ -52,20 +44,15 @@ The full `agent/*` event taxonomy is declared via declaration merging in The handle every plugin programs against: - `agent.send(content, options?)` — queue a message; starts a turn when idle -- `agent.steer(content, options?)` — steer a running turn (inject between steps); - behaves like `send` when idle -- `agent.inject(content, options?)` — inject in-session context without triggering - a turn (context/message event); next request sees it +- `agent.steer(content, options?)` — steer a running turn (inject between steps); behaves like `send` when idle +- `agent.inject(content, options?)` — inject in-session context without triggering a turn (context/message event); next request sees it - `agent.abort(reason?)` — abort the in-flight step - `agent.session`, `agent.status`, `agent.options`, `agent.id` ### Extension points -- Agent creation: `AgentLoop.create()` is the concrete implementation (in - `dsh-agent-loop`). Replace the loop by implementing `Agent` and registering - via `ctx.agents.register()`. -- Event listeners: all `agent/*` events are declared here — no dependency on the - loop package needed. +- Agent creation: `AgentLoop.create()` is the concrete implementation (in `dsh-agent-loop`). Replace the loop by implementing `Agent` and registering via `ctx.agents.register()`. +- Event listeners: all `agent/*` events are declared here — no dependency on the loop package needed. ### What is NOT here (TODO) diff --git a/packages/bash-local/README.md b/packages/bash-local/README.md index 3efab3ecfd..cceefbb67c 100644 --- a/packages/bash-local/README.md +++ b/packages/bash-local/README.md @@ -1,9 +1,6 @@ # @deepseek-ai/dsh-bash-local -Local-subprocess implementation of the `@deepseek-ai/dsh-bash` executor seam: -`LocalBashExecutor` spawns `bash -c ` per call in its own process -group, collects bounded output with full-stream spill files, and escalates -kills SIGTERM→SIGKILL across the whole group. +Local-subprocess implementation of the `@deepseek-ai/dsh-bash` executor seam: `LocalBashExecutor` spawns `bash -c ` per call in its own process group, collects bounded output with full-stream spill files, and escalates kills SIGTERM→SIGKILL across the whole group. ## Config @@ -19,33 +16,14 @@ kills SIGTERM→SIGKILL across the whole group. ## Behavior (and where it came from) -Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and -pi; the notable choices: +Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi; the notable choices: -- **Spawn per call, no shell state** — every call is a fresh non-login - `bash -c` (deterministic; no rc files). All four surveyed tools spawn per - call. `TODO(stateful-shell)` in `src/run.ts` records the two proven - stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec - sessions) for when real workflows demand them. -- **Process-group kills with escalation** — children are spawned `detached` - (own process group); kills send SIGTERM to the group, then SIGKILL after a - 3s grace (OpenCode's escalation; pipelines and subshells die with the - parent). ESRCH is tolerated; daemons that re-parent away from the group can - still survive — same caveat as the surveyed tools. -- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` - keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode - rationale) while the FULL stream is appended to a temp file whose path is - reported. The model can `grep`/`tail` the spill file with bash itself. -- **Model-friendly env** — `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` - (Codex's hardcoded set) so pagers and ANSI color don't garble results. -- **Background tasks** — `start()` returns immediately, no timeout applies - (Claude Code detaches timeouts when backgrounding), `readOutput()` is - incremental with whole-stream byte offsets, and disposal kills everything. +- **Spawn per call, no shell state** — every call is a fresh non-login `bash -c` (deterministic; no rc files). All four surveyed tools spawn per call. `TODO(stateful-shell)` in `src/run.ts` records the two proven stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec sessions) for when real workflows demand them. +- **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after a 3s grace (OpenCode's escalation; pipelines and subshells die with the parent). ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools. +- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported. The model can `grep`/`tail` the spill file with bash itself. +- **Model-friendly env** — `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` (Codex's hardcoded set) so pagers and ANSI color don't garble results. +- **Background tasks** — `start()` returns immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), `readOutput()` is incremental with whole-stream byte offsets, and disposal kills everything. ## Sandboxing -`TODO(permissions/sandbox)`: execution policy does NOT belong in this -package. Wrap the `tools/execute` waterfall (veto/ask) or implement a -sandboxing `BashExecutor` — see docs/architecture.md § plugin checklist. -Reference points: Claude Code wraps commands in sandbox-exec/bubblewrap; -Codex applies seatbelt/landlock plus an execpolicy prefix-rule engine. +`TODO(permissions/sandbox)`: execution policy does NOT belong in this package. Wrap the `tools/execute` waterfall (veto/ask) or implement a sandboxing `BashExecutor` — see docs/architecture.md § plugin checklist. Reference points: Claude Code wraps commands in sandbox-exec/bubblewrap; Codex applies seatbelt/landlock plus an execpolicy prefix-rule engine. diff --git a/packages/bash/README.md b/packages/bash/README.md index 2480240c77..12976d551b 100644 --- a/packages/bash/README.md +++ b/packages/bash/README.md @@ -1,11 +1,8 @@ # @deepseek-ai/dsh-bash -The **bash executor seam**: an abstract `BashExecutor` service (`ctx.bash`) -defining WHAT a bash backend does — run commands, manage background tasks — -without saying HOW. +The **bash executor seam**: an abstract `BashExecutor` service (`ctx.bash`) defining WHAT a bash backend does — run commands, manage background tasks — without saying HOW. -This package is one third of the bash capability, split so each concern can -evolve (and be swapped) independently: +This package is one third of the bash capability, split so each concern can evolve (and be swapped) independently: | Package | Role | |---|---| @@ -13,11 +10,7 @@ evolve (and be swapped) independently: | `@deepseek-ai/dsh-bash-local` | an implementation: local subprocesses | | `@deepseek-ai/dsh-tool-bash` | the model-facing tool schemas over `ctx.bash` | -The split mirrors the LLM seam (`LlmService`/`LlmAdapter`) and the agent-tool -survey: pi hides execution behind a `BashOperations` interface (local shell / -SSH / VM backends), Codex behind an exec-server protocol. A future sandboxed, -containerized, or remote executor implements this interface and the tool -schemas don't change. +The split mirrors the LLM seam (`LlmService`/`LlmAdapter`) and the agent-tool survey: pi hides execution behind a `BashOperations` interface (local shell / SSH / VM backends), Codex behind an exec-server protocol. A future sandboxed, containerized, or remote executor implements this interface and the tool schemas don't change. ## Service API (`ctx.bash`) @@ -30,13 +23,8 @@ schemas don't change. | `kill(id)` | Kill a running task. Returns `false` when it already finished; throws for unknown ids. | | `onTaskDone(listener)` | Completion listener (effect-based, disposer returned). Fires exactly once per task; never after the service is disposed. | -Implementations subclass `BashExecutor`, implement the abstract methods, and -call `notifyTaskDone(task)` on background completion. Disposal must kill every -running task (no orphan processes) — see the HMR-safety tests. +Implementations subclass `BashExecutor`, implement the abstract methods, and call `notifyTaskDone(task)` on background completion. Disposal must kill every running task (no orphan processes) — see the HMR-safety tests. ## Vocabulary -`BashExecSpec` (command, workdir?, timeoutMs?, signal?) → -`BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr -as `CollectedOutput`) and `BashTask`/`BashTaskRead` for the background side. -See `src/types.ts` for the full contracts. +`BashExecSpec` (command, workdir?, timeoutMs?, signal?) → `BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr as `CollectedOutput`) and `BashTask`/`BashTaskRead` for the background side. See `src/types.ts` for the full contracts. diff --git a/packages/llm-deepseek/README.md b/packages/llm-deepseek/README.md index eca584a1df..5884136510 100644 --- a/packages/llm-deepseek/README.md +++ b/packages/llm-deepseek/README.md @@ -1,13 +1,8 @@ # @deepseek-ai/dsh-llm-deepseek -DeepSeek chat-completions adapter for the harness LLM seam: hand-rolled -`fetch` + SSE translation from the official wire format (source of truth: -the API docs — guides/thinking_mode, guides/tool_calls, -api/create-chat-completion) into the `StreamChunk` protocol. +DeepSeek chat-completions adapter for the harness LLM seam: hand-rolled `fetch` + SSE translation from the official wire format (source of truth: the API docs — guides/thinking_mode, guides/tool_calls, api/create-chat-completion) into the `StreamChunk` protocol. -A second, independent implementation of the same seam exists in -`@deepseek-ai/dsh-llm-pi-ai` (library-backed). Same Config shape — pick one -per context (registering both for the same model names throws by design). +A second, independent implementation of the same seam exists in `@deepseek-ai/dsh-llm-pi-ai` (library-backed). Same Config shape — pick one per context (registering both for the same model names throws by design). ## Config @@ -22,62 +17,30 @@ per context (registering both for the same model names throws by design). reasoningEffort: high # optional; high | max — omitted ⇒ not sent ``` -`models` lists every model name this one adapter instance serves: the adapter -registers itself for each (the harness model name IS the wire `model` string), -so a `generate`/`stream` call routes to it whenever `options.model` is any of -them. Registering a second adapter for a name already taken throws -`LlmError('DUPLICATE_ADAPTER')` (the LLM service enforces one adapter per -model, all-or-nothing). +`models` lists every model name this one adapter instance serves: the adapter registers itself for each (the harness model name IS the wire `model` string), so a `generate`/`stream` call routes to it whenever `options.model` is any of them. Registering a second adapter for a name already taken throws `LlmError('DUPLICATE_ADAPTER')` (the LLM service enforces one adapter per model, all-or-nothing). -`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort` -wire field is not sent and the server applies its own default for the model. -The only accepted values are `high` and `max` (DeepSeek's official effort -levels). It is meaningful only with thinking enabled (the provider default). +`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort` wire field is not sent and the server applies its own default for the model. The only accepted values are `high` and `max` (DeepSeek's official effort levels). It is meaningful only with thinking enabled (the provider default). -`thinking`/`reasoningEffort` are adapter-level request defaults serialized as -the official top-level `thinking: {type}` / `reasoning_effort` wire fields. -They live in adapter config (not `GenerateOptions`) to keep the core -vocabulary provider-neutral. +`thinking`/`reasoningEffort` are adapter-level request defaults serialized as the official top-level `thinking: {type}` / `reasoning_effort` wire fields. They live in adapter config (not `GenerateOptions`) to keep the core vocabulary provider-neutral. ## Wire-format notes (verified live + against the official docs) -- Streaming only (`stream_options.include_usage` always on). `usage` may - arrive attached to the finish chunk or as a trailing usage-only chunk — - the translator defers both to `[DONE]`, so `usage` always precedes - `finish` and nothing follows `finish`. -- The first thinking-mode chunk carries `reasoning_content: ""` — handled - (no spurious reasoning block). -- **Reasoning passback rule**: on assistant turns that carried tool calls, - `reasoning_content` is serialized back in history (required by the API in - thinking mode); on tool-call-free turns it is dropped (ignored anyway — - saves tokens). -- `strict` on tool schemas passes through (officially Beta; the public API - wants the `/beta` base URL for it, the internal endpoint accepts it - directly). -- Cache accounting: `cacheReadTokens` ← `prompt_cache_hit_tokens` / - `prompt_tokens_details.cached_tokens`; DeepSeek reports no cache-write - metric. +- Streaming only (`stream_options.include_usage` always on). `usage` may arrive attached to the finish chunk or as a trailing usage-only chunk — the translator defers both to `[DONE]`, so `usage` always precedes `finish` and nothing follows `finish`. +- The first thinking-mode chunk carries `reasoning_content: ""` — handled (no spurious reasoning block). +- **Reasoning passback rule**: on assistant turns that carried tool calls, `reasoning_content` is serialized back in history (required by the API in thinking mode); on tool-call-free turns it is dropped (ignored anyway — saves tokens). +- `strict` on tool schemas passes through (officially Beta; the public API wants the `/beta` base URL for it, the internal endpoint accepts it directly). +- Cache accounting: `cacheReadTokens` ← `prompt_cache_hit_tokens` / `prompt_tokens_details.cached_tokens`; DeepSeek reports no cache-write metric. ## Limitations (MVP, documented deliberately) -- `prefill` throws `LlmError('UNSUPPORTED')` — DeepSeek's chat-prefix - completion is a Beta feature on the `/beta` base URL; future work. +- `prefill` throws `LlmError('UNSUPPORTED')` — DeepSeek's chat-prefix completion is a Beta feature on the `/beta` base URL; future work. - `image` blocks are skipped (no vision support on these models). - `tool_choice` is not mapped (not part of the core vocabulary). ## Errors -Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), -`RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), `HTTP_` -otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or -`MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s -(e.g. `content_filter`, `insufficient_system_resource`) become -`finish {kind: 'error', code: }` chunks. +Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), `HTTP_` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: }` chunks. ## Testing -Unit suites run against a local `node:http` mock SSE server (no network). -Real-API coverage lives in `tests/adapter.e2e.ts` (`yarn test:e2e`, -key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both -official effort levels, including the thinking+tools round trip with -reasoning passback. +Unit suites run against a local `node:http` mock SSE server (no network). Real-API coverage lives in `tests/adapter.e2e.ts` (`yarn test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback. diff --git a/packages/llm-pi-ai/README.md b/packages/llm-pi-ai/README.md index 6f5a28b48a..9d0b2f85cf 100644 --- a/packages/llm-pi-ai/README.md +++ b/packages/llm-pi-ai/README.md @@ -1,33 +1,19 @@ # @deepseek-ai/dsh-llm-pi-ai -DeepSeek adapter for the harness LLM seam backed by -[`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai) -(the LLM library behind the pi agent). +DeepSeek adapter for the harness LLM seam backed by [`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai) (the LLM library behind the pi agent). ## Why a second adapter exists -`@deepseek-ai/dsh-llm-deepseek` already talks to the same endpoint. This -package is its **design-verification twin**: same models, same wire -protocol, completely different internals — a unified LLM library with its -own event vocabulary versus hand-rolled fetch/SSE. Anything the harness -`StreamChunk` protocol cannot express for BOTH implementations is a -core-vocabulary bug. The differences it exercised on purpose: +`@deepseek-ai/dsh-llm-deepseek` already talks to the same endpoint. This package is its **design-verification twin**: same models, same wire protocol, completely different internals — a unified LLM library with its own event vocabulary versus hand-rolled fetch/SSE. Anything the harness `StreamChunk` protocol cannot express for BOTH implementations is a core-vocabulary bug. The differences it exercised on purpose: -- pi-ai hands back tool-call `arguments` as **parsed objects**; the harness - keeps raw JSON strings (re-stringified at `block-end`). -- pi-ai reports failures as **in-stream error events** (it never throws - mid-stream); these map to `finish {kind:'error'|'aborted'}` chunks — the - protocol's other sanctioned error path besides throwing (which - llm-deepseek uses). -- pi-ai folds reasoning tokens into `usage.output`; there is no separate - reasoning count to map. -- pi-ai's options omit stop sequences; `GenerateOptions.stop` is injected - via its `onPayload` hook. +- pi-ai hands back tool-call `arguments` as **parsed objects**; the harness keeps raw JSON strings (re-stringified at `block-end`). +- pi-ai reports failures as **in-stream error events** (it never throws mid-stream); these map to `finish {kind:'error'|'aborted'}` chunks — the protocol's other sanctioned error path besides throwing (which llm-deepseek uses). +- pi-ai folds reasoning tokens into `usage.output`; there is no separate reasoning count to map. +- pi-ai's options omit stop sequences; `GenerateOptions.stop` is injected via its `onPayload` hook. ## Config -Same shape as llm-deepseek (one-line swap in cordis.yml), with pi-ai's -thinking-level vocabulary: +Same shape as llm-deepseek (one-line swap in cordis.yml), with pi-ai's thinking-level vocabulary: ```yaml - id: llm @@ -41,20 +27,12 @@ thinking-level vocabulary: ## Dependency weight -pi-ai declares the openai/anthropic/google/mistral/AWS SDKs as install-time -dependencies. They are lazy-loaded — only the openai SDK actually loads for -this adapter — but they do land in `node_modules`. Accepted for a package -whose purpose is design verification. +pi-ai declares the openai/anthropic/google/mistral/AWS SDKs as install-time dependencies. They are lazy-loaded — only the openai SDK actually loads for this adapter — but they do land in `node_modules`. Accepted for a package whose purpose is design verification. ## Limitations -Same MVP contract as llm-deepseek: `prefill` throws `UNSUPPORTED`, images -are not representable, `tool_choice` is not mapped. +Same MVP contract as llm-deepseek: `prefill` throws `UNSUPPORTED`, images are not representable, `tool_choice` is not mapped. ## Testing -Unit suites run against a local `node:http` mock SSE server (pi-ai's openai -SDK happily talks to any base URL). Real-API coverage in -`tests/adapter.e2e.ts` (`yarn test:e2e`, key-gated): V4 Flash + V4 Pro across -all exposed reasoning levels (off/high/xhigh), the thinking+tools round trip, -and a cross-adapter structural-equivalence check against llm-deepseek. +Unit suites run against a local `node:http` mock SSE server (pi-ai's openai SDK happily talks to any base URL). Real-API coverage in `tests/adapter.e2e.ts` (`yarn test:e2e`, key-gated): V4 Flash + V4 Pro across all exposed reasoning levels (off/high/xhigh), the thinking+tools round trip, and a cross-adapter structural-equivalence check against llm-deepseek. diff --git a/packages/llm/README.md b/packages/llm/README.md index 22996f7605..e8138776d9 100644 --- a/packages/llm/README.md +++ b/packages/llm/README.md @@ -1,24 +1,18 @@ # dsh-llm -Provider-neutral LLM vocabulary and abstract service. This package defines the -canonical language spoken by the agent loop, session logs, and every plugin. +Provider-neutral LLM vocabulary and abstract service. This package defines the canonical language spoken by the agent loop, session logs, and every plugin. ## Service: `LlmService` (ctx key: `llm`) -An adapter registry plus streaming / non-streaming call surfaces. Both call -surfaces are interceptable via waterfall events. +An adapter registry plus streaming / non-streaming call surfaces. Both call surfaces are interceptable via waterfall events. ### Public API -- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` - Register an adapter for the given model names. Disposed with the calling fiber. +- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` Register an adapter for the given model names. Disposed with the calling fiber. - `ctx.llm.models(): string[]` — model names with a registered adapter. -- `ctx.llm.stream(options: GenerateOptions): AsyncIterable` - Stream one model call as raw chunks (token-level deltas). -- `ctx.llm.streamBlocks(options: GenerateOptions): AsyncIterable` - Stream as completed content blocks (convenience view). -- `ctx.llm.generate(options: GenerateOptions): Promise` - One model call, fully assembled. +- `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). +- `ctx.llm.streamBlocks(options: GenerateOptions): AsyncIterable` Stream as completed content blocks (convenience view). +- `ctx.llm.generate(options: GenerateOptions): Promise` One model call, fully assembled. ### Events @@ -30,35 +24,23 @@ surfaces are interceptable via waterfall events. ### Extension points -- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` - to add a new model provider. -- Wrap `llm/stream` or `llm/generate` via `ctx.on()` waterfall listeners for - caching, retry, logging, rate-limiting, etc. +- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` to add a new model provider. +- Wrap `llm/stream` or `llm/generate` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc. ### Content-block vocabulary (`types.ts`) -Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, -`tool-result`, `image`. The union is derived from the merge-extensible -`ContentBlockMap`, so plugins can add block types via declaration merging. +Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`, `image`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. -Streaming is a raw chunk protocol (`block-start`, `text-delta`, -`reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). -`BlockAssembler` is the single shared implementation that assembles chunks into -blocks/messages. +Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. ### Classes -- `LlmAdapter` — abstract base class for provider adapters. The only required - method is `stream()`. -- `BlockAssembler` — incrementally assembles raw chunks into complete content - blocks and an assistant message. Used by the agent loop (raw chunks for replay +- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`. +- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. Used by the agent loop (raw chunks for replay + assembled for history) and by `streamBlocks()`/`generate()`. -- `LlmError` — typed error with a `code` string (`NO_ADAPTER`, - `DUPLICATE_ADAPTER`). +- `LlmError` — typed error with a `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`). ### What is NOT here (TODO) - **DeepSeek V4 adapter** — the first real adapter lands in a later phase. -- **Streaming protocol review** — the chunk protocol has `TODO(review)` markers - and needs careful review before the first real adapter (DeepSeek V4 wire - format, partial JSON arguments, interleaved reasoning signatures, ...). +- **Streaming protocol review** — the chunk protocol has `TODO(review)` markers and needs careful review before the first real adapter (DeepSeek V4 wire format, partial JSON arguments, interleaved reasoning signatures, ...). diff --git a/packages/session/README.md b/packages/session/README.md index 3af8523282..04c4653dac 100644 --- a/packages/session/README.md +++ b/packages/session/README.md @@ -1,20 +1,14 @@ # dsh-session -Event-sourced session log and in-memory store. A `Session` is the append-only -source of truth for an agent's whole interaction history — the LLM message -history is *derived* from it. +Event-sourced session log and in-memory store. A `Session` is the append-only source of truth for an agent's whole interaction history — the LLM message history is *derived* from it. ## Service: `SessionStore` (ctx key: `sessions`) -Creates and holds event-sourced `Session` instances. Persistence is intentionally -not implemented here — plugins subscribe to `session/event` and flush on -`session/flush`. +Creates and holds event-sourced `Session` instances. Persistence is intentionally not implemented here — plugins subscribe to `session/event` and flush on `session/flush`. ### Public API -- `ctx.sessions.create(id?: string, seed?: SessionEvent[]): Session` - Create a session. `seed` replays/forks an existing event log. Disposed with - the calling fiber. +- `ctx.sessions.create(id?: string, seed?: SessionEvent[]): Session` Create a session. `seed` replays/forks an existing event log. Disposed with the calling fiber. - `ctx.sessions.get(id: string): Session | undefined` - `ctx.sessions.list(): Session[]` @@ -31,35 +25,24 @@ not implemented here — plugins subscribe to `session/event` and flush on Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. - `session.append(type, data): SessionEvent` — synchronous, never blocks on I/O. -- `session.deriveMessages(): Message[]` — derive the LLM message history from - the event log. Raw `assistant/chunk` events are skipped; `context/message` and - `steering/message` render as tagged synthetic user messages. +- `session.deriveMessages(): Message[]` — derive the LLM message history from the event log. Raw `assistant/chunk` events are skipped; `context/message` and `steering/message` render as tagged synthetic user messages. - `session.events`, `session.seq`, `session.id` ### Session event vocabulary (`types.ts`) -The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, -`user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, -`tool/result`, `steering/message`, `context/message`, `usage`, `error`. +The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`, `usage`, `error`. -Merge-extensible via `SessionEventMap` — a compaction plugin adds -`compaction/marker`, etc. +Merge-extensible via `SessionEventMap` — a compaction plugin adds `compaction/marker`, etc. -Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types -for typed turn boundaries — `kind`-tagged instead of strings). +Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types for typed turn boundaries — `kind`-tagged instead of strings). ### Extension points -- Persistence plugins: subscribe to `session/event` (write-behind) and drain on - `session/flush` (awaited) and fiber dispose. See - `examples/echo-agent/src/session-jsonl.ts` for the pattern. -- Replay/fork: `ctx.sessions.create(id, seed)` seeds a new session with an - existing event log. +- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. See `examples/echo-agent/src/session-jsonl.ts` for the pattern. +- Replay/fork: `ctx.sessions.create(id, seed)` seeds a new session with an existing event log. ### What is NOT here (TODO) - **Real persistence backends** (JSONL per session dir, sqlite) — future phase. -- **Session event vocabulary review** — `TODO(review)` once the loop and a - persistence plugin coexist. -- **Session branching/tree** (pi-style entry tree) — defered unless needed beyond - seed-based forking. +- **Session event vocabulary review** — `TODO(review)` once the loop and a persistence plugin coexist. +- **Session branching/tree** (pi-style entry tree) — defered unless needed beyond seed-based forking. diff --git a/packages/system-prompt/README.md b/packages/system-prompt/README.md index 979606488d..6f14e40f88 100644 --- a/packages/system-prompt/README.md +++ b/packages/system-prompt/README.md @@ -1,19 +1,14 @@ # dsh-system-prompt -System prompt assembly registry. Plugins contribute ordered text sections and -tool-schema providers; the agent loop calls `assemble()` once per step. +System prompt assembly registry. Plugins contribute ordered text sections and tool-schema providers; the agent loop calls `assemble()` once per step. ## Service: `SystemPrompt` (ctx key: `systemPrompt`) ### Public API -- `ctx.systemPrompt.section(section: PromptSection): () => void` - Contribute a section. Disposed with the calling fiber. -- `ctx.systemPrompt.tools(provider: () => ToolSchema[]): () => void` - Contribute tool schemas (evaluated at each assembly). Disposed with the calling - fiber. -- `ctx.systemPrompt.assemble(): Promise` - Assemble the current prompt. Runs through the `system-prompt/assemble` waterfall. +- `ctx.systemPrompt.section(section: PromptSection): () => void` Contribute a section. Disposed with the calling fiber. +- `ctx.systemPrompt.tools(provider: () => ToolSchema[]): () => void` Contribute tool schemas (evaluated at each assembly). Disposed with the calling fiber. +- `ctx.systemPrompt.assemble(): Promise` Assemble the current prompt. Runs through the `system-prompt/assemble` waterfall. ### Events @@ -24,24 +19,17 @@ tool-schema providers; the agent loop calls `assemble()` once per step. ### Key types -- `PromptSection` — `{ name, order, text: string | (() => string) }`. Sections - are concatenated in ascending `order`. -- `PromptAssembly` — `{ sections: PromptSection[], tools: ToolSchema[] }`. - Tool schemas are part of the assembly by design: "what the model is told it - can do" is one coherent thing, even though adapters transmit schemas as a - separate wire field. +- `PromptSection` — `{ name, order, text: string | (() => string) }`. Sections are concatenated in ascending `order`. +- `PromptAssembly` — `{ sections: PromptSection[], tools: ToolSchema[] }`. Tool schemas are part of the assembly by design: "what the model is told it can do" is one coherent thing, even though adapters transmit schemas as a separate wire field. - `renderPrompt(assembly)` — joins section texts with blank lines. -Merge-extensible: plugins can declare extra fields on `PromptAssembly` via -declaration merging. +Merge-extensible: plugins can declare extra fields on `PromptAssembly` via declaration merging. ### Extension points - Section providers: AGENTS.md reader, cwd notifier, persona config, etc. -- Tool schema providers: `ToolRegistry` registers itself as a tool provider - automatically. -- The `system-prompt/assemble` waterfall: mutate or replace the assembly - (system-prompt configurability, dynamic tool filtering). +- Tool schema providers: `ToolRegistry` registers itself as a tool provider automatically. +- The `system-prompt/assemble` waterfall: mutate or replace the assembly (system-prompt configurability, dynamic tool filtering). ### What is NOT here diff --git a/packages/tool-bash/README.md b/packages/tool-bash/README.md index 35f5de0c46..74a4ae5011 100644 --- a/packages/tool-bash/README.md +++ b/packages/tool-bash/README.md @@ -1,13 +1,8 @@ # @deepseek-ai/dsh-tool-bash -The model-facing bash tools — `bash`, `bash_output`, `bash_kill` — registered -over the `ctx.bash` executor seam (`@deepseek-ai/dsh-bash`). Pure schema + -text shaping; every process concern lives behind the seam, so sandboxed or -remote executor implementations swap in without changing what the model sees. +The model-facing bash tools — `bash`, `bash_output`, `bash_kill` — registered over the `ctx.bash` executor seam (`@deepseek-ai/dsh-bash`). Pure schema + text shaping; every process concern lives behind the seam, so sandboxed or remote executor implementations swap in without changing what the model sees. -Requires a loaded executor implementation (e.g. -`@deepseek-ai/dsh-bash-local`); the plugin stays pending until `ctx.bash` -exists (`inject: ['tools', 'bash']`). +Requires a loaded executor implementation (e.g. `@deepseek-ai/dsh-bash-local`); the plugin stays pending until `ctx.bash` exists (`inject: ['tools', 'bash']`). ## Tools @@ -21,42 +16,22 @@ exists (`inject: ['tools', 'bash']`). | `workdir` | string | Working directory for this call. | | `run_in_background` | boolean | Return a task id immediately; no timeout applies. | -`command`, `workdir`, and `timeoutMs` are resolved against the executor's -config defaults via `ctx.bash.resolve()` before execution, so the executor -seam (`BashExecSpec`) receives explicit `workdir`/`timeoutMs` values. +`command`, `workdir`, and `timeoutMs` are resolved against the executor's config defaults via `ctx.bash.resolve()` before execution, so the executor seam (`BashExecSpec`) receives explicit `workdir`/`timeoutMs` values. -Result text: stdout, then a `[stderr]` section, then status markers — -`[timed out after Nms]` whenever the executor's timer fired (reported -independently of how the process ended, so a command that traps SIGTERM and -exits 0 still shows it), `[killed by signal: …]` for a signal death, -`[exit code: N]` for a non-zero exit (reported, **not** `isError`: the model -decides how to react), and `[output truncated; full output: ]` when the -tail was kept. Only infrastructure failures (spawn errors, aborts) surface as -`isError` results. +Result text: stdout, then a `[stderr]` section, then status markers — `[timed out after Nms]` whenever the executor's timer fired (reported independently of how the process ended, so a command that traps SIGTERM and exits 0 still shows it), `[killed by signal: …]` for a signal death, `[exit code: N]` for a non-zero exit (reported, **not** `isError`: the model decides how to react), and `[output truncated; full output: ]` when the tail was kept. Only infrastructure failures (spawn errors, aborts) surface as `isError` results. ### `bash_output` -`task_id` → output produced **since the previous `bash_output` call** plus a -status line (`running` / `completed, exit code: N` / `killed`). Reads that -lost data to buffer bounds say so and point at the full-output spill file. +`task_id` → output produced **since the previous `bash_output` call** plus a status line (`running` / `completed, exit code: N` / `killed`). Reads that lost data to buffer bounds say so and point at the full-output spill file. ### `bash_kill` -`task_id` → SIGTERM→SIGKILL on the task's process group. Killing an -already-finished task is a reported no-op; unknown ids are errors. +`task_id` → SIGTERM→SIGKILL on the task's process group. Killing an already-finished task is a reported no-op; unknown ids are errors. ## Background completion notices -When a background task finishes, a short notice is injected into the owning -agent's session (`agent.inject()`, source `{kind: 'plugin', plugin: -'tool-bash'}`). Injection is **durable context for the next model request, -not a wake-up** — an idle agent stays idle until something sends a message. -That's why the tool descriptions tell the model to poll with `bash_output`. +When a background task finishes, a short notice is injected into the owning agent's session (`agent.inject()`, source `{kind: 'plugin', plugin: 'tool-bash'}`). Injection is **durable context for the next model request, not a wake-up** — an idle agent stays idle until something sends a message. That's why the tool descriptions tell the model to poll with `bash_output`. ## Permissions -`TODO(permissions)`: commands run with the executor's full authority. The -permission/sandbox seam is the `tools/execute` waterfall (veto or ask) plus -sandboxing `BashExecutor` implementations — see docs/architecture.md. -`@cordisjs/plugin-capability` (a named-permission service with a session -`test()`) is a candidate building block for that work. +`TODO(permissions)`: commands run with the executor's full authority. The permission/sandbox seam is the `tools/execute` waterfall (veto or ask) plus sandboxing `BashExecutor` implementations — see docs/architecture.md. `@cordisjs/plugin-capability` (a named-permission service with a session `test()`) is a candidate building block for that work. diff --git a/packages/tools/README.md b/packages/tools/README.md index 848c77b1b5..1a098169f6 100644 --- a/packages/tools/README.md +++ b/packages/tools/README.md @@ -1,24 +1,19 @@ # dsh-tools -Tool registry and execution waterfall. Tool plugins register their schemas and -executors; the agent loop executes calls through the `tools/execute` waterfall. +Tool registry and execution waterfall. Tool plugins register their schemas and executors; the agent loop executes calls through the `tools/execute` waterfall. ## Service: `ToolRegistry` (ctx key: `tools`) ### Public API -- `ctx.tools.register(definition: ToolDefinition): () => void` - Register a tool. Disposed with the calling fiber. +- `ctx.tools.register(definition: ToolDefinition): () => void` Register a tool. Disposed with the calling fiber. - `ctx.tools.get(name: string): ToolDefinition | undefined` -- `ctx.tools.schemas(): ToolSchema[]` - Schemas of all registered tools (without the `execute` functions). -- `ctx.tools.execute(exec: ToolExecution): Promise` - Execute one tool call through the `tools/execute` waterfall. +- `ctx.tools.schemas(): ToolSchema[]` Schemas of all registered tools (without the `execute` functions). +- `ctx.tools.execute(exec: ToolExecution): Promise` Execute one tool call through the `tools/execute` waterfall. ### Injected services -`SystemPrompt` — the registry automatically feeds its tool schemas into the -system-prompt assembly via `ctx.systemPrompt.tools()`. +`SystemPrompt` — the registry automatically feeds its tool schemas into the system-prompt assembly via `ctx.systemPrompt.tools()`. ### Events @@ -35,19 +30,13 @@ system-prompt assembly via `ctx.systemPrompt.tools()`. ### Extension points -- Tool plugins call `ctx.tools.register()` — schemas flow into the assembly - automatically. -- The `tools/execute` waterfall is the single seam for sandbox, permission, - hooks, and plan-mode plugins to wrap or veto a call. Listeners receive - `(exec, next)`: call `next()` to proceed, or return a result without calling - `next()` to short-circuit (veto). -- MCP servers: one plugin per server, discover tools, call - `ctx.tools.register()` with the server's schemas. +- Tool plugins call `ctx.tools.register()` — schemas flow into the assembly automatically. +- The `tools/execute` waterfall is the single seam for sandbox, permission, hooks, and plan-mode plugins to wrap or veto a call. Listeners receive `(exec, next)`: call `next()` to proceed, or return a result without calling `next()` to short-circuit (veto). +- MCP servers: one plugin per server, discover tools, call `ctx.tools.register()` with the server's schemas. ### Typed tool parameter schemas -First-party plugin authors can use the `defineTool()` helper (exported from this -package) for typed tool parameter schemas: +First-party plugin authors can use the `defineTool()` helper (exported from this package) for typed tool parameter schemas: ```ts import { defineTool } from '@deepseek-ai/dsh-tools' @@ -68,16 +57,11 @@ ctx.tools.register(defineTool({ })) ``` -The helper converts the author-facing `SchemaSpec` (with `required: true` as a -per-property boolean) to standard JSON Schema for the wire format. Raw -JSON-Schema tool definitions (from MCP servers) are still accepted by the -registry directly. +The helper converts the author-facing `SchemaSpec` (with `required: true` as a per-property boolean) to standard JSON Schema for the wire format. Raw JSON-Schema tool definitions (from MCP servers) are still accepted by the registry directly. -See `defineTool`, `SchemaSpec`, `InferArgs`, and `schemaSpecToJsonSchema` in the -public API for details. +See `defineTool`, `SchemaSpec`, `InferArgs`, and `schemaSpecToJsonSchema` in the public API for details. ### What is NOT here (TODO) -- **Tool shapes review** — when real tools land (e.g. a concurrency-safety hint - for parallel execution); phase 1 executes tool calls sequentially. +- **Tool shapes review** — when real tools land (e.g. a concurrency-safety hint for parallel execution); phase 1 executes tool calls sequentially. - **Parallel execution** — the loop currently iterates tool calls sequentially. diff --git a/vendor/AGENTS.md b/vendor/AGENTS.md index 6abdb5c117..edd71c7fe9 100644 --- a/vendor/AGENTS.md +++ b/vendor/AGENTS.md @@ -1,13 +1,7 @@ # AGENTS.md — Vendored Packages -This directory contains source-vendored copies of the Cordis framework and its -foundation libraries. See `vendor/README.md` for the manifest, local-modification -log, and the upstream sync procedure. +This directory contains source-vendored copies of the Cordis framework and its foundation libraries. See `vendor/README.md` for the manifest, local-modification log, and the upstream sync procedure. -**Do NOT edit `vendor/*/src/` files casually.** Every local divergence from -upstream must be logged exhaustively in `vendor/README.md` under "Local -modifications." The `vendor/*/tsconfig.json` files are the exception — -regenerated to fit the monorepo build, and they may be touched for type-checking -policy changes (e.g., `noImplicitAny`). +**Do NOT edit `vendor/*/src/` files casually.** Every local divergence from upstream must be logged exhaustively in `vendor/README.md` under "Local modifications." The `vendor/*/tsconfig.json` files are the exception — regenerated to fit the monorepo build, and they may be touched for type-checking policy changes (e.g., `noImplicitAny`). When changes are unavoidable, follow the sync procedure in `vendor/README.md`.