From 39b3db4b9cc3668cf4ce285f8d262978fe720efc Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 13 Jun 2026 18:50:13 +0800 Subject: [PATCH] docs: accuracy sweep, architecture restructure, two ADRs, review skill - AGENTS.md Commands: fix typecheck/build descriptions; add lint, lint:fix, test:coverage, knip, publint, hygiene (were undocumented). - Drop the bare `yarn demo` for explicit `demo:echo` + `demo:coding`; update README, examples READMEs (and document coding-agent in examples/README). - New cookbook guide: adding-a-vendored-package.md (the missing "add" half of vendor/README's update-only procedure). - architecture.md: add a table-of-contents and extract the Extension cookbook to docs/cookbook/extension-cookbook.md (link-preserving); drop the completed "restructure this document" TODO. - ADR 0009 (capability seams) + 0010 (twin LLM adapters), and a "when to write an ADR" standard in adr/README. - Add a committed dsh-code-review skill under .agents/skills, exposed to Claude Code via a tracked .claude/skills symlink (gitignore carve-out). --- .agents/skills/dsh-code-review/SKILL.md | 28 +++++++++ .claude/skills | 1 + .gitignore | 1 - AGENTS.md | 15 +++-- README.md | 5 +- docs/adr/0009-capability-seams.md | 27 ++++++++ docs/adr/0010-twin-llm-adapters.md | 22 +++++++ docs/adr/README.md | 8 +++ docs/architecture.md | 71 ++-------------------- docs/cookbook/adding-a-tool.md | 2 +- docs/cookbook/adding-a-vendored-package.md | 56 +++++++++++++++++ docs/cookbook/extension-cookbook.md | 48 +++++++++++++++ examples/README.md | 8 ++- examples/echo-agent/README.md | 2 +- package.json | 2 +- packages/README.md | 12 +++- packages/llm/README.md | 7 +-- vendor/README.md | 49 ++++----------- 18 files changed, 246 insertions(+), 118 deletions(-) create mode 100644 .agents/skills/dsh-code-review/SKILL.md create mode 120000 .claude/skills create mode 100644 docs/adr/0009-capability-seams.md create mode 100644 docs/adr/0010-twin-llm-adapters.md create mode 100644 docs/cookbook/adding-a-vendored-package.md create mode 100644 docs/cookbook/extension-cookbook.md diff --git a/.agents/skills/dsh-code-review/SKILL.md b/.agents/skills/dsh-code-review/SKILL.md new file mode 100644 index 0000000000..9af4380dde --- /dev/null +++ b/.agents/skills/dsh-code-review/SKILL.md @@ -0,0 +1,28 @@ +--- +name: dsh-code-review +description: Use when reviewing a pull request in the deepseek-harness repo — orients the reviewer to this codebase's standards (AGENTS.md conventions, defensive patterns, ADRs, quality gates) and the review-specific checks that code alone can't show +--- + +# Reviewing a DeepSeek-Harness PR + +This is a where-to-look map, not a rules list. The rules live in the docs below and are the source of truth — read them there so this skill never drifts out of sync with them. + +## Sources of truth (read, don't re-summarize) + +- **[AGENTS.md](../../../AGENTS.md) § Conventions** — effect-based registrations, declaration-merging for events/ctx keys, waterfall `next()` discipline, discriminated-union match-don't-chain, explicit-over-implicit at seams, the empty-`catch` rule, symmetry. Every PR is checked against these. +- **AGENTS.md § Defensive patterns (hard-won)** — each bullet is a bug class that bit us. Reviewing anything touching process lifecycle, async/await, disposal, or adapter error paths? Re-read this first. +- **AGENTS.md § Type Safety and Documentation** — the doc-sync rule (code change ⇒ update README + JSDoc in the SAME commit) and the no-hard-wrap markdown convention. +- **[packages/AGENTS.md](../../../packages/AGENTS.md)** — per-package conventions (file layout, the HMR-safety test requirement). +- **[ADR index](../../../docs/adr/README.md)** — the *why* behind the architecture. Especially [0007 quality gates](../../../docs/adr/0007-quality-gates.md) (what a PR must pass) and [0009 capability seams](../../../docs/adr/0009-capability-seams.md) (the three-package split). If a change seems to fight an ADR, that's a discussion, not a silent override. + +## Where to look first (review-specific, not in the docs) + +1. **Docs in sync?** If the PR changes a config key, default, error code, wire field, or event name, did it update the package README + module/JSDoc in the same diff? Stale docs are the most common miss (the doc-sync rule has no gate). +2. **HMR-safety test present?** Any new registry/registration needs a test that disposes the contributing fiber and asserts cleanup. Its absence is a blocking gap. +3. **Gates green?** typecheck, lint, test, test:coverage (100% per-file on `packages/*/src`), knip, build, publint, constraints. Don't re-review what a gate already enforces — trust the gate, spend attention on what gates can't check (intent, contracts, doc sync). +4. **e2e verifies the world, not the agent's self-report.** For real-API tests, confirm the assertion re-runs the command/checks the file externally — a keyword probe lets a cheating agent pass (see AGENTS.md e2e bullet). +5. **Seam discipline.** New swappable capability? Check it's split per ADR 0009 (interface / impl / consumer), and that the consumer injects the interface key, never an implementation type. + +## How to respond + +Technical, specific, non-performative — no "great catch", no "you're absolutely right". State the issue and where; cite the AGENTS.md bullet or ADR it relates to. When replying to inline threads on GitHub, reply in the thread (`gh api repos/{owner}/{repo}/pulls/{n}/comments/{id}/replies`), not as a top-level comment. If a suggestion would fight an ADR or an established convention, say so and link it rather than relitigating in the thread. diff --git a/.claude/skills b/.claude/skills new file mode 120000 index 0000000000..2b7a412b8f --- /dev/null +++ b/.claude/skills @@ -0,0 +1 @@ +../.agents/skills \ No newline at end of file diff --git a/.gitignore b/.gitignore index 20b383e21c..414bf3032e 100644 --- a/.gitignore +++ b/.gitignore @@ -6,7 +6,6 @@ lib/ .yarn/ yarn-error.log examples/*/*.jsonl -.claude/ coverage/ .vscode/ .DS_Store diff --git a/AGENTS.md b/AGENTS.md index 54aed8ecac..28be76ff13 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -51,12 +51,19 @@ scripts/ repo maintenance scripts (vendor-manifest guard, publint runner). ```sh yarn install # Yarn 4 workspaces (node-modules linker), node >= 24 yarn test # vitest run (packages/*/tests/**/*.spec.ts) +yarn test:coverage # vitest run --coverage (per-file 100% gate on packages/*/src) yarn test:e2e # real-API tests (packages|examples/*/tests/**/*.e2e.ts); # self-skips without DEEPSEEK_API_KEY — see Secrets below -yarn typecheck # tsc -b tsconfig.build.json (declarations only) -yarn build # typecheck + tsdown JS bundles into each package's lib/ -yarn demo # run examples/echo-agent (needs --expose-internals, the - # script passes it; type "echo hi" to see a tool call) +yarn typecheck # tsc -b tsconfig.build.json (declarations) + tsc -p + # tsconfig.typecheck.json (tests/examples typecheck too) +yarn lint # eslint . +yarn lint:fix # eslint . --fix +yarn build # tsc -b tsconfig.build.json && tsdown (JS bundles into lib/) +yarn knip # dead-code / unused-dependency check +yarn publint # package.json publish-correctness check (publishable packages/*) +yarn hygiene # knip + publint + yarn constraints +yarn demo:echo # run examples/echo-agent (no API key; type "echo hi" to + # see a tool call) — the mock skeleton yarn demo:coding # run examples/coding-agent — the real agent (needs # DEEPSEEK_API_KEY; give it a coding task) ``` diff --git a/README.md b/README.md index f08bcc096a..c13ab43210 100644 --- a/README.md +++ b/README.md @@ -12,8 +12,9 @@ This monorepo is built on the [Cordis](https://github.com/cordiverse/cordis) fra ```sh yarn install -yarn test # vitest -yarn demo # runnable echo-agent example +yarn test # vitest +yarn demo:echo # runnable echo-agent example (no API key needed) +yarn demo:coding # the real DeepSeek coding agent (needs DEEPSEEK_API_KEY) ``` For agent instructions see [AGENTS.md](AGENTS.md). For the architecture design see [docs/architecture.md](docs/architecture.md). Each subdirectory has its own README.md with local context: [packages/](packages/), [vendor/](vendor/). diff --git a/docs/adr/0009-capability-seams.md b/docs/adr/0009-capability-seams.md new file mode 100644 index 0000000000..a41f67cc48 --- /dev/null +++ b/docs/adr/0009-capability-seams.md @@ -0,0 +1,27 @@ +# ADR 0009: Capability seams — interface / implementation / consumer split + +Status: accepted (2026-06-13) + +## Context + +The harness has swappable capabilities — bash execution today, sandboxed/remote executors and alternative model providers tomorrow. A capability has three concerns that change at different rates and for different reasons: the *contract* (what the capability is), the *implementation* (how it runs), and the *consumer surface* (what the model and other plugins program against). Bundling them in one package couples those rates of change — swapping a local executor for a sandboxed one would churn the tool schemas the model sees, even though the model-facing contract never changed. + +This is distinct from "who provides vs. needs a capability at runtime", which Cordis already answers with services + `inject` (a provider registers `ctx.bash`; a consumer declares `inject: ['bash']` and its fiber pends until the service exists). That mechanism is necessary but doesn't dictate package boundaries; this ADR does. + +## Decision + +A swappable capability is **three packages**: + +1. **Interface** — an abstract service + the vocabulary types, owning the `ctx.` and depending only on cordis (e.g. `dsh-bash`: `BashExecutor`, `BashRunResult`, `BashTask`). +2. **Implementation** — a concrete subclass loaded as a plugin (e.g. `dsh-bash-local`: subprocesses, process-group kills, spill-file truncation). Sandboxed/remote backends are sibling packages implementing the same interface. +3. **Consumer** — what the model and plugins see (e.g. `dsh-tool-bash`: the `bash`/`bash_output`/`bash_kill` tool schemas). Consumers `inject` the interface key and never import implementation types. + +Implementation and consumer then evolve independently: a sandboxed executor replaces `dsh-bash-local` without touching a tool schema. + +Alternatives considered: **one combined package** — rejected because it recouples the three rates of change the split exists to separate (the whole point). **`@cordisjs/plugin-capability`** — a different axis entirely: it is a permission/capability-*security* service (named permissions with inheritance, tested against a session via `ctx.capability.test`), a candidate for the deferred permissions/sandbox work on the `tools/execute` veto seam, NOT a mechanism for swapping implementations. Confusing the two ("capability") is the trap this ADR names. + +The split is not mandatory when the parts are genuinely one concern: the LLM seam folds interface + consumer into `dsh-llm` (the consumer is the loop itself, not a swappable schema surface) with adapters as the implementation packages. Don't split preemptively — a capability with one conceivable implementation and one consumer stays one package until a second appears. + +## Consequences + +More packages and more boilerplate per capability (a `package.json`/`tsconfig`/README trio, the inject wiring). Bought: implementations and consumers ship and version independently, and a new backend never risks the model-facing contract. The rule is documented in [AGENTS.md](../../AGENTS.md) § Conventions ("Capability seams are three packages") and [architecture.md](../architecture.md) § "Capability seams"; the bash trio is the reference template. When to fold vs. split is a judgment call the architecture doc spells out — this ADR records *why* the default is to split. diff --git a/docs/adr/0010-twin-llm-adapters.md b/docs/adr/0010-twin-llm-adapters.md new file mode 100644 index 0000000000..a1af0d97dc --- /dev/null +++ b/docs/adr/0010-twin-llm-adapters.md @@ -0,0 +1,22 @@ +# ADR 0010: Two LLM adapters as a design-verification twin + +Status: accepted (2026-06-13) + +## Context + +`dsh-llm` owns a provider-neutral streaming vocabulary — the `StreamChunk` protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`) and the content-block types ([ADR 0004](0004-own-content-block-vocabulary.md)). A vocabulary defined against a single adapter risks baking that adapter's quirks into the "neutral" contract: anything the one implementation happens to do becomes the de-facto spec, and the abstraction is unverified until a second provider arrives — by which point the leak is expensive to fix. + +## Decision + +Ship **two** adapters against the one contract from the start, deliberately built on different internals: + +- `dsh-llm-deepseek` — hand-rolled `fetch` + SSE parsing against the DeepSeek API. +- `dsh-llm-pi-ai` — the same endpoint through the `@earendil-works/pi-ai` library (its own event vocabulary). + +The rule they enforce: **anything the StreamChunk vocabulary cannot express for BOTH implementations is a core-vocabulary bug**, caught immediately rather than at the next provider. The pair pinned down conventions now documented on `StreamChunk` in `dsh-llm/src/types.ts`: usage emitted before finish, nothing after finish, tool-call `arguments` as raw JSON strings end-to-end, and the two sanctioned error paths (throw from `stream()` *or* end with `finish {kind:'error'|'aborted'}`) that a consumer must handle on both sides — a divergence the library-backed adapter surfaced that a single hand-rolled adapter would have hidden. + +Alternatives considered: **a single adapter** — less code and half the e2e cost, but leaves the "provider-neutral" claim unverified; the vocabulary would encode DeepSeek-via-fetch assumptions silently. **A mock second adapter** — cheaper but doesn't exercise a real provider's wire quirks, so it proves little. The twin is real-on-real. + +## Consequences + +Double the adapter maintenance and double the key-gated e2e surface (both adapters cover V4 Flash and Pro across representative thinking/effort modes). Bought: a continuously-verified neutrality guarantee for the most leak-prone abstraction in the codebase, and a worked second example for adapter authors. The two share the core Config shape (`apiKey`/`baseURL`/`models`) so a deployment swaps mostly one line, but the reasoning knob differs — `dsh-llm-deepseek` takes `thinking`/`reasoningEffort`, `dsh-llm-pi-ai` takes a single `reasoning` level — so a swap translates that field. If the maintenance cost ever outweighs the verification value (e.g. once conformance tests from [RFC 004](../rfc/004-architectural-conformance.md) cover the contract mechanically), retiring the twin to a single adapter + the conformance kit would be a new ADR superseding this one. diff --git a/docs/adr/README.md b/docs/adr/README.md index 033b973689..f73a3c21da 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -4,6 +4,12 @@ Short, immutable records of the *why* behind decisions that shape this codebase. Format: one file per decision, numbered, with Status / Context / Decision / Consequences. An ADR is never edited into a different decision — supersede it with a new one and cross-link. +## When to write an ADR + +Write one when a decision is all three of: **durable** (it shapes the codebase beyond a single function or package), **contested** (there was a real alternative you rejected, and a reasonable engineer might have chosen it), and **surprising** (a future reader would otherwise ask "why on earth is it done this way?"). The ADR captures the *why* and *what we gave up* — the parts code and docs can't. + +Do NOT write an ADR for: a mechanical or local choice (a variable name, a one-file refactor); anything already enforced and explained by a gate or a convention in AGENTS.md; or a still-provisional decision tagged `TODO(...)` in the code — record those as TODOs and promote to an ADR only once they settle. When in doubt, the test is the "why on earth" question: if the code alone would mislead a careful reader about intent, write the ADR. + | # | Title | Status | |---|---|---| | [0001](0001-vendor-cordis-as-source.md) | Vendor Cordis as source, not npm dependencies | accepted | @@ -14,3 +20,5 @@ Format: one file per decision, numbered, with Status / Context / Decision / Cons | [0006](0006-tool-schemas-in-prompt-assembly.md) | Tool schemas are part of the system-prompt assembly | accepted | | [0007](0007-quality-gates.md) | Mechanical quality gates over prose guidelines | accepted | | [0008](0008-tsdown-over-dumble.md) | tsdown for JS bundling instead of dumble | accepted | +| [0009](0009-capability-seams.md) | Capability seams — interface / implementation / consumer split | accepted | +| [0010](0010-twin-llm-adapters.md) | Two LLM adapters as a design-verification twin | accepted | diff --git a/docs/architecture.md b/docs/architecture.md index 8c7f136923..e3f6594a78 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -8,6 +8,8 @@ The harness core is deliberately tiny: a handful of abstract services plus one c Requirement context: [Coding Harness MVP 需求分析][mvp-doc]. +**Contents:** [Layering](#layering) · [Service map](#service-map) · [Capability seams](#capability-seams-interface--implementation--consumer) · [The vocabulary (dsh-llm)](#the-vocabulary-dsh-llm) · [Event-sourced sessions](#event-sourced-sessions-dsh-session) · [Prompt assembly](#prompt-assembly-dsh-system-prompt) · [Tool pipeline](#tool-pipeline-dsh-tools) · [Agents and the loop](#agents-dsh-agent-and-the-loop-dsh-agent-loop) ([lifecycle](#loop-lifecycle-session--turn--step), [event taxonomy](#event-taxonomy), [waterfall semantics](#cordis-waterfall-semantics-important)) · [Plugin sanity checklist](#plugin-sanity-checklist) · [Extension cookbook](#extension-cookbook) · [Deferred work](#deferred-work-todo) + [microkernel-doc]: https://trtgsjkv6r.feishu.cn/wiki/VS9Lw1kQki6mDJk2UHocyuphnsc [mvp-doc]: https://trtgsjkv6r.feishu.cn/wiki/ZwK6wfBE9i91V6kzMGYcgRGanxg @@ -86,7 +88,7 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source - `user/message` → user message - `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation) - `tool/result` → user message carrying a `tool-result` block -- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: revisit the envelope once a real adapter exists. +- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: the real adapters now exist (the original precondition); the envelope still wants a deliberate review against live model behavior (`TODO(review)` in dsh-session). Replay/fork = `ctx.sessions.create(id, seedEvents)`. Trace/telemetry = listen to `session/event`. @@ -104,7 +106,7 @@ Tool schemas are deliberately **part of the assembly**: "what the model is told `execute()` runs through the **`tools/execute` waterfall** — the single seam where sandbox, permission, hooks, and plan-mode plugins wrap or veto a call. This collapses Claude Code's validate → PreToolUse → permission → execute → PostToolUse pipeline into ordered waterfall listeners. -**TODO**: tool shapes get revisited when real tools land (e.g. a concurrency-safety hint for parallel execution; phase 1 executes tool calls sequentially). +**TODO**: tool shapes get revisited now that real tools exist (the bash suite landed; the `TODO(review)` in dsh-tools is still open) — e.g. a concurrency-safety hint for parallel execution; phase 1 executes tool calls sequentially. ## Agents (dsh-agent) and the loop (dsh-agent-loop) @@ -222,75 +224,12 @@ Every MVP feature (including the TODO-marked ones), with the mechanism that impl ## Extension cookbook -### A tool plugin - -```ts -import type { Context } from 'cordis' -import { defineTool } from '@deepseek-ai/dsh-tools' - -export const name = 'my-tool' -export const inject = ['tools'] - -export function apply(ctx: Context) { - ctx.tools.register(defineTool({ - name: 'read_file', - description: 'Read a file from disk.', - parameters: { - path: { type: 'string', required: true, description: 'Absolute file path' }, - }, - async execute(args) { - // args is typed: { path: string } - const text = await readFile(args.path, 'utf8') - return [{ type: 'text', text }] - }, - })) -} -``` - -(Raw JSON-Schema `ToolDefinition`s are still accepted by `ctx.tools.register()` directly — that's how MCP-sourced tools arrive. `defineTool` is the typed sugar for first-party tools.) - -### A hook plugin (permission gate) - -```ts -export const name = 'permission-gate' - -export function apply(ctx: Context) { - ctx.on('tools/execute', async (exec, next) => { - if (!(await isAllowed(exec))) { - return { - callId: exec.callId, - content: [{ type: 'text', text: 'Denied by policy.' }], - isError: true, - } - } - return next() - }) -} -``` - -### A UI plugin - -```ts -export const name = 'my-ui' -export const inject = ['agents'] - -export function apply(ctx: Context) { - ctx.on('agent/stream-chunk', (agent, turn, step, chunk) => { - if (chunk.type === 'text-delta') render(chunk.text) - }) - onUserInput(text => ctx.agents.get('main')?.send([{ type: 'text', text }])) -} -``` - -Two complete runnable wirings exist: [`examples/echo-agent`](../examples/echo-agent) (mock model + echo tool — the all-mock skeleton check) and [`examples/coding-agent`](../examples/coding-agent) (DeepSeek V4 + the bash tool suite — the real thing; `yarn demo:coding`). Both load from `cordis.yml` with HMR. - -Step-by-step guides live in [`docs/cookbook`](./cookbook): adding a package, adding a tool, adding an LLM adapter. +Code skeletons for the three plugin shapes (tool, hook/permission-gate, UI) and the two runnable example wirings live in [docs/cookbook/extension-cookbook.md](./cookbook/extension-cookbook.md). Step-by-step guides: [adding a package](./cookbook/adding-a-package.md), [adding a tool](./cookbook/adding-a-tool.md), [adding an LLM adapter](./cookbook/adding-an-llm-adapter.md), [adding a vendored package](./cookbook/adding-a-vendored-package.md). ## Deferred work (TODO) Tracked here deliberately — each is designed-for but not implemented: -- **Restructure this document** — it has grown long; split it into focused sections (or per-area files) so readers can navigate it without scrolling the whole thing. - **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent channels beyond `send`/`steer`/events. - **Persistence backends** (JSONL session dirs, sqlite) on the `session/event` + `session/flush` seam. - **Compaction implementation** (auto thresholds, summarization prompts) on the `agent/request` seam, with its session-event types added by declaration merging. diff --git a/docs/cookbook/adding-a-tool.md b/docs/cookbook/adding-a-tool.md index 88bbde7a84..ec84d12e2a 100644 --- a/docs/cookbook/adding-a-tool.md +++ b/docs/cookbook/adding-a-tool.md @@ -47,7 +47,7 @@ Follow tool-bash's background pattern: a `run_in_background` flag returns a task ## Permissions / sandboxing -Prefer not to build policy into the tool. The seam is the `tools/execute` waterfall (veto or wrap — see the permission-gate example in docs/architecture.md), or a sandboxing implementation behind the tool's executor seam. +Prefer not to build policy into the tool. The seam is the `tools/execute` waterfall (veto or wrap — see the permission-gate example in [extension-cookbook.md](./extension-cookbook.md)), or a sandboxing implementation behind the tool's executor seam. ## Tests every tool needs diff --git a/docs/cookbook/adding-a-vendored-package.md b/docs/cookbook/adding-a-vendored-package.md new file mode 100644 index 0000000000..8a66ed08a8 --- /dev/null +++ b/docs/cookbook/adding-a-vendored-package.md @@ -0,0 +1,56 @@ +# Cookbook: adding a vendored package + +When the harness needs another upstream Cordis package (e.g. `@cordisjs/plugin-http`), it is **vendored** as pinned source under `vendor/`, not added as an npm dependency — see [ADR 0001](../adr/0001-vendor-cordis-as-source.md) for why. [vendor/README.md](../../vendor/README.md) covers *updating* an already-vendored package; this guide is the file-by-file checklist for adding a **new** one. (Verified against the existing vendored set; if it drifts, fix it here.) + +## 1. Copy the source in + +``` +vendor// + package.json # from upstream; set "private": true, keep name/exports/type + tsconfig.json # extends ../../tsconfig.base.json (see shape below) + src/ # the upstream src/ verbatim + README.md LICENSE # if upstream ships them +``` + +`tsconfig.json` mirrors the other vendored packages — `rootDir: src`, `outDir: lib`, the strictness relaxations upstream code needs, and a `references` entry for every other vendored package it imports: + +```jsonc +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", "outDir": "lib", + "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, + "noImplicitOverride": false, "noUnusedLocals": false, "noUnusedParameters": false + }, + "include": ["src"], + "references": [{ "path": "../cordis" }, { "path": "../cosmokit" }] +} +``` + +`package.json` invariants: `"private": true` (vendored packages are never published), keep upstream's `name`/`version`/`exports`/`type`, and list its cordis deps in `peerDependencies` (matching the upstream manifest). Transitive upstream deps must themselves be vendored or already present — vendoring one package often means vendoring its dependency tree (e.g. `@cordisjs/plugin-http` pulls `@cordisjs/fetch-file`). + +## 2. Register it in the root configs + +| File | Change | +|---|---| +| `tsconfig.base.json` | add `"": ["./vendor//src"]` to `paths` | +| `tsconfig.typecheck.json` | add `"": ["./vendor//lib"]` — this file points at built declarations, not src. If the package's `types` entry isn't `lib/index.d.ts`, point at that built file instead (e.g. `logger-console` maps to `./vendor/logger-console/lib/shared`, matching its `"types": "lib/shared.d.ts"`). | +| `tsconfig.build.json` | add `{ "path": "./vendor/" }` to `references` (before the `packages/*` entries) | +| `vendor/README.md` | add a manifest table row (dir, npm name, version, upstream repo, commit SHA) and log any local modifications | +| `scripts/publint-all.ts` | only if the vendored package is itself published from here (vendored deps normally are not — skip) | + +Covered automatically by globs — no edits needed: root `package.json` workspaces (`vendor/*`), `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`. A per-package `vendor//tsdown.config.ts` is needed ONLY if the build shape diverges from the root default (dual ESM/CJS or multiple entries — see `vendor/schemastery` and `vendor/logger-console`). + +## 3. Mind the manifest guard + +`scripts/check-vendor-manifest.sh` (a pre-commit hook) fails if anything under `vendor/*/src` is staged without `vendor/README.md` also staged. Stage the manifest update alongside the source so the commit passes. + +## 4. Verify + +```sh +yarn install # registers the workspace +yarn typecheck # the base→lib path split means: run once after a fresh add +yarn build && yarn test && yarn constraints +``` + +Note the `tsconfig` two-map split (called out in [AGENTS.md](../../AGENTS.md) § Secrets/.env): `lint`'s type-aware rules resolve vendored packages through their built `lib/` declarations, so run `yarn typecheck` (which builds them) once after adding the package or lint reports unresolved-type errors. diff --git a/docs/cookbook/extension-cookbook.md b/docs/cookbook/extension-cookbook.md new file mode 100644 index 0000000000..fcf082f690 --- /dev/null +++ b/docs/cookbook/extension-cookbook.md @@ -0,0 +1,48 @@ +# Cookbook: extension plugin shapes + +The three plugin shapes you write against the harness extension surface, as illustrative snippets (elided imports and helper stubs — not copy-paste-complete). For the full step-by-step guides see [adding a package](./adding-a-package.md), [adding a tool](./adding-a-tool.md), and [adding an LLM adapter](./adding-an-llm-adapter.md); for the seams these hook into see [docs/architecture.md](../architecture.md). + +## A tool plugin + +A tool registers on `ctx.tools`. The annotated `defineTool` example (typed `execute` args, result shaping, the `run_in_background` pattern) lives in [adding-a-tool.md](./adding-a-tool.md) — that guide is the source of truth for the tool shape. Raw JSON-Schema `ToolDefinition`s are also accepted by `ctx.tools.register()` directly (that is how MCP-sourced tools arrive); `defineTool` is the typed sugar for first-party tools. + +## A hook plugin (permission gate) + +A hook wraps the `tools/execute` waterfall to veto or rewrite a call — the seam where sandbox, permission, and plan-mode plugins live. + +```ts +export const name = 'permission-gate' + +export function apply(ctx: Context) { + ctx.on('tools/execute', async (exec, next) => { + if (!(await isAllowed(exec))) { + return { + callId: exec.callId, + content: [{ type: 'text', text: 'Denied by policy.' }], + isError: true, + } + } + return next() + }) +} +``` + +## A UI plugin + +A UI plugin consumes `agent/stream-chunk` and session events for rendering, and drives input back in via `agent.send()` / `agent.steer()`. + +```ts +export const name = 'my-ui' +export const inject = ['agents'] + +export function apply(ctx: Context) { + ctx.on('agent/stream-chunk', (agent, turn, step, chunk) => { + if (chunk.type === 'text-delta') render(chunk.text) + }) + onUserInput(text => ctx.agents.get('main')?.send([{ type: 'text', text }])) +} +``` + +## Runnable wirings + +Two complete examples load their plugin trees from `cordis.yml` with HMR: [`examples/echo-agent`](../../examples/echo-agent) (mock model + echo tool — the all-mock skeleton check, `yarn demo:echo`) and [`examples/coding-agent`](../../examples/coding-agent) (DeepSeek V4 + the bash tool suite — the real thing, `yarn demo:coding`). diff --git a/examples/README.md b/examples/README.md index 9c9b091aed..592dc66930 100644 --- a/examples/README.md +++ b/examples/README.md @@ -12,6 +12,12 @@ A mock model + echo tool + stdio UI + JSONL persistence demo. Demonstrates: - Persisting session events to JSONL via the `session/event` + `session/flush` pattern - A minimal stdio UI consuming `agent/stream-chunk` and session events -Run with: `yarn demo` (or `node --expose-internals --import tsx examples/echo-agent/start.ts`) +Run with: `yarn demo:echo` When prompted, type "echo " to trigger a tool call round-trip. + +## coding-agent + +The real thing: DeepSeek V4 + the bash tool suite + stdio chat + JSONL persistence, wired from `cordis.yml`. Where echo-agent proves the skeleton with mocks, this is a usable coding assistant. + +Run with: `yarn demo:coding` (needs `DEEPSEEK_API_KEY` in the environment or a gitignored repo-root `.env`). See [coding-agent/README.md](coding-agent/README.md) for details. diff --git a/examples/echo-agent/README.md b/examples/echo-agent/README.md index 9bde765ea3..87a52f2dd9 100644 --- a/examples/echo-agent/README.md +++ b/examples/echo-agent/README.md @@ -23,7 +23,7 @@ Runnable demo: stdin chat with a scripted mock model and an echo tool. ## Run ```sh -yarn demo +yarn demo:echo # or: node --expose-internals --import tsx examples/echo-agent/start.ts ``` diff --git a/package.json b/package.json index 6d389e0ae5..f4c3279082 100644 --- a/package.json +++ b/package.json @@ -22,7 +22,7 @@ "knip": "knip", "publint": "tsx scripts/publint-all.ts", "hygiene": "yarn knip && yarn publint && yarn constraints", - "demo": "node --expose-internals --import tsx examples/echo-agent/start.ts", + "demo:echo": "node --expose-internals --import tsx examples/echo-agent/start.ts", "demo:coding": "node --expose-internals --import tsx examples/coding-agent/start.ts", "postinstall": "lefthook install" }, diff --git a/packages/README.md b/packages/README.md index 1a85d7c6f1..e61d3a976a 100644 --- a/packages/README.md +++ b/packages/README.md @@ -6,14 +6,19 @@ Harness packages, all under the `@deepseek-ai/dsh-*` scope. Each package is a Co ``` dsh-llm (no harness deps — pure vocabulary) +dsh-bash (no harness deps — abstract executor seam) dsh-session ← dsh-llm dsh-system-prompt ← dsh-llm dsh-agent ← dsh-llm, dsh-session dsh-tools ← dsh-llm, dsh-system-prompt, dsh-agent +dsh-bash-local ← dsh-bash (BashExecutor impl) +dsh-tool-bash ← dsh-bash, dsh-tools (bash tool schemas) +dsh-llm-deepseek ← dsh-llm (DeepSeek adapter) +dsh-llm-pi-ai ← dsh-llm (pi-ai-backed adapter) dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent ``` -The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. +The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [ADR 0009](../docs/adr/0009-capability-seams.md)). ## What goes where @@ -25,6 +30,11 @@ The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-l | `tools/` | Tool registry + `tools/execute` waterfall | `ctx.tools` | | `agent/` | Agent interface, registry, `agent/*` event vocabulary | `ctx.agents` | | `agent-loop/` | THE concrete plugin: `LoopAgent` + the loop driver | `ctx.agentLoop` | +| `bash/` | Abstract bash executor seam (interface + vocabulary) | `ctx.bash` | +| `bash-local/` | Local-subprocess `BashExecutor` implementation | (registers `ctx.bash`) | +| `tool-bash/` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) | +| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | +| `llm-pi-ai/` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) | Each package has its own `README.md` with purpose, service API, events, extension points, and deliberate non-goals (TODOs). diff --git a/packages/llm/README.md b/packages/llm/README.md index e8138776d9..22a85bdf0d 100644 --- a/packages/llm/README.md +++ b/packages/llm/README.md @@ -38,9 +38,8 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta - `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`. - `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. Used by the agent loop (raw chunks for replay + assembled for history) and by `streamBlocks()`/`generate()`. -- `LlmError` — typed error with a `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`). +- `LlmError` — typed error with a `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) and an optional numeric `status` when the failure came from a non-2xx provider response. -### What is NOT here (TODO) +### Real adapters -- **DeepSeek V4 adapter** — the first real adapter lands in a later phase. -- **Streaming protocol review** — the chunk protocol has `TODO(review)` markers and needs careful review before the first real adapter (DeepSeek V4 wire format, partial JSON arguments, interleaved reasoning signatures, ...). +Two adapters implement `LlmAdapter` against this vocabulary, deliberately built on different internals to keep the contract honest (see [ADR 0010](../../docs/adr/0010-twin-llm-adapters.md)): [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) (hand-rolled fetch/SSE) and [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) (via `@earendil-works/pi-ai`). The pair pinned down the `StreamChunk` conventions now documented in `types.ts` (usage before finish, raw-string tool arguments, the two sanctioned error paths). diff --git a/vendor/README.md b/vendor/README.md index cd3d234d80..8ab3498053 100644 --- a/vendor/README.md +++ b/vendor/README.md @@ -1,14 +1,10 @@ # Vendored Packages -This directory contains source-vendored copies of the Cordis framework and its -foundation libraries. They are copied into this monorepo instead of being -depended on via npm, so that the harness fully owns its framework layer -(auditable, patchable, pinned). +This directory contains source-vendored copies of the Cordis framework and its foundation libraries. They are copied into this monorepo instead of being depended on via npm, so that the harness fully owns its framework layer (auditable, patchable, pinned). -All vendored packages keep their **original npm names** (they are resolved -through Yarn workspaces) and are marked `private: true` — they are never -published from this repo. Upstream MIT `LICENSE` files are preserved in each -package directory. +All vendored packages keep their **original npm names** (they are resolved through Yarn workspaces) and are marked `private: true` — they are never published from this repo. Upstream MIT `LICENSE` files are preserved in each package directory. + +This file covers the manifest, the local-modification log, and the procedure for **updating** an existing vendored package. To **add a new** one, see the cookbook guide: [docs/cookbook/adding-a-vendored-package.md](../docs/cookbook/adding-a-vendored-package.md). ## Manifest @@ -26,44 +22,25 @@ Upstream workspace: `cordis-workspace` (local checkout: `~/repos/cordis-workspac | `hmr/` | `@cordisjs/plugin-hmr` | 1.0.15 | https://github.com/deepseek-harness/cordis (`packages/hmr`) | `abb0a307cb1d3b0947f455d590cf5ba922d4caa4` | | `logger-console/` | `@cordisjs/plugin-logger-console` | 1.0.0 | https://github.com/deepseek-harness/cordis (`packages/logger-console`) | `abb0a307cb1d3b0947f455d590cf5ba922d4caa4` | -Third-party dependencies of the vendored packages stay on npm: -`@standard-schema/spec`, `js-yaml`, `chokidar`, `picomatch`, -`@babel/code-frame`, `supports-color`. +Third-party dependencies of the vendored packages stay on npm: `@standard-schema/spec`, `js-yaml`, `chokidar`, `picomatch`, `@babel/code-frame`, `supports-color`. -Intentionally **not** vendored (verified unused by this set): `reggol`, -`@cordisjs/utils`, `@cordisjs/element`, `@cordisjs/unyaml` (dev-time YAML -import hook only). +Intentionally **not** vendored (verified unused by this set): `reggol`, `@cordisjs/utils`, `@cordisjs/element`, `@cordisjs/unyaml` (dev-time YAML import hook only). ## Local modifications Keep this log exhaustive — every divergence from upstream must be listed. -1. **`hmr/src/index.ts`**: removed the `./locales/en-US.yml` / - `./locales/zh-CN.yml` imports, the `.i18n({...})` call on the `Config` - schema, and the `src/locales/` directory. Rationale: those imports require - a runtime YAML loader hook (`@cordisjs/unyaml`) that we do not vendor; - the i18n texts only localize config descriptions. -2. **All `package.json` files**: regenerated — added `private: true`, added - `src` to `files` and a `./src/*` export where missing, removed upstream - `devDependencies`/`scripts`/`repository` fields. Dependency and - peer-dependency ranges preserved. -3. **All `tsconfig.json` files**: regenerated to extend the repo-root - `tsconfig.base.json` and declare project references. -4. **`schemastery/tsdown.config.ts` and `logger-console/tsdown.config.ts`**: - ours, not upstream files — per-package build-shape overrides (dual - ESM+CJS output; separate node/browser entries) for the repo-root tsdown - build. Like the regenerated tsconfigs, they are not part of the upstream - sync surface. +1. **`hmr/src/index.ts`**: removed the `./locales/en-US.yml` / `./locales/zh-CN.yml` imports, the `.i18n({...})` call on the `Config` schema, and the `src/locales/` directory. Rationale: those imports require a runtime YAML loader hook (`@cordisjs/unyaml`) that we do not vendor; the i18n texts only localize config descriptions. +2. **All `package.json` files**: regenerated — added `private: true`, added `src` to `files` and a `./src/*` export where missing, removed upstream `devDependencies`/`scripts`/`repository` fields. Dependency and peer-dependency ranges preserved. +3. **All `tsconfig.json` files**: regenerated to extend the repo-root `tsconfig.base.json` and declare project references. +4. **`schemastery/tsdown.config.ts` and `logger-console/tsdown.config.ts`**: ours, not upstream files — per-package build-shape overrides (dual ESM+CJS output; separate node/browser entries) for the repo-root tsdown build. Like the regenerated tsconfigs, they are not part of the upstream sync surface. ## Sync procedure To update a vendored package from upstream: -1. In the upstream workspace, note `git rev-parse HEAD` of the relevant - submodule. -2. Copy the package's `src/` (and `bin.js`, `README.md`, `LICENSE` if changed) - over the vendored directory. -3. Re-apply the local modifications listed above (or drop them if upstream - made them unnecessary — update the log either way). +1. In the upstream workspace, note `git rev-parse HEAD` of the relevant submodule. +2. Copy the package's `src/` (and `bin.js`, `README.md`, `LICENSE` if changed) over the vendored directory. +3. Re-apply the local modifications listed above (or drop them if upstream made them unnecessary — update the log either way). 4. Update the version and commit hash in the manifest table. 5. Run `yarn install && yarn test && yarn build` at the repo root.