diff --git a/AGENTS.md b/AGENTS.md index db20d3f7e4..f119a3c121 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,6 +73,15 @@ packages/ Harness packages, grouped by role at packages///. bash/ abstract bash executor seam (ctx.bash) — interface only bash-local/ local-subprocess BashExecutor implementation tool-bash/ model-facing bash/bash_output/bash_kill tool schemas + compact/ compaction capability family + compact/ abstract compaction seam (ctx.compact); backend + tool deferred + subagent/ subagent capability family + subagent/ provider-registry seam (ctx.subagents) + subagent-inprocess/ shared in-process run driver (library, registers nothing) + subagent-spawn/ in-process fresh-child backend + subagent-fork/ in-process backend seeded from the parent's completed-turn prefix + subagent-acp/ out-of-process child over ACP + tool-subagent/ model-facing delegation tool over ctx.subagents todo/ todo/planning capability family tool-todo/ model-facing todo_write tool: writes the whole task list to the session log (todo/write), rendered as a stdio checklist / @@ -95,6 +104,7 @@ packages/ Harness packages, grouped by role at packages///. feeds stdin lines to the agent (shared by the demos) llm-replay/ record/replay adapter: short-circuits llm/stream from a recorded session JSONL (keyless snapshot tests) + subagent-mock/ scripted SubagentProvider for deterministic seam/tool tests util/ low-level zero-dependency utilities shared across groups brand/ type-only Branded nominal-typing primitive (no runtime code, no harness deps; owns the brand for cross-boundary ids) @@ -181,7 +191,7 @@ pnpm run demo:acp # run examples/acp-agent — the coding agent as an ACP CI is the backstop, not the first place a gate runs. Before you open a non-draft PR or move one from draft to ready, run the same gates CI runs, on your own tree, and confirm they pass — do not lean on CI (or a Codex pass) to discover a red gate you could have caught locally. The CI-equivalent local run is: ```sh -pnpm run typecheck && pnpm run lint && pnpm run test:coverage && pnpm run test:snapshot && pnpm run doc-sync && pnpm run hygiene && pnpm run build +pnpm run typecheck && pnpm run lint && pnpm run test:coverage && pnpm run test:snapshot && pnpm run doc-sync && pnpm run verify-module-graph && pnpm run build && pnpm run hygiene ``` **`pnpm run test:coverage`, NOT `pnpm run test`, is the gating test command.** `pnpm run test` runs `vitest run` with no coverage; CI's node job runs `test:coverage`, which enforces a **per-file 100%** threshold on `packages/*/*/src`. A suite that is green under `test` can still fail CI on an uncovered line — and that uncovered line is often *dead code* the 100% gate is correctly flagging for deletion (see [§ Defensive patterns](#defensive-patterns-hard-won) "Line coverage is not behavior coverage"), not a missing test to bolt on. `hygiene` (knip + publint + workspace constraints + NodeNext types) and `test:snapshot` (keyless ACP replay) are likewise CI gates that `test` alone does not cover. When you rely on a Codex convergence pass for sign-off, check WHICH commands it ran: a pass that ran `test` but not `test:coverage`/`hygiene`/`doc-sync` has not exercised those gates. @@ -256,9 +266,9 @@ Verbose documentation is fine **as long as docs and code stay strictly in sync** **Document the CURRENT state — the "what" and "why" — never the PROCESS or HISTORY of how it got there.** A comment, JSDoc, or doc paragraph describes what the code *is* and why it is that way, as if it had always been so. Do NOT narrate the change that produced it: no "previously X, now Y", "changed from", "used to", "this replaces", "the old map", "renamed", "moved here", "as of this PR", or "(was …)". **In particular, NEVER name the change unit a reader cannot see — the PR, commit, or stack position that introduced the code — in a comment, JSDoc, OR a test name/description.** A `// (PR D's per-agent teardown)` aside, a `* Tests for the cancel primitive (PR C).` module doc, or an `it('… identity no longer matters')` title that only makes sense relative to a prior design are all the same violation: the reader of the current tree has no "PR D" or "old design" to anchor against, and the reference rots the moment the stack merges. Name the *mechanism* (`the session's AgentHandle teardown`), not the PR. Such phrasing rots the instant the next change lands, and a reader of the current code does not need the diff narrated in prose — that belongs in the commit message, the PR description, or an RFC (the durable home for "why we moved away from X"). Write "the owner token lives on the task in the executor" — not "ownership *now* lives on the executor instead of a plugin-local map". When a contrast genuinely aids understanding (a non-obvious choice between live alternatives), frame it against the alternative as a standing fact ("stored on the executor, NOT the tool plugin, so it survives an HMR reload"), not against the codebase's past. The same rule governs review-fix commits: the *commit message* records what the review caught; the *code comment* it touches states only the resulting truth. RFCs (`docs/rfc/`, grouped into `proposed/` / `implemented/` / `rejected/`) record the *why* behind choices a future reader would otherwise re-litigate (the vendoring policy, event-sourcing, the schema DSL are the existing examples). A PR that introduces such a decision — a new third-party runtime dependency over the vendoring default, a cross-package contract, a security/isolation model, a deviation from a documented architecture rule — writes the RFC in `implemented/` **in the same PR**, and links it from the relevant code. A proposal for future work not yet built goes in `proposed/`. A PR whose changes are mechanical, self-evident, or already covered by an existing RFC needs none — do not manufacture an RFC for a routine change. When unsure, the test is: would a competent maintainer six months from now ask "why was it done this way?" and be unable to answer from the code alone? If yes, write it. See [docs/rfc/README.md](docs/rfc/README.md) for the naming scheme and [docs/AGENTS.md](docs/AGENTS.md) for the cross-link convention. -**Markdown is not hard-wrapped**: write one line per paragraph and let the editor soft-wrap. Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. This applies to prose only: leave fenced code blocks, tables, and list structure intact (a wrapped list item folds to one line per bullet). Code comments / JSDoc are exempt — they stay under the linter's column limit. `pnpm run verify-md-wrap` (part of `doc-sync`) enforces this across `README.md`, `docs/**/*.md`, `packages/*/*.md`, and `AGENTS.md` / `packages/AGENTS.md`; `pnpm run verify-md-links` (also part of `doc-sync`) checks that every relative cross-link in those files resolves. +**Markdown is not hard-wrapped**: write one line per paragraph and let the editor soft-wrap. Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. This applies to prose only: leave fenced code blocks, tables, and list structure intact (a wrapped list item folds to one line per bullet). Code comments / JSDoc are exempt — they stay under the linter's column limit. `pnpm run verify-md-wrap` (part of `doc-sync`) enforces this across `README.md`, `docs/**/*.md`, `packages/*/*.md`, and `AGENTS.md` / `packages/AGENTS.md`; `pnpm run verify-md-links` (also part of `doc-sync`) checks that every relative cross-link in those files plus `examples/**/*.md` and `.agents/skills/**/*.md` resolves. -**Editing these instructions**: `AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it (at the repo root and in `packages/`). Always edit `AGENTS.md` — never write through the `CLAUDE.md` symlink or replace it with a regular file. +**Editing these instructions**: `AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it (at the repo root and in `packages/` / `examples/`). Always edit `AGENTS.md` — never write through the `CLAUDE.md` symlink or replace it with a regular file. ## Vendoring Policy diff --git a/docs/architecture.md b/docs/architecture.md index 315715c828..158050fddb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -24,6 +24,7 @@ For a catalog of the **data structures** this architecture moves around — the │ @deepseek-ai/dsh-agent-loop (the ONE concrete plugin) │ │ @deepseek-ai/dsh-bash-local (bash impl) │ │ @deepseek-ai/dsh-tool-bash (bash tool schemas) │ +│ @deepseek-ai/dsh-subagent-* (subagent providers) │ │ @deepseek-ai/dsh-session-persistence-jsonl (persistence impl)│ ├─────────────────────────────────────────────────────────────┤ │ @deepseek-ai/dsh-agent (vocabulary + registry) │ @@ -33,6 +34,8 @@ For a catalog of the **data structures** this architecture moves around — the │ @deepseek-ai/dsh-session-persistence (persistence seam) │ │ @deepseek-ai/dsh-llm (abstract model service) │ │ @deepseek-ai/dsh-bash (abstract bash executor) │ +│ @deepseek-ai/dsh-compact (abstract compaction seam) │ +│ @deepseek-ai/dsh-subagent (provider registry seam) │ ├─────────────────────────────────────────────────────────────┤ │ vendor/: cordis, loader, include, group, timer, hmr, │ │ logger-console, cosmokit, schemastery │ @@ -54,6 +57,7 @@ Dependency rule: **extension** plugins depend on interface packages, never on `d | `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `ReactLoopAgent`s and drives their loops | | `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks | | `ctx.compact` | `CompactService` (abstract) | dsh-compact | compaction seam: decide when history is too large, summarize an older range into a single surface node | +| `ctx.subagents` | `SubagentService` | dsh-subagent | named provider registry for delegating a task to child agents | All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically. @@ -86,11 +90,11 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source - `user/message` → user message - `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation; an empty-content `assistant/message`, which exists only to host a max-tokens step's `usage`, is skipped too) - `tool/result` → user message carrying a `tool-result` block -- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: the real adapters now exist (the original precondition); the envelope still wants a deliberate review against live model behavior (`TODO(review)` in dsh-session). +- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. Live-adapter review has validated the tagged-envelope rendering against current DeepSeek behavior; provider-specific mismatches belong in that adapter. Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`. -**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic. +**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage, seed boundary) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data, source_event_seqs, surface_op)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic. ## Prompt assembly (dsh-system-prompt) @@ -202,7 +206,7 @@ Every MVP feature (including the TODO-marked ones), with the mechanism that impl | Tool sandbox (landlock / sandbox-exec) | wrap `tools/execute`, or implement a sandboxing `BashExecutor` (the dsh-bash seam) | | Permission system / AskUserQuestion | wrap `tools/execute` (veto or ask); register an ask tool | | Plan mode | wrap `tools/execute` (deny writes) + `agent/request` (inject mode prompt) | -| Sub-agents (spawn / fork / steer) | TODO seam on `AgentLoop.create()`; fork = seed Session with parent events; `steer()` on the child handle | +| Sub-agent delegation | Implemented as the `ctx.subagents` provider-registry seam: `dsh-subagent-spawn` starts a fresh in-process child, `dsh-subagent-fork` seeds a child from the parent's completed-turn prefix, `dsh-subagent-acp` drives an out-of-process child over ACP, and `dsh-tool-subagent` exposes one configured provider to the model | | MCP | one plugin per server: discover tools → `ctx.tools.register()` | | Skills | section + tool registration; `inject()` skill content on invocation | | Memory | section provider + tool | @@ -220,7 +224,7 @@ Code skeletons for the three plugin shapes (tool, hook/permission-gate, UI) and Tracked here deliberately — each is designed-for but not implemented: -- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent channels beyond `send`/`steer`/events. +- **Inter-agent channels beyond delegation** (shared state, streaming child output, background/poll semantics) remain out of scope for the current `ctx.subagents` seam. - **Compaction implementation** (auto thresholds, summarization prompts) on the `agent/request` seam, with its session-event types added by declaration merging. - **Parallel tool execution** (concurrency-safety hints on ToolDefinition). - **Session branching/tree** (pi-style entry tree) if needed beyond seed-based forking. diff --git a/docs/core-data-structures/persistence.md b/docs/core-data-structures/persistence.md index 8d8f032514..9b902a1c51 100644 --- a/docs/core-data-structures/persistence.md +++ b/docs/core-data-structures/persistence.md @@ -78,6 +78,6 @@ Replay/fork is therefore `ctx.sessions.create(id, { seed: seedEvents })`; resumi Both implement the same abstract `SessionPersistence` (create/append/load/list over `SessionEvent`) and pass `runPersistenceContract`, proving the seam is genuinely backend-agnostic: - **[dsh-session-persistence-jsonl](../../packages/session-persistence/session-persistence-jsonl)** — an append-only JSONL log per session with crash-safe atomic writes, the interrupted-turn crash recovery above, and a read/replay path. -- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data)` maps 1:1 onto the event, so there is no parallel persisted schema to keep in sync. +- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data, source_event_seqs, surface_op)` maps 1:1 onto the event, including optional surface metadata, so there is no parallel persisted schema to keep in sync. Multiple backends sharing one on-disk session coordinate writes through the [shared persistence write-coordinator](../rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md). diff --git a/docs/development.md b/docs/development.md index 99f67e671d..0918b1f31b 100644 --- a/docs/development.md +++ b/docs/development.md @@ -78,6 +78,7 @@ The GitHub workflow runs these gates on each pull request: - `pnpm run build` - `pnpm run hygiene` - an echo-agent smoke test that checks the demo's tool call, tool result, and JSONL output +- built-bin smoke tests that run the published `lib/bin.js` entrypoints under plain `node` `pnpm run hygiene` is the local shorthand for `pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types`; CI also runs `pnpm run constraints` as an earlier fail-fast step, then runs the full hygiene script after `pnpm run build`. @@ -121,6 +122,12 @@ The coding-agent demo uses the real DeepSeek adapter and needs `DEEPSEEK_API_KEY pnpm run demo:coding ``` +The ACP server demo exposes the same coding agent over JSON-RPC stdio and also needs `DEEPSEEK_API_KEY`: + +```sh +pnpm run demo:acp +``` + ## TODO markers Use one of three comment tags to flag known issues in the code, ordered by urgency: diff --git a/examples/acp-agent/README.md b/examples/acp-agent/README.md index 1df36715ec..58ba92cd0e 100644 --- a/examples/acp-agent/README.md +++ b/examples/acp-agent/README.md @@ -6,7 +6,7 @@ The DeepSeek Harness coding agent exposed as an **Agent Client Protocol (ACP)** pnpm run demo:acp # needs DEEPSEEK_API_KEY (repo-root .env or env) ``` -This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand) plus the two swappable backends (`llm-deepseek`, `bash-local`). The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC. +This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand), the swappable DeepSeek and bash backends, and the optional model-facing `subagent`/`subagent_fork`/`todo_write` tool entries. The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC. ## stdout is the protocol @@ -32,7 +32,7 @@ The editor sets each session's `cwd` to the project it opens; the agent's bash t ## Snapshot tests (record-once / replay-deterministic) -This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `@deepseek-ai/dsh-llm-replay`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`". The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md](../../docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md) for the full design. +This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `@deepseek-ai/dsh-llm-replay`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`". The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [the ACP snapshot tests RFC](../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md) for the full design. ## MVP limitations diff --git a/examples/acp-agent/cordis.yml b/examples/acp-agent/cordis.yml index 96071ab564..1e9060ecae 100644 --- a/examples/acp-agent/cordis.yml +++ b/examples/acp-agent/cordis.yml @@ -1,9 +1,8 @@ # The acp-agent plugin tree: the ACP server. Also the snapshot RECORD config # (the dsh-acp-agent bin selects it for DSH_SNAPSHOT=record): a real llm-deepseek -# run whose persisted log the snapshot harness harvests. Just the two swappable -# backends — the DeepSeek adapter and the local bash executor — plus the ACP -# server app (@deepseek-ai/dsh-acp-agent), which bundles the agent-core spine, -# JSONL persistence, and the ACP bridge. +# run whose persisted log the snapshot harness harvests. The swappable DeepSeek +# adapter and local bash executor, the ACP server app (@deepseek-ai/dsh-acp-agent), +# and the optional model-facing subagent/todo tools loaded below. # # CRITICAL: this tree loads NO stdout logger and NO hmr — stdout is reserved for # the ACP JSON-RPC protocol (see packages/ui/acp). That guarantee is now a diff --git a/examples/coding-agent/README.md b/examples/coding-agent/README.md index 1c2acbcb86..dc4c22d8a9 100644 --- a/examples/coding-agent/README.md +++ b/examples/coding-agent/README.md @@ -1,7 +1,6 @@ # coding-agent -The first REAL agent wiring: DeepSeek V4 + the bash tool suite + stdio chat -+ JSONL persistence, loaded from `cordis.yml`. Where echo-agent proves the skeleton with mocks, this example is a usable coding assistant. +The real stdio coding-agent wiring: DeepSeek V4 + the bash tool suite + subagent delegation + `todo_write` + stdio chat + JSONL persistence, loaded from `cordis.yml`. Where echo-agent proves the skeleton with mocks, this example is a usable coding assistant. ## Run it @@ -34,7 +33,7 @@ The id is wired through `cordis.yml` (`resumeSessionId: !!js process.env.RESUME_ ## What each leaf entry demonstrates -This example is a thin leaf `cordis.yml`: it picks the swappable backends and loads one app package. The spine (sessions, system-prompt, tools, agents, invariants, `agent-loop`) and the front-door cluster (console logger, JSONL persistence, readline UI, the pre-created `main` agent) all live inside the [`@deepseek-ai/dsh-stdio-agent`](../../packages/ui/stdio-agent) app and the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) bundle it loads — so the leaf has only four entries: +This example is a thin leaf `cordis.yml`: it picks the swappable backends, loads one app package, and adds product tools that are intentionally outside the shared spine. The spine (sessions, system-prompt, tools, agents, invariants, `agent-loop`) and the front-door cluster (console logger, JSONL persistence, readline UI, the pre-created `main` agent) live inside the [`@deepseek-ai/dsh-stdio-agent`](../../packages/ui/stdio-agent) app and the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) bundle it loads; the leaf wires the backends and model-facing optional tools: | Entry | Demonstrates | |---|---| @@ -42,11 +41,15 @@ This example is a thin leaf `cordis.yml`: it picks the swappable backends and lo | `llm-deepseek` | real `LlmAdapter` via config (`!!js process.env.…` secrets); swap one line to `@deepseek-ai/dsh-llm-pi-ai` for the library-backed twin | | `bash` (`dsh-bash-local`) | the executor implementation — the swappable half of the bash seam. The model-facing `bash`/`bash_output`/`bash_kill` tool schemas (`tool-bash`) come from `agent-core`, so only the executor is a leaf choice | | `stdio-agent` (`@deepseek-ai/dsh-stdio-agent`) | the app bundle: the agent-core spine + console logger + JSONL persistence + readline UI + a pre-created `main` agent. Its config carries the model, system prompt, `persistenceRoot` (`./.sessions`), and `resumeSessionId` — so persistence and the agent are configured here, not wired as separate leaf plugins | +| `subagent`, `subagent-spawn`, `subagent-fork` | the subagent provider registry plus the two in-process backends: a fresh child and a child seeded with the parent's completed-turn prefix | +| `tool-subagent`, `tool-subagent-fork` | two model-facing `dsh-tool-subagent` loads, each bound to a different provider and exposed under a distinct tool name (`subagent`, `subagent_fork`) | +| `tool-todo` | the model-facing `todo_write` tool; writes the whole task list to the session log and renders as a checklist in stdio | ## End-to-end tests (`pnpm run test:e2e`, key-gated) - `tests/full-loop.e2e.ts` — the canary: real model runs `echo e2e-ok` through the real bash tool; asserts `tool/call`/`tool/result` session events and the final answer. - `tests/coding-task.e2e.ts` — the swebench-style smoke: a temp dir holds `add.js` (with `a - b` where `a + b` belongs) and a failing `add.test.js`; the agent must fix the bug and verify. The test re-runs `node add.test.js` ITSELF and inspects the files — agent claims are not trusted. - `tests/resume.e2e.ts` — durable continuity across processes: run 1 tells the real model a secret code and persists the turn to a temp JSONL root, then the whole context is disposed; run 2 is a fresh context over the same root that RESUMES the session id and asks the model to recall the code. The recall can only come from the rehydrated log. +- `tests/todo-write.e2e.ts` — a real model drives the real `todo_write` tool and the test verifies the resulting `todo/write` session event. -Both self-skip without `DEEPSEEK_API_KEY`. +These self-skip without `DEEPSEEK_API_KEY`. diff --git a/packages/README.md b/packages/README.md index 3997fd5190..28e4dcbe83 100644 --- a/packages/README.md +++ b/packages/README.md @@ -36,17 +36,18 @@ dsh-bash-local ← dsh-bash (BashExecutor impl) dsh-tool-bash ← dsh-bash, dsh-tools (bash tool schemas) dsh-llm-deepseek ← dsh-llm (DeepSeek adapter) dsh-llm-pi-ai ← dsh-llm (pi-ai-backed adapter) -dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent +dsh-agent-loop ← dsh-llm, dsh-session, dsh-session-persistence, dsh-system-prompt, dsh-tools, dsh-agent dsh-invariants ← dsh-llm, dsh-session, dsh-agent (dev-mode contract checks) -dsh-acp ← dsh-agent, dsh-llm, dsh-session, dsh-session-persistence (ACP JSON-RPC bridge) +dsh-acp ← dsh-agent, dsh-llm, dsh-session, dsh-session-persistence, dsh-tools (ACP JSON-RPC bridge) dsh-ui-stdio ← dsh-agent, dsh-llm, dsh-session (stdio readline UI plugin) dsh-llm-replay ← dsh-llm, dsh-session (record/replay adapter for keyless snapshot tests) dsh-subagent ← dsh-agent, dsh-llm, dsh-tools (abstract subagent provider-registry seam) -dsh-subagent-mock ← dsh-subagent (scripted provider for tests) -dsh-subagent-spawn ← dsh-subagent, dsh-agent, dsh-session, dsh-llm (in-process fresh child + shared run driver) -dsh-subagent-fork ← dsh-subagent-spawn, dsh-agent, dsh-session (in-process child seeded from parent log) +dsh-subagent-inprocess ← dsh-subagent, dsh-agent, dsh-session, dsh-llm (shared in-process run driver) +dsh-subagent-mock ← dsh-subagent, dsh-agent, dsh-llm (scripted provider for tests) +dsh-subagent-spawn ← dsh-subagent, dsh-subagent-inprocess (in-process fresh child backend) +dsh-subagent-fork ← dsh-subagent, dsh-subagent-inprocess, dsh-agent, dsh-session (in-process child seeded from parent log) dsh-subagent-acp ← dsh-subagent, dsh-agent, dsh-llm, @agentclientprotocol/sdk (out-of-process child over ACP) -dsh-tool-subagent ← dsh-subagent, dsh-tools, dsh-agent (model-facing delegation tool) +dsh-tool-subagent ← dsh-subagent, dsh-tools, dsh-agent, dsh-llm (model-facing delegation tool) dsh-tool-todo ← dsh-tools, dsh-agent, dsh-session (model-facing todo_write tool; whole list on the session log) dsh-agent-core ← timer, dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent, dsh-invariants, dsh-tool-bash, dsh-agent-loop (the providerless spine, as one bundle plugin) dsh-stdio-agent ← dsh-agent-core, dsh-ui-stdio, dsh-session-persistence-jsonl, dsh-agent, dsh-session (stdio chat APP + bin) @@ -82,7 +83,8 @@ The rule: **extension** plugins depend on interfaces, never on the concrete loop | `ui-stdio/` | `support` | Minimal stdio (readline) UI plugin: renders `agent/*` events, feeds stdin lines to the agent | (drives `ctx.agents`) | | `llm-replay/` | `support` | Record/replay adapter: short-circuits `llm/stream` with chunks from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) | | `subagent/` | `subagent` | Abstract subagent seam: named-provider registry for delegating to child agents | `ctx.subagents` | -| `subagent-spawn/` | `subagent` | In-process backend: a fresh child agent (+ the shared in-process run driver) | (registers on `ctx.subagents`) | +| `subagent-inprocess/` | `subagent` | Shared in-process subagent run driver used by spawn/fork; pure library, registers nothing | (none) | +| `subagent-spawn/` | `subagent` | In-process backend: a fresh child agent | (registers on `ctx.subagents`) | | `subagent-fork/` | `subagent` | In-process backend: a child agent seeded with the parent's completed-turn prefix | (registers on `ctx.subagents`) | | `subagent-acp/` | `subagent` | Out-of-process backend: a child agent in a spawned subprocess, driven over the Agent Client Protocol | (registers on `ctx.subagents`) | | `subagent-mock/` | `support` | Scripted `SubagentProvider` for testing the seam through the real load path | (registers on `ctx.subagents`) | diff --git a/packages/compact/compact/README.md b/packages/compact/compact/README.md index 9ef5b73005..642a40e08d 100644 --- a/packages/compact/compact/README.md +++ b/packages/compact/compact/README.md @@ -7,7 +7,7 @@ This package is the interface tier of the compaction capability, split so each c | Package | Role | |---|---| | `@deepseek-ai/dsh-compact` (this) | the interface: abstract service + `compact/*` events + `CompactionResult` | -| `@deepseek-ai/dsh-compact-basic` | a backend: char/4 estimation + token-budget retention + `llm.stream()` summarization | +| `@deepseek-ai/dsh-compact-basic` (deferred) | a backend: char/4 estimation + token-budget retention + `llm.stream()` summarization | | `@deepseek-ai/dsh-tool-compact` (deferred) | the model-facing `/compact` tool over `ctx.compact` | Unlike the bash seam, this interface depends on `@deepseek-ai/dsh-session` and `@deepseek-ai/dsh-llm` — the contract's verbs are defined over a `Session` and its output is the `ContentBlock` vocabulary, so they cannot be expressed without naming those packages. That deviation from the "interface depends only on cordis" guidance is intentional and recorded in the [compaction capability-seam RFC](../../../docs/rfc/proposed/feature/2026-06-18-compaction-capability-seam.md). @@ -53,4 +53,4 @@ The `compact/*` events extend `SessionEventMap` (merge-extensible) via declarati ## Implementing a backend -Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and load the subclass as a plugin — it registers as `ctx.compact`. See `@deepseek-ai/dsh-compact-basic` for the reference implementation. +Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and load the subclass as a plugin — it registers as `ctx.compact`. A tokenizer-, template-, or model-backed implementation can live as a sibling package without changing callers. diff --git a/packages/compact/compact/src/index.ts b/packages/compact/compact/src/index.ts index 9e58e5c905..34b353c790 100644 --- a/packages/compact/compact/src/index.ts +++ b/packages/compact/compact/src/index.ts @@ -6,13 +6,13 @@ * Implementations subclass {@link CompactService}, implement * {@link CompactService.compactIfNeeded} and {@link CompactService.compactRegion}, * and load as a plugin — registering as `ctx.compact` (one implementation per - * context). `@deepseek-ai/dsh-compact-basic` (char/4 estimation + token-budget - * retention + `ctx.llm.stream()` summarization) is the first. A tokenizer- or - * template-based backend swaps in without touching consumers. + * context). A tokenizer-, template-, or model-backed implementation can live + * as a sibling package; callers stay on the same `ctx.compact` seam without + * touching consumers. * * The split follows the capability-seams RFC — interface (this) / - * implementation (`dsh-compact-basic`) / consumer (a `/compact` tool, deferred) - * — modeled on the bash trio. Unlike `dsh-bash`, this interface necessarily + * implementation (deferred) / consumer (a `/compact` tool, deferred) — modeled + * on the bash trio. Unlike `dsh-bash`, this interface necessarily * depends on `dsh-session` and `dsh-llm`: the contract's verbs are defined over * a `Session` and its output is the `ContentBlock` vocabulary. That deviation * from the "interface depends only on cordis" guidance is intentional and diff --git a/packages/core/session/README.md b/packages/core/session/README.md index 2a90d0f792..7989d30b44 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -8,7 +8,7 @@ Creates and holds event-sourced `Session` instances. Persistence is intentionall ### Public API -- `ctx.sessions.create(id?: SessionId, options?: { seed?: SessionEvent[]; meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number } }): Session` — Create a session. `options.seed` replays/forks an existing event log; `options.meta` attaches creation metadata (validated absolute `cwd`, `parentSession` lineage) as the immutable `SessionHeader`. The store fills `version`/`id` and defaults `createdAt` to now; a caller reconstructing a persisted session passes the original `createdAt` to preserve it. Disposed with the calling fiber. +- `ctx.sessions.create(id?: SessionId, options?: { seed?: SessionEvent[]; meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number; seedLength?: number } }): Session` — Create a session. `options.seed` replays/forks an existing event log; `options.meta` attaches creation metadata (validated absolute `cwd`, `parentSession` lineage, seed boundary) as the immutable `SessionHeader`. The store fills `version`/`id` and defaults `createdAt` to now; a caller reconstructing a persisted session passes the original `createdAt` and persisted `seedLength` to preserve them. Disposed with the calling fiber. - `ctx.sessions.get(id: SessionId): Session | undefined` - `ctx.sessions.list(): Session[]` @@ -38,7 +38,7 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. - `session.deriveMessages(): Message[]` — derive the LLM message history by walking the surface linked list (skipping non-surface events like chunks and boundaries; a `replace` shadows the nodes it covers). The surface is the single source of derived history — there is no raw-log fallback. - `session.surface: SurfaceManager` — the derived surface, lazily rebuilt from `surfaceOp` markers in the log. Processes only new events (delta) on each access — the log is append-only, so prior events never change. - `session.events`, `session.seq`, `session.id` -- `session.header: SessionHeader` — immutable creation metadata (`version`, `id`, `createdAt`, optional `cwd`/`parentSession`). Kept out of the event log (a storage concern, not replayable state); a minimal header (stamped with the current `SESSION_FORMAT_VERSION`) is synthesized for bare `Session` construction. +- `session.header: SessionHeader` — immutable creation metadata (`version`, `id`, `createdAt`, optional `cwd`/`parentSession`/`seedLength`). Kept out of the event log (a storage concern, not replayable state); a minimal header (stamped with the current `SESSION_FORMAT_VERSION`) is synthesized for bare `Session` construction. ### Surface types @@ -49,9 +49,9 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. ### Session event vocabulary (`types.ts`) -The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`. Token usage rides on `assistant/message.usage`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. +The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`, `todo/write`. Token usage rides on `assistant/message.usage`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. -Merge-extensible via `SessionEventMap` — a compaction plugin adds `compaction/marker`, etc. +Merge-extensible via `SessionEventMap` — the compaction seam adds `compact/start`, `compact/summary`, and `compact/end`. Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types for typed turn boundaries — `kind`-tagged instead of strings). @@ -62,7 +62,7 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata) ### Metadata types (`types.ts`) -- `SessionHeader` — immutable session metadata, written once: `{ version, id, createdAt, cwd?, parentSession? }`. Owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export it rather than own it (which would force a package cycle). +- `SessionHeader` — immutable session metadata, written once: `{ version, id, createdAt, cwd?, parentSession?, seedLength? }`. Owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export it rather than own it (which would force a package cycle). ### Extension points diff --git a/packages/session-persistence/session-persistence-jsonl/README.md b/packages/session-persistence/session-persistence-jsonl/README.md index b8df12e547..9a76381614 100644 --- a/packages/session-persistence/session-persistence-jsonl/README.md +++ b/packages/session-persistence/session-persistence-jsonl/README.md @@ -10,7 +10,7 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence .jsonl # header line + one SessionEvent per line (verbatim) ``` -- The first `.jsonl` line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession? }`; every subsequent line is one `SessionEvent` JSON, **verbatim including `assistant/chunk`** so `seq` stays contiguous (`events[i].seq === i`). +- The first `.jsonl` line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession?, seedLength? }`; every subsequent line is one `SessionEvent` JSON, **verbatim including `assistant/chunk`** so `seq` stays contiguous (`events[i].seq === i`). - Session ids are unvalidated branded strings, so they are percent-encoded to a single safe path segment before use (no traversal, no collision). ## Config diff --git a/packages/session-persistence/session-persistence/README.md b/packages/session-persistence/session-persistence/README.md index 928e7b033d..8bd3fed568 100644 --- a/packages/session-persistence/session-persistence/README.md +++ b/packages/session-persistence/session-persistence/README.md @@ -2,7 +2,7 @@ The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface. -The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here. +The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage, seed boundary) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here. ## Service API (`ctx.sessionPersistence`) @@ -44,8 +44,8 @@ The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and Import `runPersistenceContract` from `tests/contract.ts` (the public-API contract) and `runCoordinatorContract` from `tests/coordinator-contract.ts` (the shared write-path orchestration: adoption, HMR, collision, dispose-drain, crash-tail repair) and call each with a fixture for your backend. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics AND the same orchestration, so a backend's own spec is left with only storage-mechanics tests (path sanitization, fsync rollback; schema version, transaction rollback) on top. -Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store. +Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data, source_event_seqs, surface_op)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store. ## Metadata types -Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`). +Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`, `seedLength?`). diff --git a/packages/support/README.md b/packages/support/README.md index 52f7f6fe25..942734fc1e 100644 --- a/packages/support/README.md +++ b/packages/support/README.md @@ -7,5 +7,6 @@ Packages that exist to serve development, testing, and the examples rather than | `invariants/` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) | | `ui-stdio/` | Minimal stdio (readline) UI plugin: renders `agent/*` events, feeds stdin lines to the agent | (drives `ctx.agents`) | | `llm-replay/` | Record/replay adapter: short-circuits `llm/stream` from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) | +| `subagent-mock/` | Scripted `SubagentProvider` for deterministic seam/tool tests | (registers on `ctx.subagents`) | -`invariants` runs only in dev mode (contract checks, not runtime behavior). `ui-stdio` and `llm-replay` were extracted from the examples for reuse and to bring them under the per-file coverage gate; they back the demos and the snapshot test tier. A package graduates OUT of `support/` into a product group only when it gains documented product consumers. +`invariants` runs only in dev mode (contract checks, not runtime behavior). `ui-stdio` and `llm-replay` were extracted from the examples for reuse and to bring them under the per-file coverage gate; they back the demos and the snapshot test tier. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers. diff --git a/scripts/verify-md-links.ts b/scripts/verify-md-links.ts index 57bb824b32..65527ab75b 100644 --- a/scripts/verify-md-links.ts +++ b/scripts/verify-md-links.ts @@ -20,12 +20,13 @@ * resolved against the linking file's directory, and the result must exist on * disk. This is checker, not fixer: it reports and never rewrites. * - * Scope is the other doc-sync gates' set plus the two AGENTS.md files AND the - * repo-authored agent-skill Markdown under `.agents/skills/` — those skill - * files cross-link into the docs tree (e.g. the dsh-code-review skill cites the - * RFC index), so a rename must not silently break them either: README.md, - * docs/** /*.md, packages/* /README.md, AGENTS.md, packages/AGENTS.md, - * .agents/skills/** /*.md. The root and packages/ CLAUDE.md are symlinks to the + * Scope is the other doc-sync gates' set plus example Markdown, the two + * AGENTS.md files AND the repo-authored agent-skill Markdown under + * `.agents/skills/` — those skill files cross-link into the docs tree (e.g. the + * dsh-code-review skill cites the RFC index), so a rename must not silently + * break them either: README.md, docs/** /*.md, packages/* /README.md, + * examples/** /*.md, AGENTS.md, packages/AGENTS.md, .agents/skills/** /*.md. + * The root, packages/, and examples/ CLAUDE.md files are symlinks to the * AGENTS.md files, so they are deduped by real path. * * Run: `tsx scripts/verify-md-links.ts`. @@ -42,14 +43,15 @@ import type { Nodes } from 'mdast' const root = resolve(import.meta.dirname, '..') /** - * Files to check: doc-typecheck's scope, the AGENTS.md pair, and repo-authored - * agent-skill Markdown (which this repo's own docs reorg rewrites links in). + * Files to check: doc-typecheck's scope, example Markdown, the AGENTS.md pair, + * and repo-authored agent-skill Markdown. */ const PATTERNS = [ 'README.md', 'docs/**/*.md', 'packages/*/*.md', 'packages/*/*/*.md', + 'examples/**/*.md', 'AGENTS.md', 'packages/AGENTS.md', '.agents/skills/**/*.md',